speeding_ant Posted July 14, 2011 Share Posted July 14, 2011 Sounds good! I'm willing to test it out. I've found another potential bug. In maintenance mode, Avahi is still advertising AFP and SMB. Was this the expected behavior? Quote Link to comment
madburg Posted July 14, 2011 Share Posted July 14, 2011 I would like to add a vote for including the netatalk latest beta for us AFP folks - the current version is just not very good, rather have the BETA! What is different/better about the latest netatalk beta? Evidently there's a disconnect issue with TimeMachine in latest 2.2-beta4 which is fixed in version 2.2.0; but NetAFP gave me two options for getting 2.2.0: a) wait until they "reopen" the whole project ("a few weeks/months") <-- a direct quote b) pay them 8000 euro (!) c) wait for the big guys to pay up and then a) may come faster. Quote Link to comment
bbqninja Posted July 14, 2011 Share Posted July 14, 2011 Is there any chance of letting us set the max volume size for AFP exported volumes? I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk. Actually, in general an option to set a max size per share would be really useful. Quote Link to comment
speeding_ant Posted July 14, 2011 Share Posted July 14, 2011 Does "Min. Free Space" do what you want? Set your TM share to include only one disk, set the Min. Free Space value to the difference in space that you want it to take up. Select Fill-up as your Allocation method. Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 Sounds good! I'm willing to test it out. I've found another potential bug. In maintenance mode, Avahi is still advertising AFP and SMB. Was this the expected behavior? This is by design. "Maintenance mode" is intended to ease certain tasks related to the hard disks. I still want the Flash device visible, and the server in general, to be visible on the network (so you can use telnet, ssh, etc). I also intend to let the Flash be exported via AFP as well - I just need to sit down and work out permissions for it, because being FAT formatted, permissions are a little strange. Edit: I should add: I don't want netatalk to be creating the crazy cnid database, etc. for the flash - that's another reason why it's not exported currently via AFP. Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 Is there any chance of letting us set the max volume size for AFP exported volumes? I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk. Yes, I'll add that. Should that apply only to shares exported with "AFP - timemachine" setting? Actually, in general an option to set a max size per share would be really useful. Hey that's a good idea, I'll look into it. If you were able to set a limit on a user share size, I guess you would still want the above afp/timemachine limit as well? Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 Does "Min. Free Space" do what you want? Set your TM share to include only one disk, set the Min. Free Space value to the difference in space that you want it to take up. Select Fill-up as your Allocation method. "Min free space" only applies when creating a new file. If you extend an existing file (open in append mode), nothing will stop it from growing beyond "Min free space". Quote Link to comment
speeding_ant Posted July 15, 2011 Share Posted July 15, 2011 Fair enough! I guess one other confusing issue would be that Maintenance mode still allows users to see the Shares tab, and access the "add share" button. However it won't show the shares until you get out of Maintenance mode. AFP does create some crazy files... The share size limit would be extremely useful in many situations. It should be a default setting! If that is possible, how cool! Quote Link to comment
Nezil Posted July 15, 2011 Share Posted July 15, 2011 I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of: Jul 15 16:29:05 unRAID shfs/user: shfs_setxattr: setxattr: /mnt/cache/TV (95) Operation not supported Nothing else is obvious other than that. I've tried deleting all the .AppleDB files to have the re-created after the upgrade to beta9, and while this worked the first time (after waiting ages for what I assume was them being rebuilt the first time I tried to connect to each share); I've just tried connecting on a different computer in the network and it's really slow again. AFP was never fast in previous versions, but it was better than this. I may have to downgrade to beta8d for the time being. I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard? Quote Link to comment
muzo178 Posted July 15, 2011 Share Posted July 15, 2011 I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of: Jul 15 16:29:05 unRAID shfs/user: shfs_setxattr: setxattr: /mnt/cache/TV (95) Operation not supported Nothing else is obvious other than that. I've tried deleting all the .AppleDB files to have the re-created after the upgrade to beta9, and while this worked the first time (after waiting ages for what I assume was them being rebuilt the first time I tried to connect to each share); I've just tried connecting on a different computer in the network and it's really slow again. AFP was never fast in previous versions, but it was better than this. I may have to downgrade to beta8d for the time being. I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard? i am experiencing the same behavior in beta9. after the crawling initial connect, it is running quite alright. Quote Link to comment
Nezil Posted July 15, 2011 Share Posted July 15, 2011 Yes, I agree that it does get better once it finally connects... So far the delay in connecting is about 2 or 3 minutes, and the OSX finder locks up for the duration, so it's pretty annoying. One thing I haven't tried is only connecting to individual disks rather than user shares. I know that there are issues with sharing both via AFP, and given the choice I'd rather share the user shares. Those errors that we're seeing are all related to the user share file system. Quote Link to comment
AwesomeAustn Posted July 15, 2011 Share Posted July 15, 2011 Is the security turned off for some reason in beta9? I am able to use anything as a username and no password to access my shares. Edit: I went back to beta8d and same thing. Edit: I found out that that my new files did not get their permissions changed. I guess anyone is allowed to access the shares with the new files because a couple of files don't have the correct permissions. syslog.txt Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 Is the security turned off for some reason in beta9? I am able to use anything as a username and no password to access my shares. Edit: I went back to beta8d and same thing. Everything should work the same. What protocol are you using (SMB/AFP/NFS)? Edit: I found out that that my new files did not get their permissions changed. I guess anyone is allowed to access the shares with the new files because a couple of files don't have the correct permissions. How did you determine this? Quote Link to comment
AwesomeAustn Posted July 15, 2011 Share Posted July 15, 2011 SMB I went to the other shares where I did not add new files and it asks for a password. I just checked again, and one share that I did not add any files to is accessible without proper login. Should I run new permissions script again? Should I delete passwd, shadow, and smbpasswd again? Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 SMB I went to the other shares where I did not add new files and it asks for a password. I just checked again, and one share that I did not add any files to is accessible without proper login. Should I run new permissions script again? Should I delete passwd, shadow, and smbpasswd again? You are not giving me any information that I can use to diagnose what might be happening. Is this all taking place on the same PC? What OS (WinXP/Win7/etc)? What is the "Security" set to for your shares (Public/Secure/Private)? After having first run 'new permissions' in the past, have you manually messed with the permission bits (like run a 'chmod' command for some reason)? On the webGui navigate to "Utils/Development Utilities/Vars". Select all the text that shows up with a grey background, copy to a file and either upload here or send to me pleaes: [email protected] Quote Link to comment
AwesomeAustn Posted July 15, 2011 Share Posted July 15, 2011 Same PC. Windows 7 64-bit Some Private, some Secure, none Public I use this command to change the permissions of the cache drive so I can delete or edit files. I download to the cache drive. newperms /mnt/cache Also this command to stop pyLoad so it doesn't get hung on unmounting disks when trying to stop chmod -R 0755 /mnt/cache/.pyLoad Quote Link to comment
limetech Posted July 15, 2011 Author Share Posted July 15, 2011 Same PC. Windows 7 64-bit Some Private, some Secure, none Public I use this command to change the permissions of the cache drive so I can delete or edit files. I download to the cache drive. newperms /mnt/cache Also this command to stop pyLoad so it doesn't get hung on unmounting disks when trying to stop chmod -R 0755 /mnt/cache/.pyLoad What is 'pyLoad'? Is this the application that's creating files somewhere? I think the problem here is that you are using an add-on application that is breaking the security model. But I need to understand what's happening in order to suggest a fix. Quote Link to comment
AwesomeAustn Posted July 15, 2011 Share Posted July 15, 2011 PyLoad is a download manager. Found it here http://lime-technology.com/forum/index.php?topic=10883.0. Quote Link to comment
squirrellydw Posted July 16, 2011 Share Posted July 16, 2011 My drives are not spinning down. Any idea, syslog attched syslog-2011-07-15.txt Quote Link to comment
bbqninja Posted July 16, 2011 Share Posted July 16, 2011 Is there any chance of letting us set the max volume size for AFP exported volumes? I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk. Yes, I'll add that. Should that apply only to shares exported with "AFP - timemachine" setting? Actually, in general an option to set a max size per share would be really useful. Hey that's a good idea, I'll look into it. If you were able to set a limit on a user share size, I guess you would still want the above afp/timemachine limit as well? To be honest, if you had the ability to limit user share sizes then the AFP or timemachine specific setting would not be necessary. For example, I would set my mythtv export to 3TB, my time machine share (where all my apple machines back up to) to 2TB, and have the rest of my user shares (videos/music/etc) set to unlimited. If the user share on the virtual mount shows up as 2TB, that's all that AFP should report back to time machine, so no special config necessary. I could be wrong, but if I'm not then it's a single and simple setting per share! Edited to add: if sizes for user shares is out of scope for 5.0 then it would be very helpful if you could do the AFP/TM specific one, then add user share sizing for 5.1. That said, since you're using FUSE to mount user shares it shouldn't be too difficult to do, just depends on your amount of time Quote Link to comment
MrLondon Posted July 16, 2011 Share Posted July 16, 2011 I am no unable to shutdown my unRaid server, I used the unmenu user script shutdown server and as soon as I done that I could see on the console some kind of fsraiser error message and even after 8 hours it still has not shutdown. This started to happen after upgrading to beta8d and still happens with beta9. I have to press the power button to shut it down. Any ideas? Via the console I still get the login screen so I can issues commands, just cannot get into the web interace anymore and smb is also not working. Really don't want to press the power button for 5 sec again. syslog-2011-07-15.txt Quote Link to comment
limetech Posted July 16, 2011 Author Share Posted July 16, 2011 I am no unable to shutdown my unRaid server, I used the unmenu user script shutdown server and as soon as I done that I could see on the console some kind of fsraiser error message and even after 8 hours it still has not shutdown. This started to happen after upgrading to beta8d and still happens with beta9. I have to press the power button to shut it down. Any ideas? Via the console I still get the login screen so I can issues commands, just cannot get into the web interace anymore and smb is also not working. Really don't want to press the power button for 5 sec again. Others have reported this bug and I'm trying to track it down. Your system log is from -beta8d, would be nice to see log if you can make it happen again with -beta9. Quote Link to comment
MrLondon Posted July 16, 2011 Share Posted July 16, 2011 I copied the current syslog to the /boot as per wikie. However when I try to press sync it just flashes the cursor and is hanging there fore along time. By the way because I'm still copying large amounts of data I don't have a parity drive right now, does that mean it's save to hard power down the server? Now power cycled the unit and it came up normally, but I really dont want to continue to do this. Here is the current syslog. syslog-16072011.zip Quote Link to comment
aht961 Posted July 16, 2011 Share Posted July 16, 2011 I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of: I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard? The initial connection is slow, but once established, I have no problems in transfer speeds (to cache drive not the array): AFP 73MB/s Samba 44 MB/s (Os X 10.6.8, using 5.0-B9 and a disk image with a size of 512MB for testing the transfer speed.). If I transfer to the array, then the transfer speed slows down to 34MB/s to both AFP and SMB as was the case with the earlier unRaid versions. Quote Link to comment
Nezil Posted July 16, 2011 Share Posted July 16, 2011 OK, I've spent quite a lot of time today playing with Netatalk, and trying to understand exactly how it works... this is what I've learnt so far. Looking at this quote from the Netatalk 2.1 help pages: Unlike other protocols like smb or nfs, the AFP protocol mostly refers to files and directories by ID and not by a path (the IDs are also called CNID, that means Catalog Node ID). A typical AFP request uses a directory ID and a filename, something like "server, please open the file named 'Test' in the directory with id 167". For example "Aliases" on the Mac basically work by ID (with a fallback to the absolute path in more recent AFP clients. But this applies only to Finder, not to applications). Every file in an AFP volume has to have a unique file ID, IDs must, according to the specs, never be reused, and IDs are 32 bit numbers (Directory IDs use the same ID pool). So, after ~4 billion files/folders have been written to an AFP volume, the ID pool is depleted and no new file can be written to the volume. No whining please :-) Netatalk needs to map IDs to files and folders in the host filesystem. To achieve this, several different CNID backends are available and can be choosed by the cnidscheme option in the AppleVolumes.default(5) configuration file. A CNID backend is basically a database storing ID <-> name mappings. The CNID Databases are by default located in the .AppleDB folder in every afpd volume root. With the new ADv2 format, afpd stores the files/directories ID in the corresponding .AppleDouble file as well. And the warning below it: There are some CNID related things you should keep in mind when working with netatalk: Don't nest volumes. CNID backends are databases, so they turn afpd into a file server/database mix. Keep this in mind, killing an afpd process with kill -9 will likely leave the database unusable. If there's no more space on the filesystem left, the database will get corrupted. You can work around this by either using the -dbpath option and put the database files into another location or, if you use quotas, make sure the .AppleDB folder is owned by a user/group without a quota. It seems that browsing a directory tree on an AFP share is not really browsing a tree structure at all... each file and folder has a unique ID, which needs to be created by Netatalk when you browse a share. My testing has shown that the database is actually only created as you move through a share, with a new .AppleDouble file being created for every subdirectory of a directory that you enter. If like me, you have a Movie / TV Show / Music directory with a lot of subdirectories, there is a very long delay while each folder and file is indexed by the CNID database daemon. Once it's done, browsing folders is very fast, as we're just looking at the database for the structure. I actually found out that if you mount each of your shares, and then run a find . command in the OS X Terminal, you can build this CNID database in one go, rather than have it build as you browse. I figured I may have some corruption in the CNID database, so I decided to rebuild it from scratch. In order to do this, I stopped netatalk by running /etc/rc.d/rc.atalk stop and then deleted all the .AppleDB and .AppleDouble files on all of the disks in the array by running find disk*/ -name .AppleD* -exec rm -r {} \; from the /mnt directory. After I'd done this I was able to restart netatalk, and then browse around to rebuild the database. This all sounds good so far, but I discovered other issues... On re-booting, the CNID database seems to be re-created, so connecting and browsing becomes slow again. Occasionally I've got errors about the CNID database being corrupt after rebooting, even after I'd completely re-created them in the previous boot. I wondered if there might be some issues creating these .AppleDouble and .AppleDB files on the fuser file system, and had read about there being a way to move the AppleDB folders away to a different location, and prevent the CNID data from being cached in the .AppleDouble folders. To do this, you need to edit the /etc/netatalk/AppleVolumes.default- file (make sure you edit the file with a -, as it's used by unRAID each time you make a change to your shares configuration). At the bottom of the file is a line that lists the defaults for AFP Shares, where you can add the option 'nocnidcache' and 'dbpath:'. First I created a subdirectory in /var to hold these databases in the RAM disk. The line in my AppleVolumes.default- file then looked like this: :DEFAULT: cnidscheme:dbd options:upriv,usedots,nocnidcache dbpath:/var/dbd/$v I stopped netatalk, removed all the CNID files using the process above, the restarted everything. Without .AppleDouble directories being created, the initial browsing of folder is much much quicker. The downside to putting the .AppleDB folders in /var, is that they disappear on reboot. This may not be an issue if they really are being re-generated on each boot anyway of course. I've also tried having the dbpath for the .AppleDB files on my USB disk that I've been using for Plex and Slimserver, though I'm still having issues with Kernel Oops: when I try to reboot with this drive in place. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.