unRAID Server Release 5.0-beta9 Available


Recommended Posts

  • Replies 157
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Posted Images

I would like to add a vote for including the netatalk latest beta for us AFP folks - the current version is just not very good, rather have the BETA!

 

What is different/better about the latest netatalk beta?  Evidently there's a disconnect issue with TimeMachine in latest 2.2-beta4 which is fixed in version 2.2.0; but NetAFP gave me two options for getting 2.2.0:

a) wait until they "reopen" the whole project ("a few weeks/months") <-- a direct quote

b) pay them 8000 euro (!)

 

c) wait for the big guys to pay up and then a) may come faster.

Link to comment

Is there any chance of letting us set the max volume size for AFP exported volumes?  I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk.

 

Actually, in general an option to set a max size per share would be really useful.

Link to comment

Sounds good! I'm willing to test it out.

 

I've found another potential bug. In maintenance mode, Avahi is still advertising AFP and SMB. Was this the expected behavior?

 

This is by design. "Maintenance mode" is intended to ease certain tasks related to the hard disks.  I still want the Flash device visible, and the server in general, to be visible on the network (so you can use telnet, ssh, etc).  I also intend to let the Flash be exported via AFP as well - I just need to sit down and work out permissions for it, because being FAT formatted, permissions are a little strange.

 

Edit: I should add: I don't want netatalk to be creating the crazy cnid database, etc. for the flash - that's another reason why it's not exported currently via AFP.

Link to comment

Is there any chance of letting us set the max volume size for AFP exported volumes?  I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk.

 

Yes, I'll add that.  Should that apply only to shares exported with "AFP - timemachine" setting?

 

Actually, in general an option to set a max size per share would be really useful.

Hey that's a good idea, I'll look into it.  If you were able to set a limit on a user share size, I guess you would still want the above afp/timemachine limit as well?

Link to comment

Does "Min. Free Space" do what you want? Set your TM share to include only one disk, set the Min. Free Space value to the difference in space that you want it to take up. Select Fill-up as your Allocation method.

 

"Min free space" only applies when creating a new file.  If you extend an existing file (open in append mode), nothing will stop it from growing beyond "Min free space".

Link to comment

Fair enough! I guess one other confusing issue would be that Maintenance mode still allows users to see the Shares tab, and access the "add share" button. However it won't show the shares until you get out of Maintenance mode.

 

AFP does create some crazy files...  ;)

 

The share size limit would be extremely useful in many situations. It should be a default setting! If that is possible, how cool!

Link to comment

I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of:

 

Jul 15 16:29:05 unRAID shfs/user: shfs_setxattr: setxattr: /mnt/cache/TV (95) Operation not supported

 

Nothing else is obvious other than that.

 

I've tried deleting all the .AppleDB files to have the re-created after the upgrade to beta9, and while this worked the first time (after waiting ages for what I assume was them being rebuilt the first time I tried to connect to each share); I've just tried connecting on a different computer in the network and it's really slow again.

 

AFP was never fast in previous versions, but it was better than this. I may have to downgrade to beta8d for the time being.

 

I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard?

Link to comment

I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of:

 

Jul 15 16:29:05 unRAID shfs/user: shfs_setxattr: setxattr: /mnt/cache/TV (95) Operation not supported

 

Nothing else is obvious other than that.

 

I've tried deleting all the .AppleDB files to have the re-created after the upgrade to beta9, and while this worked the first time (after waiting ages for what I assume was them being rebuilt the first time I tried to connect to each share); I've just tried connecting on a different computer in the network and it's really slow again.

 

AFP was never fast in previous versions, but it was better than this. I may have to downgrade to beta8d for the time being.

 

I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard?

 

i am experiencing the same behavior in beta9. after the crawling initial connect, it is running quite alright.

Link to comment

Yes, I agree that it does get better once it finally connects... So far the delay in connecting is about 2 or 3 minutes, and the OSX finder locks up for the duration, so it's pretty annoying.

 

One thing I haven't tried is only connecting to individual disks rather than user shares. I know that there are issues with sharing both via AFP, and given the choice I'd rather share the user shares. Those errors that we're seeing are all related to the user share file system.

Link to comment

Is the security turned off for some reason in beta9? I am able to use anything as a username and no password to access my shares.

 

Edit: I went back to beta8d and same thing.

Edit: I found out that that my new files did not get their permissions changed. I guess anyone is allowed to access the shares with the new files because a couple of files don't have the correct permissions.

syslog.txt

Link to comment

Is the security turned off for some reason in beta9? I am able to use anything as a username and no password to access my shares.

 

Edit: I went back to beta8d and same thing.

 

Everything should work the same. What protocol are you using (SMB/AFP/NFS)?

 

Edit: I found out that that my new files did not get their permissions changed. I guess anyone is allowed to access the shares with the new files because a couple of files don't have the correct permissions.

How did you determine this?

 

 

Link to comment

SMB

I went to the other shares where I did not add new files and it asks for a password. I just checked again, and one share that I did not add any files to is accessible without proper login. Should I run new permissions script again? Should I delete passwd, shadow, and smbpasswd again?

 

You are not giving me any information that I can use to diagnose what might be happening.

 

Is this all taking place on the same PC?  What OS (WinXP/Win7/etc)?

What is the "Security" set to for your shares (Public/Secure/Private)?

After having first run 'new permissions' in the past, have you manually messed with the permission bits (like run a 'chmod' command for some reason)?

 

On the webGui navigate to "Utils/Development Utilities/Vars".  Select all the text that shows up with a grey background, copy to a file and either upload here or send to me pleaes: tomm@lime-technology.com

Link to comment

Same PC. Windows 7 64-bit

 

Some Private, some Secure, none Public

 

I use this command to change the permissions of the cache drive so I can delete or edit files. I download to the cache drive.

newperms /mnt/cache

 

Also this command to stop pyLoad so it doesn't get hung on unmounting disks when trying to stop

chmod -R 0755 /mnt/cache/.pyLoad

Link to comment

Same PC. Windows 7 64-bit

 

Some Private, some Secure, none Public

 

I use this command to change the permissions of the cache drive so I can delete or edit files. I download to the cache drive.

newperms /mnt/cache

 

Also this command to stop pyLoad so it doesn't get hung on unmounting disks when trying to stop

chmod -R 0755 /mnt/cache/.pyLoad

 

What is 'pyLoad'?  Is this the application that's creating files somewhere?

 

I think the problem here is that you are using an add-on application that is breaking the security model.  But I need to understand what's happening in order to suggest a fix.

Link to comment

Is there any chance of letting us set the max volume size for AFP exported volumes?  I don't want TM filling up my whole file server, and it now auto-resizes sparse bundles to the same size as available disk.

 

Yes, I'll add that.  Should that apply only to shares exported with "AFP - timemachine" setting?

 

Actually, in general an option to set a max size per share would be really useful.

Hey that's a good idea, I'll look into it.  If you were able to set a limit on a user share size, I guess you would still want the above afp/timemachine limit as well?

 

To be honest, if you had the ability to limit user share sizes then the AFP or timemachine specific setting would not be necessary.  For example, I would set my mythtv export to 3TB, my time machine share (where all my apple machines back up to) to 2TB, and have the rest of my user shares (videos/music/etc) set to unlimited.  If the user share on the virtual mount shows up as 2TB, that's all that AFP should report back to time machine, so no special config necessary.  I could be wrong, but if I'm not then it's a single and simple setting per share!

 

 

Edited to add:

if sizes for user shares is out of scope for 5.0 then it would be very helpful if you could do the AFP/TM specific one, then add user share sizing for 5.1.  That said, since you're using FUSE to mount user shares it shouldn't be too difficult to do, just depends on your amount of  time :)

Link to comment

I am no unable to shutdown my unRaid server, I used the unmenu user script shutdown server and as soon as I done that I could see on the console some kind of fsraiser error message and even after 8 hours it still has not shutdown. This started to happen after upgrading to beta8d and still happens with beta9. I have to press the power button to shut it down. Any ideas? Via the console I still get the login screen so I can issues commands, just cannot get into the web interace anymore and smb is also not working. Really don't want to press the power button for 5 sec again.

syslog-2011-07-15.txt

Link to comment

I am no unable to shutdown my unRaid server, I used the unmenu user script shutdown server and as soon as I done that I could see on the console some kind of fsraiser error message and even after 8 hours it still has not shutdown. This started to happen after upgrading to beta8d and still happens with beta9. I have to press the power button to shut it down. Any ideas? Via the console I still get the login screen so I can issues commands, just cannot get into the web interace anymore and smb is also not working. Really don't want to press the power button for 5 sec again.

 

Others have reported this bug and I'm trying to track it down.  Your system log is from -beta8d, would be nice to see log if you can make it happen again with -beta9.

Link to comment

I copied the current syslog to the /boot as per wikie. However when I try to press sync it just flashes the cursor and is hanging there fore along time. By the way because I'm still copying large amounts of data I don't have a parity drive right now, does that mean it's save to hard power down the server? Now power cycled the unit and it came up normally, but I really dont want to continue to do this. Here is the current syslog.

syslog-16072011.zip

Link to comment

I have so far found AFP to be painfully slow at initially connecting in beta9, the logs only show a lot of:

I'm surprised that I'm the only one seeing this. Really no-one else having performance issues with beta9 / Snow Leopard?

 

The initial connection is slow, but once established, I have no problems in transfer speeds (to cache drive not the array): AFP 73MB/s Samba 44 MB/s (Os X 10.6.8, using 5.0-B9 and a disk image with a size of 512MB for testing the transfer speed.). If I transfer to the array, then the transfer speed slows down to 34MB/s to both AFP and SMB as was the case with the earlier unRaid versions.

Link to comment

OK, I've spent quite a lot of time today playing with Netatalk, and trying to understand exactly how it works... this is what I've learnt so far.

 

Looking at this quote from the Netatalk 2.1 help pages:

 

Unlike other protocols like smb or nfs, the AFP protocol mostly refers to files and directories by ID and not by a path (the IDs are also called CNID, that means Catalog Node ID). A typical AFP request uses a directory ID and a filename, something like "server, please open the file named 'Test' in the directory with id 167". For example "Aliases" on the Mac basically work by ID (with a fallback to the absolute path in more recent AFP clients. But this applies only to Finder, not to applications).

 

Every file in an AFP volume has to have a unique file ID, IDs must, according to the specs, never be reused, and IDs are 32 bit numbers (Directory IDs use the same ID pool). So, after ~4 billion files/folders have been written to an AFP volume, the ID pool is depleted and no new file can be written to the volume. No whining please :-)

 

Netatalk needs to map IDs to files and folders in the host filesystem. To achieve this, several different CNID backends are available and can be choosed by the cnidscheme option in the AppleVolumes.default(5) configuration file. A CNID backend is basically a database storing ID <-> name mappings.

 

The CNID Databases are by default located in the .AppleDB folder in every afpd volume root. With the new ADv2 format, afpd stores the files/directories ID in the corresponding .AppleDouble file as well.

 

And the warning below it:

 

There are some CNID related things you should keep in mind when working with netatalk:

 

  • Don't nest volumes.
  • CNID backends are databases, so they turn afpd into a file server/database mix. Keep this in mind, killing an afpd process with kill -9 will likely leave the database unusable.
  • If there's no more space on the filesystem left, the database will get corrupted. You can work around this by either using the -dbpath option and put the database files into another location or, if you use quotas, make sure the .AppleDB folder is owned by a user/group without a quota.

 

It seems that browsing a directory tree on an AFP share is not really browsing a tree structure at all... each file and folder has a unique ID, which needs to be created by Netatalk when you browse a share. My testing has shown that the database is actually only created as you move through a share, with a new .AppleDouble file being created for every subdirectory of a directory that you enter.

 

If like me, you have a Movie / TV Show / Music directory with a lot of subdirectories, there is a very long delay while each folder and file is indexed by the CNID database daemon. Once it's done, browsing folders is very fast, as we're just looking at the database for the structure. I actually found out that if you mount each of your shares, and then run a

find .

command in the OS X Terminal, you can build this CNID database in one go, rather than have it build as you browse.

 

I figured I may have some corruption in the CNID database, so I decided to rebuild it from scratch. In order to do this, I stopped netatalk by running

/etc/rc.d/rc.atalk stop

and then deleted all the .AppleDB and .AppleDouble files on all of the disks in the array by running

find disk*/ -name .AppleD* -exec rm -r {} \;

from the /mnt directory.

 

After I'd done this I was able to restart netatalk, and then browse around to rebuild the database.

 

This all sounds good so far, but I discovered other issues...

 

On re-booting, the CNID database seems to be re-created, so connecting and browsing becomes slow again.

 

Occasionally I've got errors about the CNID database being corrupt after rebooting, even after I'd completely re-created them in the previous boot.

 

I wondered if there might be some issues creating these .AppleDouble and .AppleDB files on the fuser file system, and had read about there being a way to move the AppleDB folders away to a different location, and prevent the CNID data from being cached in the .AppleDouble folders. To do this, you need to edit the /etc/netatalk/AppleVolumes.default- file (make sure you edit the file with a -, as it's used by unRAID each time you make a change to your shares configuration). At the bottom of the file is a line that lists the defaults for AFP Shares, where you can add the option 'nocnidcache' and 'dbpath:'.

 

First I created a subdirectory in /var to hold these databases in the RAM disk. The line in my AppleVolumes.default- file then looked like this:

 

:DEFAULT: cnidscheme:dbd options:upriv,usedots,nocnidcache dbpath:/var/dbd/$v

 

I stopped netatalk, removed all the CNID files using the process above, the restarted everything.

 

Without .AppleDouble directories being created, the initial browsing of folder is much much quicker. The downside to putting the .AppleDB folders in /var, is that they disappear on reboot. This may not be an issue if they really are being re-generated on each boot anyway of course.

 

I've also tried having the dbpath for the .AppleDB files on my USB disk that I've been using for Plex and Slimserver, though I'm still having issues with Kernel Oops: when I try to reboot with this drive in place.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.