January 6, 201115 yr I think the resident AFP experts are starcat and dlmh (forgive me if you are an expert not listed here). I'm almost ready to release -beta3 and I have a few questions regarding netatalk... 1. Time Machine support - this is enabled for a share simply by specifying 'tm' as one of the options in AppleVolumes.default. Is there any reason not to enable this option for all shares? If not, then it needs to be a user-configurable share option, correct? 2. I want to use the 'dbd' CNID backend scheme exclusively because that's the new default in netatalk 2.1. Is there any reason a user would want to configure 'cdb' instead? 3. To sticky or not to sticky, that is the question... all the documentation says you need to set the sticky bit on directories so that files created in a directory are always set to the same group as the directory... but is this really necessary? 4. Initial release is based on netatalk 2.0.5, is there any compelling reason to move to 2.1? Question regarding avahi... 5. If you have both afp and samba services defined, if you see the machine icon in Finder, how do you know what protocol is actually being used (ie, whether it's using SMB or AFP)?
January 6, 201115 yr I think the resident AFP experts are starcat and dlmh (forgive me if you are an expert not listed here). I don't profess to be an expert but I should be able to answer some of the questions below I'm almost ready to release -beta3 and I have a few questions regarding netatalk... 1. Time Machine support - this is enabled for a share simply by specifying 'tm' as one of the options in AppleVolumes.default. Is there any reason not to enable this option for all shares? The only reason I can think of is if you do not want someone to use it as a TimeMachine capable spot to backup to. I would prefer to specify which shares are TimeMachine capable and which are not. By NOT setting 'tm' by default it would avoid my room mate, sister, parents, etc from seeing the "Backup" share and assuming that is the share to use for the TimeMachine backup. If not, then it needs to be a user-configurable share option, correct? Yes, that is correct. Make it an opt-in option just like you do with the cache drive stuff now. 2. I want to use the 'dbd' CNID backend scheme exclusively because that's the new default in netatalk 2.1. Is there any reason a user would want to configure 'cdb' instead? From my quick reading and glancing I don't see any reason not to use 'dbd' by default, especially if you upgrade to 2.1. 3. To sticky or not to sticky, that is the question... all the documentation says you need to set the sticky bit on directories so that files created in a directory are always set to the same group as the directory... but is this really necessary? Not sure... Again a quick read and Wiki of it seems to make me think that when set it allows ONLY the owner that wrote the files to change them. So, if a user was set up on the share as having read/write access but they are not the ones that wrote the files they WOULD NOT be able to delete/change them... if I read the wiki article correctly. Would an unRAID setting override the sticky bit? I can see an instance were one users writes a file to the server (a Word document) and another user with read/write access to the share wants to modify it. If I read correctly and understand it then the sticky bit would not allow this to happen as the one trying to modify the file was not the one who created it. For something like unRAID I would probably leave the sticky bit OFF. If you are feeling gracious you could add it as an option in the shares section (I could see instances were a person would want to give write access to a share but not allow a person to modify a file already there). 4. Initial release is based on netatalk 2.0.5, is there any compelling reason to move to 2.1? I did see this: "fix a serious error in networking IO code" as one of the fixes in the 2.1.3 release. There are numerous other fixes but that one stuck out to me in my quick glances at all the changlogs. Question regarding avahi... 5. If you have both afp and samba services defined, if you see the machine icon in Finder, how do you know what protocol is actually being used (ie, whether it's using SMB or AFP)? That, I do not know. The machine icon might look a little different and it might show up as a different 'Kind.' My smb exports from my server running 5.0b2 show up with a 'Kind' of "Sharepoint."
January 6, 201115 yr 1) I would allow all shares to be TM capable and be listed as possible TM targets on Mac clients. Let the user decide on the client where to put the backup without messing up with the server. This is the more logical and easier way to do it. 2) I would also use dbd as the default. 3) Just checked what Qnap is doing and they use indeed +s in the group (or user depending on configuration). The Apple implementation for TM is to have the sparsebundles for all backup clients on a single share and thus the sticky bit. Would make this the default and allow to be modified (if necessary at all). 4) I would prefer to go to the latest stable if possible, mainly because of some bug fixes. 5) There is a way to have a setting and depending on that the Finder would show an Xserve icon next to the computer in the left Finder pane. I suggest setting that if the mount is AFP, and without to get the regular icon if the exports are of SMB type. I may post details regarding to how to do this.
January 6, 201115 yr I think the resident AFP experts are starcat and dlmh (forgive me if you are an expert not listed here). I'm almost ready to release -beta3 and I have a few questions regarding netatalk... 1. Time Machine support - this is enabled for a share simply by specifying 'tm' as one of the options in AppleVolumes.default. Is there any reason not to enable this option for all shares? If not, then it needs to be a user-configurable share option, correct? As far as I know, there's no drawback of using this flag for all your shares, but you might not want to be able to, as prostuff1 pointed out. Leaving it as an option for the user is always the best. 2. I want to use the 'dbd' CNID backend scheme exclusively because that's the new default in netatalk 2.1. Is there any reason a user would want to configure 'cdb' instead? The Netatalk FAQ mentions something about slower update speed and retrieval, but I've tried both and never saw any difference. 3. To sticky or not to sticky, that is the question... all the documentation says you need to set the sticky bit on directories so that files created in a directory are always set to the same group as the directory... but is this really necessary? This WAS necessary because of the way unRAID treated file permissions before 5.0 (root/root and not [user]/user), but since you've moved to a different user system, I don't think this is really necessary anymore. 4. Initial release is based on netatalk 2.0.5, is there any compelling reason to move to 2.1? Unless you're willing to move to the 2.2 alpha series, there's no real reason to upgrade, although 2.1.5 is the latest stable version. The 2.2 series brings loads of speed improvements, so it might be useful to upgrade when it reaches stable. Question regarding avahi... 5. If you have both afp and samba services defined, if you see the machine icon in Finder, how do you know what protocol is actually being used (ie, whether it's using SMB or AFP)? This has been a PITA, but OSX normally doesn't actually announce SMB over Bonjour/zeroconf if there's an AFP service available. If you use the same service name for AFP and SMB, you will probably get an error somewhere from the Avahi daemon. To solve this, I added the ".SMB" (or ".AFP" or you like) to the service name. (%h.SMB). This way I can choose whatever protocol I want to use.
January 6, 201115 yr 5. If you have both afp and samba services defined, if you see the machine icon in Finder, how do you know what protocol is actually being used (ie, whether it's using SMB or AFP)? This has been a PITA, but OSX normally doesn't actually announce SMB over Bonjour/zeroconf if there's an AFP service available. If you use the same service name for AFP and SMB, you will probably get an error somewhere from the Avahi daemon. To solve this, I added the ".SMB" (or ".AFP" or you like) to the service name. (%h.SMB). This way I can choose whatever protocol I want to use. Yeah... I actually can't think of any reason why someone on a Mac would use SMB? If they are both available for an unRaid share, a Mac would pick up AFP and all other windows clients will stay with SMB.
January 6, 201115 yr An Xserve icons shows in the finder when <txt-record>model=Xserve</txt-record> is configured in /etc/avahi/services/samba.service
January 7, 201115 yr Author 1) I would allow all shares to be TM capable and be listed as possible TM targets on Mac clients. Let the user decide on the client where to put the backup without messing up with the server. This is the more logical and easier way to do it. So you are saying that the 'tm' option should be set for all share exports defined in AppleVolumes.default, and that there is no need to have a per-share config variable for this?
January 7, 201115 yr Author Unless you're willing to move to the 2.2 alpha series, there's no real reason to upgrade, although 2.1.5 is the latest stable version. The 2.2 series brings loads of speed improvements, so it might be useful to upgrade when it reaches stable. netatalk 2.1 requires a newer version of Berkeley DB that what's available for Slack, so I would have to create the package myself, which could introduce problems if I don't specify the right set of config options out of ignorance So I will be releasing 5.0-beta3 with netatalk 2.05 to work out initial issues, and then schedule netatalk 2.1 (or 2.2) for unRaid 5.1.
January 7, 201115 yr 1) I would allow all shares to be TM capable and be listed as possible TM targets on Mac clients. Let the user decide on the client where to put the backup without messing up with the server. This is the more logical and easier way to do it. So you are saying that the 'tm' option should be set for all share exports defined in AppleVolumes.default, and that there is no need to have a per-share config variable for this? Basically, yes. Should be the default at least. When turning on TimeMachine on the Mac, the user is asked to select a volume from all that are available to TM. TM can't use more that one volume at a time and during config always ask for selection. Allowing for removal of "tm" on a per share basis somewhere in the unRAID settings is an added goodie of course for more advanced users.
January 7, 201115 yr Author In case it's not obvious, I don't have a great deal of experience working with OS X but I'm coming up to speed very quickly. However the only Mac I have access to at the moment is OS X 10.5.8 which is "Leopard" I guess. So if my testing works ok with Leopard, are there any issues for later revisions of OS X? Another question: in the Windows/Samba world, there is still 'netbios' available for device discovery on the local subnet. This is handy so that you can type the name of your server in a browser address bar and get to the webGui, for example you can type "//tower". On a Mac however, it doesn't use netbios so you have to find and type the IP address of the server into the browser address bar. What is the correct solution to this so user can type server name instead?
January 7, 201115 yr In case it's not obvious, I don't have a great deal of experience working with OS X but I'm coming up to speed very quickly. However the only Mac I have access to at the moment is OS X 10.5.8 which is "Leopard" I guess. So if my testing works ok with Leopard, are there any issues for later revisions of OS X? Not that I have noticed, nothing much changed on the network side with Leopard/Snow Leopard. Another question: in the Windows/Samba world, there is still 'netbios' available for device discovery on the local subnet. This is handy so that you can type the name of your server in a browser address bar and get to the webGui, for example you can type "//tower". On a Mac however, it doesn't use netbios so you have to find and type the IP address of the server into the browser address bar. What is the correct solution to this so user can type server name instead? With Avahi installed, you can type http://tower.local to access the unraid machine by name (if the user didn't change the name).
January 7, 201115 yr In case it's not obvious, I don't have a great deal of experience working with OS X but I'm coming up to speed very quickly. However the only Mac I have access to at the moment is OS X 10.5.8 which is "Leopard" I guess. So if my testing works ok with Leopard, are there any issues for later revisions of OS X? Another question: in the Windows/Samba world, there is still 'netbios' available for device discovery on the local subnet. This is handy so that you can type the name of your server in a browser address bar and get to the webGui, for example you can type "//tower". On a Mac however, it doesn't use netbios so you have to find and type the IP address of the server into the browser address bar. What is the correct solution to this so user can type server name instead? The thing we have been telling mac users to do is add an entry to the hosts file on there Mac. This works but if you have a DHCP address for the server it could get screwed up. I just ran across this post over on Mac OS X Hints.
January 7, 201115 yr Apple uses Bonjour which is their own implementation of zeroconf (service discovery protocol), and Avahi is the opensource side of the things. So, having Avahi running on unRAID any Mac is able to discover the name. "tower.local" would be the default as .local is the top level domain used in multicast DNS of zeroconf for an internal organization (also recommended by Microsoft in their AD), however I think that plain "tower" should work also as the domainname isn't necessary in that case.
January 8, 201115 yr Author Apple uses Bonjour which is their own implementation of zeroconf (service discovery protocol), and Avahi is the opensource side of the things. So, having Avahi running on unRAID any Mac is able to discover the name. "tower.local" would be the default as .local is the top level domain used in multicast DNS of zeroconf for an internal organization (also recommended by Microsoft in their AD), however I think that plain "tower" should work also as the domainname isn't necessary in that case. That was the missing piece of the puzzle! First cut at AFP support seems to be working pretty well now. Just a few more tasks to take care of before I can release -beta3..
January 10, 201115 yr Apple uses Bonjour which is their own implementation of zeroconf (service discovery protocol), and Avahi is the opensource side of the things. So, having Avahi running on unRAID any Mac is able to discover the name. "tower.local" would be the default as .local is the top level domain used in multicast DNS of zeroconf for an internal organization (also recommended by Microsoft in their AD), however I think that plain "tower" should work also as the domainname isn't necessary in that case. That was the missing piece of the puzzle! First cut at AFP support seems to be working pretty well now. Just a few more tasks to take care of before I can release -beta3.. How did you solve the issue with the afpd files created on the same location on different disks? You know, those files that will start cluttering your log file with warnings from the shfs deamon.
January 10, 201115 yr Author Apple uses Bonjour which is their own implementation of zeroconf (service discovery protocol), and Avahi is the opensource side of the things. So, having Avahi running on unRAID any Mac is able to discover the name. "tower.local" would be the default as .local is the top level domain used in multicast DNS of zeroconf for an internal organization (also recommended by Microsoft in their AD), however I think that plain "tower" should work also as the domainname isn't necessary in that case. That was the missing piece of the puzzle! First cut at AFP support seems to be working pretty well now. Just a few more tasks to take care of before I can release -beta3.. How did you solve the issue with the afpd files created on the same location on different disks? You know, those files that will start cluttering your log file with warnings from the shfs deamon. I think there are some "restrictions", or at least some caveats to using AFP. Correct me if I'm wrong. This is all because of the "CNID" database that needs to be maintained. If you are going to use AFP you have to decide for each disk whether you are going to use that disk strictly as a "disk share" or if it will be part of a "user share". Suppose you export disk1 for AFP. Then in finder you will see a disk1 share. The instant you navigate to the share, netatalk creates a number of apple-specific files (such as .AppleDouble, Network Trash, etc) as well as the .AppleDB folder that contains the CNID database. This is all created in the root of disk1. Similar, suppose you don't export any of the disks for AFP, and instead you export a user share, call it "Movies". As soon as you navigate to the share, netatalk will create all those files. Now the exact disk they get created on depends on the usual rules for which disks are included with a share and the allocation method. But here's the key: the CNID database will apply to the user share as a whole, not to individual disks. So let's say the netatalk stuff for the Movies share was created on disk2, and over time you copy lots of files to the Movies share and end getting written across say disk2, disk3, and disk4. Now if you export disk2 as an AFP disk share, the CNID database will refer to files outside the scope of disk2 - and I don't know what will happen, probably the files will show up in a directory listing, but not be accessible? This is something I haven't tested yet. Because of this, I'm not sure if the code should impose restrictions automatically - eg, if it detects a disk share has been exported with AFP, it will not let that disk participate in a user share exported with AFP - but this can be impossible to enforce under some circumstances. Make sense? It's also possible I still have some fuzzy thinking on this
January 10, 201115 yr I think there are some "restrictions", or at least some caveats to using AFP. Correct me if I'm wrong. This is all because of the "CNID" database that needs to be maintained. If you are going to use AFP you have to decide for each disk whether you are going to use that disk strictly as a "disk share" or if it will be part of a "user share". Suppose you export disk1 for AFP. Then in finder you will see a disk1 share. The instant you navigate to the share, netatalk creates a number of apple-specific files (such as .AppleDouble, Network Trash, etc) as well as the .AppleDB folder that contains the CNID database. This is all created in the root of disk1. Similar, suppose you don't export any of the disks for AFP, and instead you export a user share, call it "Movies". As soon as you navigate to the share, netatalk will create all those files. Now the exact disk they get created on depends on the usual rules for which disks are included with a share and the allocation method. But here's the key: the CNID database will apply to the user share as a whole, not to individual disks. So let's say the netatalk stuff for the Movies share was created on disk2, and over time you copy lots of files to the Movies share and end getting written across say disk2, disk3, and disk4. Now if you export disk2 as an AFP disk share, the CNID database will refer to files outside the scope of disk2 - and I don't know what will happen, probably the files will show up in a directory listing, but not be accessible? This is something I haven't tested yet. Because of this, I'm not sure if the code should impose restrictions automatically - eg, if it detects a disk share has been exported with AFP, it will not let that disk participate in a user share exported with AFP - but this can be impossible to enforce under some circumstances. Make sense? It's also possible I still have some fuzzy thinking on this Well, this was exactly the problem I was facing with AFP and eventually THE reason that I switched back to using SMB. If you only export your disks, then there isn't a problem for the netatalk daemon, only the shfs daemon will start complaining about those "duplicate files". But when you start using user shares which have overlapping disks, the netatalk daemon will start mixing those files, leading to unkown issues. I guess the way unRAID is setup is quite unique and thus this issue has never really occurred. I should note however that I never really noticed any issues causes by the netatalk daemon, but I'm sure one was bound to show up with unknown effects on data integrity(?).
January 10, 201115 yr Typical APF shares are done at the disk or folder level. A bigger share might include a lesser share but there is no way to make partially overlapping shares. How about this. 1) If an AFP disk share is made then that disk cannot be included in a user share. A disk should be able to contain multiple non-overlapping user shares, however. Or, worst case, 2) each disk will only be allowed to be included in a single AFP share, be it user or disk. Once an APF share is removed all of the AFP meta-data files associated with it can be cleared/removed.
January 10, 201115 yr Author Typical APF shares are done at the disk or folder level. A bigger share might include a lesser share but there is no way to make partially overlapping shares. How about this. 1) If an AFP disk share is made then that disk cannot be included in a user share. A disk should be able to contain multiple non-overlapping user shares, however. Or, worst case, 2) each disk will only be allowed to be included in a single AFP share, be it user or disk. Once an APF share is removed all of the AFP meta-data files associated with it can be cleared/removed. How about this: - if user shares are enabled, disk shares can not be exported via AFP, and - if any disk shares are being exported via AFP, user shares can not be enabled
January 10, 201115 yr Typical APF shares are done at the disk or folder level. A bigger share might include a lesser share but there is no way to make partially overlapping shares. How about this. 1) If an AFP disk share is made then that disk cannot be included in a user share. A disk should be able to contain multiple non-overlapping user shares, however. Or, worst case, 2) each disk will only be allowed to be included in a single AFP share, be it user or disk. Once an APF share is removed all of the AFP meta-data files associated with it can be cleared/removed. How about this: - if user shares are enabled, disk shares can not be exported via AFP, and - if any disk shares are being exported via AFP, user shares can not be enabled I think that this is reasonable. User shares are a great function in unRaid and their smooth functioning with afp should be prioritized.
January 10, 201115 yr How about this: - if user shares are enabled, disk shares can not be exported via AFP, and - if any disk shares are being exported via AFP, user shares can not be enabled Hmm, the greatest functions of unRAID are 1) being able to read from virtualized user shares (i.e. single point for accessing all data) and 2) still being able to write to individual disks. Now, I can't imagine anyone not using user shares (for all media related stuff and what overlapped disks are good for, etc), but that feature automatically excludes AFP. No good. What for is then AFP, for TM only? TM doesn't necessary need AFP, and a TM backup doesn't need to be RAID protected at all. Edit: If TM is the only reason for AFP, then what about exporting the cache disk via AFP and excluding TM's directory from being moved to the array and that's it.
January 10, 201115 yr Author Now, I can't imagine anyone not using user shares (for all media related stuff and what overlapped disks are good for, etc), but that feature automatically excludes AFP. No the 'user share' feature does not exclude AFP. What you can't do is export a disk share via AFP that corresponds to one of the disks being used by a user share being exported by AFP. Here I think is the problem: Suppose you have this directory structure to start with: /mnt/disk1/ Movies/ AMovie.mov /mnt/disk2/ Movies/ BMovie.mov Hence you have these shares: disk1 disk2 Movies Suppose you now export all shares via AFP, resulting in: /mnt/disk1/ .AppleDB/ .AppleDesktop/ .AppleDouble/ Network Trash Folder/ Temporary Items/ Movies/ .AppleDB/ .AppleDesktop/ .AppleDouble/ Network Trash Folder/ Temporary Items/ AMovie.mov /mnt/disk2/ .AppleDB/ .AppleDesktop/ .AppleDouble/ Network Trash Folder/ Temporary Items/ Movies/ BMovie.mov /mnt/user/Movies .AppleDB/ .AppleDesktop/ .AppleDouble/ Network Trash Folder/ Temporary Items/ AMovie.mov BMovie.mov In this example, those apple files in the root of disk1 and disk2 correspond just to the files on disk1 and disk2. The additional set of apple files in disk1/Movies corresponds to all the files in the Movies shares (I show them being created on disk1 but could be created on any disk of the share). Issue is that a given file, say AMovie.mov will have two CNID's associated with it, one for it's existence on disk1, the other for it's existence on Movies (though the two CNID's are in different volume name spaces). So I don't know if this is an issue or not. Also, say you rename disk1/Movies/AMovie.mov to disk1/Movies/CMovie.mov, I think the directory listing of user share Movies will not pick up this rename - this would be a problem. Similar problem if you delete disk1/Movies/AMovie.mov - the deletion is reflected in the CNID database for disk1, but not for the CNID data for Movies. I could make a change in user share file system to exclude those apple files so they don't show up as duplicates or interpreted as user shares themselves, but I'm not sure about the rename/delete issue.
January 10, 201115 yr Issue is that a given file, say AMovie.mov will have two CNID's associated with it, one for it's existence on disk1, the other for it's existence on Movies (though the two CNID's are in different volume name spaces). So I don't know if this is an issue or not. Also, say you rename disk1/Movies/AMovie.mov to disk1/Movies/CMovie.mov, I think the directory listing of user share Movies will not pick up this rename - this would be a problem. Similar problem if you delete disk1/Movies/AMovie.mov - the deletion is reflected in the CNID database for disk1, but not for the CNID data for Movies. Tom, exactly. There must be some sort of mapping performed transparently by unRAID and probably build by you into the shfs driver. Disk shares remain as they are as Disk Shares, with all those .Apple* Files. The shfs takes care of the mapping into the virtualized User Share volume. Everything else is something that can be done by simply dropping the packages on current unRAID version.
January 10, 201115 yr Typical APF shares are done at the disk or folder level. A bigger share might include a lesser share but there is no way to make partially overlapping shares. How about this. 1) If an AFP disk share is made then that disk cannot be included in a user share. A disk should be able to contain multiple non-overlapping user shares, however. Or, worst case, 2) each disk will only be allowed to be included in a single AFP share, be it user or disk. Once an APF share is removed all of the AFP meta-data files associated with it can be cleared/removed. How about this: - if user shares are enabled, disk shares can not be exported via AFP, and - if any disk shares are being exported via AFP, user shares can not be enabled This works for me. Except could it be, "- if user shares are enabled, disk shares that are included in a current AFP user share can not be exported via AFP, and "- if any disk shares are being exported via AFP, AFP user shares can not be enabled that include the existing AFP disk shares" I.e., any particular disk may either be an AFP disk share or included in an AFP user share. I think overlapping shares is an AFP problem unique to UnRAID. Trying to solve this problem in the first AFP release is looking for trouble. If you can allow the restrictions to be relaxed as a command line option then advanced users can try it out for a while and see what problems crop up.
January 10, 201115 yr Author This works for me. Except could it be, "- if user shares are enabled, disk shares that are included in a current AFP user share can not be exported via AFP, and "- if any disk shares are being exported via AFP, AFP user shares can not be enabled that include the existing AFP disk shares" I.e., any particular disk may either be an AFP disk share or included in an AFP user share. Why would you want to do this? The way I see it, an unRaid server is typically set up in one of three "modes": a) Only use disk shares, no user shares at all. b) Only use user shares, no disk shares at all. c) Use both, but only for the purpose creating new files via disk shares so that you have absolute control over which disks the files get created on, but for reading, you access through the user shares. It's mode c) that is problematic with AFP. Is there a mode d) ?
Archived
This topic is now archived and is closed to further replies.