gfjardim Posted November 13, 2013 Share Posted November 13, 2013 I've done some preliminary work compiling netatalk 3.1.0 to unRAID. The package must be placed under /boot/extra directory, and will overwrite the default netatalk installation. For now, the default cnid database is located under /var/netatalk directory. You can change this by creating a file named "afp.cfg" under the /boot/config directory, and adding there this line: vol dbpath = /path/to/database Every line inserted into the /boot/config/afp.cfg will be imported into the netatalk configuration, under /etc/afp.conf. The rc.d init script should be able to import the netatalk v2 configuration file generated by unRAID to this file. https://dl.dropboxusercontent.com/u/18726846/netatalk-3.1.0-i486-unRAID.txz As I pointed on the topic name, this is extremely cutting edge, so be careful. Quote Link to comment
gfjardim Posted November 15, 2013 Author Share Posted November 15, 2013 OK, I did some testing these days, and found netatalk 3.1.0 to be far more stable than the v2, no dropouts or other misbehavior. The database can be stored on the RAM or on a disk. Scanning almost 78k files generated a database with 171MB: Guilhermes-MacBook-Pro:volumes Guilherme$ df -h Filesystem Size Used Avail Capacity iused ifree %iused Mounted on /dev/disk0s2 111Gi 61Gi 50Gi 56% 16039524 13055778 55% / devfs 191Ki 191Ki 0Bi 100% 664 0 100% /dev /dev/disk1s2 465Gi 259Gi 206Gi 56% 67947364 54065302 56% /Volumes/HDD map -hosts 0Bi 0Bi 0Bi 100% 0 0 100% /net map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /home //guilherme@Servidor._afpovertcp._tcp.local/Documentos 15Ti 14Ti 1.0Ti 93% 3632074684 274843078 93% /Volumes/Documentos //guilherme@Servidor._afpovertcp._tcp.local/Filmes 15Ti 14Ti 1.0Ti 93% 3632074684 274843078 93% /Volumes/Filmes //guilherme@Servidor._afpovertcp._tcp.local/Fotos 15Ti 14Ti 1.0Ti 93% 3632074684 274843078 93% /Volumes/Fotos //guilherme@Servidor._afpovertcp._tcp.local/Seriados 15Ti 14Ti 1.0Ti 93% 3632074684 274843078 93% /Volumes/Seriados //guilherme@Servidor._afpovertcp._tcp.local/Shows 15Ti 14Ti 1.0Ti 93% 3632074684 274843078 93% /Volumes/Shows //guilherme@Servidor._afpovertcp._tcp.local/TimeMachine 1.8Ti 1.3Ti 546Gi 71% 345135635 143228095 71% /Volumes/TimeMachine Guilhermes-MacBook-Pro:volumes Guilherme$ cd Documentos Guilhermes-MacBook-Pro:Documentos Guilherme$ find . -type f | wc -l 20087 Guilhermes-MacBook-Pro:Documentos Guilherme$ cd /volumes/FIlmes Guilhermes-MacBook-Pro:FIlmes Guilherme$ find . -type f | wc -l 33249 Guilhermes-MacBook-Pro:FIlmes Guilherme$ cd /volumes/Fotos Guilhermes-MacBook-Pro:Fotos Guilherme$ find . -type f | wc -l 13053 Guilhermes-MacBook-Pro:Fotos Guilherme$ cd /volumes/Seriados Guilhermes-MacBook-Pro:Seriados Guilherme$ find . -type f | wc -l 2688 Guilhermes-MacBook-Pro:Seriados Guilherme$ cd /volumes/Shows Guilhermes-MacBook-Pro:Shows Guilherme$ find . -type f | wc -l 257 Guilhermes-MacBook-Pro:Shows Guilherme$ cd /volumes/TimeMachine Guilhermes-MacBook-Pro:TimeMachine Guilherme$ find . -type f | wc -l 8603 root@Servidor:~# du -sh /var/netatalk 171M /var/netatalk Maybe it's time for Tom to upgade netatalk in unRAID. Any server with more than 2G of RAM should run it with ease, and lower memory servers can always use a disk to store the CNID database. Quote Link to comment
limetech Posted November 15, 2013 Share Posted November 15, 2013 For "production" wouldn't you want to store a per-share DB in persistent storage? For example, perhaps in the root of the share, or on the cache disk. If the DB is kept in RAM as you have here, of course it's gone upon reboot. But maybe that's all right? Because netatalk stores the CNID in an extended attribute and will regenerate it on-the-fly, I believe. Quote Link to comment
gfjardim Posted November 16, 2013 Author Share Posted November 16, 2013 For "production" wouldn't you want to store a per-share DB in persistent storage? For example, perhaps in the root of the share, or on the cache disk. If the DB is kept in RAM as you have here, of course it's gone upon reboot. But maybe that's all right? Because netatalk stores the CNID in an extended attribute and will regenerate it on-the-fly, I believe. Maybe I understood this wrong, but netatalk only uses the database as a bridge between a file protocol that relies on inode to address an archive, and a filesystem that is based in the file path to accomplish the same goal. So, in my limited cognition, the CNID database is only a method to make this translation consistent. Unlike other protocols like smb or nfs, the AFP protocol mostly refers to files and directories by ID and not by a path (the IDs are also called CNID, that means Catalog Node ID). A typical AFP request uses a directory ID and a filename, something like "server, please open the file named 'Test' in the directory with id 167". For example "Aliases" on the Mac basically work by ID (with a fallback to the absolute path in more recent AFP clients. But this applies only to Finder, not to applications). Every file in an AFP volume has to have a unique file ID, IDs must, according to the specs, never be reused, and IDs are 32 bit numbers (Directory IDs use the same ID pool). So, after ~4 billion files/folders have been written to an AFP volume, the ID pool is depleted and no new file can be written to the volume. No whining please :-) Netatalk needs to map IDs to files and folders in the host filesystem. To achieve this, several different CNID backends are available and can be choosed by the cnidscheme option in the AppleVolumes.default(5) configuration file. A CNID backend is basically a database storing ID <-> name mappings. That been said, since netatalk will always generate a CNID to every file or directory on a mapped share and store it into a database, it's irrelevant that the database is or not persistent. The non persistent database will make the mount process slower when the share is accessed for the first time, since a CNID will be generated to every file, but then every access will be faster, since the translation database will be stored on the RAM. Quote Link to comment
limetech Posted November 16, 2013 Share Posted November 16, 2013 Here is the relevant section in the netatalk documentation: http://netatalk.sourceforge.net/3.0/htmldocs/configuration.html#CNID-backends (BTW, funny the first sentence reads reads, "Unlike other protocols like SMB or NFS, the AFP protocol mostly refers to files and directories by ID and not by a path..."., because this is more-or-less how NFS works too, and why it's also an archaic broken protocol. No way Apple would be moving to SMB if AFP or NFS could take them into the future.) Maybe you can do some experiments? Configure so that the CNID DB is in ram and then browse the file system via OSX client. Maybe access some files. Now restart the server, and when the server is back up, browse the same files you did earlier from your OSX client. Just wondering what AFP will do if it references a file by CNID on the server, and that CNID is not in the server CNID database. (This is like "stale file handle" in NFS.) Quote Link to comment
gfjardim Posted November 17, 2013 Author Share Posted November 17, 2013 I didn't see any misbehavior from netatalk storing the database on the RAM. Nov 16 17:13:22 Servidor afpd[22023]: Netatalk AFP/TCP listening on 192.168.0.100:548 Nov 16 17:13:22 Servidor emhttp: Restart SMB... Nov 16 17:13:22 Servidor emhttp: shcmd (89): killall -HUP smbd Nov 16 17:13:22 Servidor emhttp: shcmd (90): ps axc | grep -q rpc.mountd Nov 16 17:13:22 Servidor emhttp: _shcmd: shcmd (90): exit status: 1 Nov 16 17:13:22 Servidor emhttp: Start AFP... Nov 16 17:13:22 Servidor emhttp: shcmd (91): /etc/rc.d/rc.atalk start |& logger Nov 16 17:13:22 Servidor netatalk[22012]: Received SIGHUP, sending all processes signal to reload config Nov 16 17:13:22 Servidor logger: Nov 16 17:13:22 Servidor emhttp: shcmd (92): cp /etc/avahi/services/afp.service- /etc/avahi/services/afp.service Nov 16 17:13:22 Servidor afpd[22023]: Netatalk AFP/TCP listening on 192.168.0.100:548 Nov 16 17:13:22 Servidor avahi-daemon[22021]: Files changed, reloading. Nov 16 17:13:22 Servidor avahi-daemon[22021]: Service group file /services/afp.service changed, reloading. Nov 16 17:13:22 Servidor emhttp: shcmd (93): /usr/local/sbin/emhttp_event svcs_restarted Nov 16 17:13:22 Servidor emhttp_event: svcs_restarted Nov 16 17:13:22 Servidor netatalk[22012]: Received SIGHUP, sending all processes signal to reload config Nov 16 17:13:23 Servidor avahi-daemon[22021]: Service "Servidor" (/services/afp.service) successfully established. Nov 16 17:13:47 Servidor afpd[22023]: Netatalk AFP/TCP listening on 192.168.0.100:548 Nov 16 17:13:51 Servidor afpd[22949]: AFP3.3 Login by guilherme Quote Link to comment
gfjardim Posted November 17, 2013 Author Share Posted November 17, 2013 Tom, i've been testing some variables. I think netatalk 3 don't permit per share database, it's stored in one centralized path. So there are really only 3 choices to store it: cache, diskX or memory. Quote Link to comment
dgaschk Posted November 20, 2013 Share Posted November 20, 2013 I've installed this because I've had to reboot almost weekly to get TM working. I think that it takes a long time for the AFP shares to get ready. Perhaps due to db building... TM is currently running and I'll report back if I have problems. I added this to /boot/config/afp.cfg but I'm not sure that it's needed. I may not have waited long enough... ; ; Netatalk 3.x configuration file ; [Global] ; Global server settings mimic model = RackMac uam list = uams_dhx.so uams_dhx2.so uams_guest.so guest account = nobody zeroconf = yes afpstats = yes Quote Link to comment
gfjardim Posted November 21, 2013 Author Share Posted November 21, 2013 I've installed this because I've had to reboot almost weekly to get TM working. I think that it takes a long time for the AFP shares to get ready. Perhaps due to db building... TM is currently running and I'll report back if I have problems. I added this to /boot/config/afp.cfg but I'm not sure that it's needed. I may not have waited long enough... ; ; Netatalk 3.x configuration file ; [Global] ; Global server settings mimic model = RackMac uam list = uams_dhx.so uams_dhx2.so uams_guest.so guest account = nobody zeroconf = yes afpstats = yes You can add anything you want, but remove the [Global] section header from it. The init script will put everything from /boot/config/afp.cfg in the [Global] section. Hope this will help you with your problem. Quote Link to comment
limetech Posted November 23, 2013 Share Posted November 23, 2013 The time machine problem that was talked about in this thread is separate discussion from netatalk 3.1.0, so I split it off and moved here: http://lime-technology.com/forum/index.php?topic=30390.0 Quote Link to comment
dgaschk Posted December 10, 2013 Share Posted December 10, 2013 5.0.4 started giving me TM problems. TM fails and the TM share could not be accessed at all. Rebooting the client and server did not fix the issue. The share was visible but not accessible. I re-enabled netatalk 3.1.0 and the issue is gone. I will report if 3.1.0 starts having problems. I removed my custom afp.cfg and the system works with the default settings. Quote Link to comment
DaleWilliams Posted December 11, 2013 Share Posted December 11, 2013 I'm finding unraid, 5.0.4 and TM under Mavericks to be rock solid. No connection problems. And no longer have to manually Connect to Server/Tower in order to initiate the first TM update. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.