limetech Posted July 10, 2013 Share Posted July 10, 2013 modifed 7-11-2013 to correctly handle directories with spaces There was a bug fixed back in -rc13: - shfs: fixed removexattr() for directories to look at all disks even if xattr not present on some For those using AFP this bug may have caused extraneous extended attributes to be left on the top-level directories of your data disks (i.e., your share directories). This may also cause the 'mover' to not complete correctly when the target directory is on one of those disks. You might see error messages in syslog like this: rsync: get_xattr_data: lgetxattr(".","user.org.netatalk.supports-eas.1qmk2g",0) failed: Exec format I have developed a script that should clean up this mess with the extended attributes on directories. Save the attached file to the root of your flash device (same place where bzimage/bzroot are). Next, you will need to look at each of the top-level directories on all of your disks. For example, let's say you have /mnt/disk1/Movies directory. From telnet session first type this: getfattr –d /mnt/disk1/Movies If this does not produce any messages, then the directory is clean and you can move on to the next directory. If it does produce a list of several of those “user.org.netatalk.supports-eas” attributes along with possibly a few errors, then you should next type this: /boot/rmattrs /mnt/disk1/Movies If it just returns and produces no messages, then type this: getfattr –d /mnt/disk1/Movies And verify that nothing is listed by this now either. View the directory and see if everything looks ok: vdir /mnt/disk1/Movies If one of those steps fails, STOP and post here before proceeding! If everything worked, then you will need to do the above procedure for each top-level directory on each data disk. Yes this might be tedious but I’d rather do this one-by-one, than risk something going wrong with a more complex script. For a directory with spaces, use this syntax: /boot/rmattrs "/mnt/disk1/TV Shows" or /boot/rmattrs /mnt/disk1/TV\ Shows rmattrs Quote Link to comment
prostuff1 Posted July 11, 2013 Share Posted July 11, 2013 Very nice to see this. I am messing around with it on my server which I know has some of these problems and I have discovered one issue. Your script does not like spaces in the directory names. Not sure if you want me to upload my fixed version or send it your direction but my modified one is working for me. For reference I have a share called "TV Shows" that is was failing on. Quote Link to comment
madburg Posted July 11, 2013 Share Posted July 11, 2013 Thanks Tom Anytime I would sticky this and maybe put a link in the 5.0 docs your working on completing. Quote Link to comment
jbartlett Posted July 11, 2013 Share Posted July 11, 2013 Very nice to see this. I am messing around with it on my server which I know has some of these problems and I have discovered one issue. Your script does not like spaces in the directory names. Not sure if you want me to upload my fixed version or send it your direction but my modified one is working for me. For reference I have a share called "TV Shows" that is was failing on. On the off chance you may not have tried, did you try calling the script with the directory wrapped in quotes? -John Quote Link to comment
subwars Posted July 11, 2013 Share Posted July 11, 2013 so this is only a problem for people using or were using AFP? if not, no need to worry about it? Quote Link to comment
limetech Posted July 11, 2013 Author Share Posted July 11, 2013 so this is only a problem for people using or were using AFP? if not, no need to worry about it? That's correct, unless some program or plugin accessing files on your server adds/removes extended attributes on directories (not likely). Quote Link to comment
brimnac Posted July 11, 2013 Share Posted July 11, 2013 Very nice to see this. I am messing around with it on my server which I know has some of these problems and I have discovered one issue. Your script does not like spaces in the directory names. Not sure if you want me to upload my fixed version or send it your direction but my modified one is working for me. For reference I have a share called "TV Shows" that is was failing on. I had that same issue, too, so I just removed the space on the folder per disk basis. I thought it was just me. Quote Link to comment
prostuff1 Posted July 11, 2013 Share Posted July 11, 2013 On the off chance you may not have tried, did you try calling the script with the directory wrapped in quotes? -John Yup, tried that and it still did not want to place nice. Quote Link to comment
theone Posted July 11, 2013 Share Posted July 11, 2013 When I run "getfattr" I get (under RC16b): root@Tower:~# getfattr –d /mnt/disk1/Media getfattr: –d: No such file or directory Is this OK? Quote Link to comment
madburg Posted July 11, 2013 Share Posted July 11, 2013 When I run "getfattr" I get (under RC16b): root@Tower:~# getfattr –d /mnt/disk1/Media getfattr: –d: No such file or directory Is this OK? Yes, you don't want ANY entries with "user.org.netatalk.supports-eas.xxxx" to come back. Its important to note (Tom should add) depending on how many files you have, is how long it will take to search for extended attributes, and if it finds them, the move and move back will depend on how many files as well (if I am not mistaken). Quote Link to comment
peter_sm Posted July 11, 2013 Share Posted July 11, 2013 I see this ....... root@tower:/boot# getfattr -d /mnt/disk2/Movie /mnt/disk2/Movie: user.org.netatalk.supports-eas.2mxSuY: Input/output error getfattr: Removing leading '/' from absolute path names # file: mnt/disk2/Movie user.org.netatalk.supports-eas.nW4p1q=0seWVzAA== user.org.netatalk.supports-eas.zVDgV6=0seWVzAA== Quote Link to comment
madburg Posted July 11, 2013 Share Posted July 11, 2013 I see this ....... root@tower:/boot# getfattr -d /mnt/disk2/Movie /mnt/disk2/Movie: user.org.netatalk.supports-eas.2mxSuY: Input/output error getfattr: Removing leading '/' from absolute path names # file: mnt/disk2/Movie user.org.netatalk.supports-eas.nW4p1q=0seWVzAA== user.org.netatalk.supports-eas.zVDgV6=0seWVzAA== None of those should exist. root@tower:/boot# getfattr -d /mnt/disk2/Movie /mnt/disk2/Movie: user.org.netatalk.supports-eas.2mxSuY: Input/output error That is a damaged/corrupted one, running mover (provided u have a cache drive) or rync with extended attribute support script will choke to the movie user share. user.org.netatalk.supports-eas.nW4p1q=0seWVzAA== user.org.netatalk.supports-eas.zVDgV6=0seWVzAA== [/code] These two got written and should have been immediately removed, weren't and are read each time only confusing matters. Tom's script will remove all three, none will come back. His script may have an issue with spaces in a share name, so I would hold until he comments back. In your case "Movies" does not contain a space in the name and can be run again the "Movie" directory for EACH disk it resides on. Quote Link to comment
limetech Posted July 11, 2013 Author Share Posted July 11, 2013 Very nice to see this. I am messing around with it on my server which I know has some of these problems and I have discovered one issue. Your script does not like spaces in the directory names. Not sure if you want me to upload my fixed version or send it your direction but my modified one is working for me. For reference I have a share called "TV Shows" that is was failing on. Yes, please email to me: [email protected] Thanks! Quote Link to comment
prostuff1 Posted July 11, 2013 Share Posted July 11, 2013 Yes, please email to me: [email protected] Thanks! Sent, let me know if you don't get it. Quote Link to comment
limetech Posted July 11, 2013 Author Share Posted July 11, 2013 Yes, please email to me: [email protected] Thanks! Sent, let me know if you don't get it. Yes, thank you very much. I edited the first post and uploaded the new script. Quote Link to comment
brandon Posted July 14, 2013 Share Posted July 14, 2013 Not sure if it's related, but this fix didn't clear up my mover issue: http://lime-technology.com/forum/index.php?topic=23100.10 I did however have a bunch of "user.org.netatalk.supports-eas" messages show up for some shares, and now they don't thanks to this tool. Quote Link to comment
Interstellar Posted July 14, 2013 Share Posted July 14, 2013 I found some on my disk 6 [1-5 ok] root: root@Tower:~# getfattr -d /mnt/disk6 # file: mnt/disk6 user.org.netatalk.supports-eas.5eEg2D user.org.netatalk.supports-eas.D19lJz user.org.netatalk.supports-eas.GczGE0 user.org.netatalk.supports-eas.J0IAGj user.org.netatalk.supports-eas.cW7w71 user.org.netatalk.supports-eas.mU9ttc Ran /boot/rmattrs /mnt/disk6, which then caused an OOM errors and the server fell over. I don't know if it is the script or something else, but I'm going to try again and watch memory usage!! Confirmed: As soon as I start the script the memory usage goes through the roof (cached memory fills up) and then everything eventually falls over. Yep: Even at stock (as safemode doesn't allow any access to the server at all ) Now I get: root@Tower:~# getfattr -d /mnt/disk6 /mnt/disk6: user.org.netatalk.supports-eas.5eEg2D: Input/output error /mnt/disk6: user.org.netatalk.supports-eas.D19lJz: Input/output error getfattr: Removing leading '/' from absolute path names # file: mnt/disk6 user.org.netatalk.supports-eas.GczGE0=0seWVzAA== /mnt/disk6: user.org.netatalk.supports-eas.J0IAGj: Input/output error /mnt/disk6: user.org.netatalk.supports-eas.cW7w71: Input/output error /mnt/disk6: user.org.netatalk.supports-eas.mU9ttc: Input/output error Disk and the data is fine, it seems. Solution? Quote Link to comment
prostuff1 Posted July 14, 2013 Share Posted July 14, 2013 You are supposed to run it on the top level folders on the disk, not the disk itself. Quote Link to comment
brandon Posted July 15, 2013 Share Posted July 15, 2013 You are supposed to run it on the top level folders on the disk, not the disk itself. By top level folders, you mean user shares (if they're being used)? Just want to make sure I did it right (and not wrong as per my problem above). ex. I have 13 drives, and I used "getfattr -d /mnt/disk#/Movies" (where # was 1-12), and they all reported "user.org.netatalk.supports-eas.J0IAGj: Input/output error" and a few "Removing leading '/' from absolute path names" Then I ran "/boot/rmattrs /mnt/disk#/Movies" for each of the drives (since they all reported errors), and not testing again with getfattr -d nothing shows up. Is this the right practice? Quote Link to comment
prostuff1 Posted July 15, 2013 Share Posted July 15, 2013 Correct, if User Shares are turned on then all folders under /mnt/disk*/ are "top level" folders. Quote Link to comment
Interstellar Posted July 15, 2013 Share Posted July 15, 2013 You are supposed to run it on the top level folders on the disk, not the disk itself. I had suspected that, I'm assuming it will sort itself out. Ill run it on my user shares now! Quote Link to comment
itimpi Posted July 15, 2013 Share Posted July 15, 2013 Ill run it on my user shares now! Note that the instructions tell you to run it on the disk shares (rather than the user shares) and one folder at a time. How important this is I am not sure, but I guess there must be a reason Tom specified doing at that level of granularity. Quote Link to comment
madburg Posted July 15, 2013 Share Posted July 15, 2013 e.g. User Share = "TV SHows" / Disks Included = disk1-4 Disk1 Top level directory = "TV SHows" Disk2 Top level directory = "TV SHows" (user.org.netatalk.supports-eas.xxx) Disk3 Top level directory = "TV SHows" Disk4 Top level directory = "TV SHows" So you would run: getfattr –d /mnt/disk1/TV\ Shows/ getfattr –d /mnt/disk2/TV\ Shows/ getfattr –d /mnt/disk3/TV\ Shows/ getfattr –d /mnt/disk4/TV\ Shows/ You would notice that when you got to disk #2 the command outputted you had an "user.org.netatalk.supports-eas.xxx" So now you would run: /boot/rmattrs "/mnt/disk2/TV Shows" Now verify the apple eas extended attribute is now gone: getfattr –d /mnt/disk2/TV\ Shows/ Don't run against User Share nor a Disk itself, run "getfattr –d" against the top level directory on EACH disk, to see whether it contains a eas extended attribute. If it finds one or more, run Tom's script to have them removed them. The script is actually creating a new directory (e.g. TV Shows~temp) on the disk, moving all the data from your current (e.g. TV Shows) directory you selected to run against and then deletes the original (e.g. TV Shows) directory (Because the apple eas extended attribute is tied to the top level directory). Then re-names the temp (e.g. TV Shows~temp) directory to the original name (e.g. TV Shows). So you have to follow the directions if you don't want to lose data; hence why Tom want you to go one by one and stop immediately if something went wrong. Hope I don't get a shellacking but you could speed up the process of FINDING if you have apple eas extended attributes by running getfattr –dR /mnt/user (if you have a cache drive) getfattr –dR /mnt/user0 (if you don't have a cache drive) So it would scan all in one shot for you and should any come back you should note the DISK & Top Level Directory it references the apple eas extend attribute resides on, so you know which disk and top level directory to run the "rmattrs" against. Note as an example the following is not an apple eas extended attribute, it is a file extended attribute and IS correct: uninstall_flash_player_osx.dmg user.com.apple.metadata:kMDItemDownloadedDate=0sYnBsaXN0MDChATNBtRZGToOIIggKAAAAAAAAAQEAAAAAAAAAAgAAAAAAAAAAAAAAAAAAABM= user.com.apple.metadata:kMDItemWhereFroms=0sYnBsaXN0MDCiAQJfEFdodHRwOi8vZnBkb3dubG9hZC5tYWNyb21lZGlhLmNvbS9nZXQvZmxhc2hwbGF5ZXIvY3VycmVudC91bmluc3RhbGxfZmxhc2hfcGxheWVyX29zeC5kbWdfEElodHRwOi8vaGVscHguYWRvYmUuY29tL2ZsYXNoLXBsYXllci9rYi91bmluc3RhbGwtZmxhc2gtcGxheWVyLW1hYy1vcy5odG1sCAtlAAAAAAAAAQEAAAAAAAAAAwAAAAAAAAAAAAAAAAAAALE= user.com.apple.quarantine="0001;4f660ece;Safari;605F0A3D-7CE8-4202-A649-C2A42459336C|com.apple.Safari" Quote Link to comment
defected07 Posted July 22, 2013 Share Posted July 22, 2013 Hope I don't get a shellacking but you could speed up the process of FINDING if you have apple eas extended attributes by running getfattr –dR /mnt/user (if you have a cache drive) getfattr –dR /mnt/user0 (if you don't have a cache drive) So it would scan all in one shot for you and should any come back you should note the DISK & Top Level Directory it references the apple eas extend attribute resides on, so you know which disk and top level directory to run the "rmattrs" against. I'm not sure this is correct. Maybe I'm not using the right flags/options, but my results did not indicate which disk the file resided on -- just referenced the absolute User Share path. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.