mover crashing / kernel panic (with X option in rsync)

[ThOr101]

I am hoping someone can help.

My mover will consistently crash my unraid box when mover runs.

If I change the command to remove the X it doesn't crash, so my rsync line looks likes this:

 

rsync -i -dIWRpEAogt --numeric-ids --inplace {} /mnt/user0

 

I've done an reiserfsck on every drive and had NO errors to correct.

It almost seems as though the share is a little wonky.  So how do I correct that?

 

--THOR!

[Joe L.]

I am hoping someone can help.

My mover will consistently crash my unraid box when mover runs.

If I change the command to remove the X it doesn't crash, so my rsync line looks likes this:

 

rsync -i -dIWRpEAogt --numeric-ids --inplace {} /mnt/user0

 

I've done an reiserfsck on every drive and had NO errors to correct.

It almost seems as though the share is a little wonky.  So how do I correct that?

 

--THOR!

What version of unRAID are you using?  If the current 5.0-rc11, then report this to lime-tech in the release thread so it can get fixed.

Sounds like handling extended attributes is causing the crash.

 

Joe L.

[madburg]

If its not to late, can you check the syslogs in /var/log and search for 'shfs' errors.

I was getting them and mover had issues trying to move apple files. Parts of my logs can be seen here http://lime-technology.com/forum/index.php?topic=25689.0

 

[ThOr101]

I am hoping someone can help.

My mover will consistently crash my unraid box when mover runs.

If I change the command to remove the X it doesn't crash, so my rsync line looks likes this:

 

rsync -i -dIWRpEAogt --numeric-ids --inplace {} /mnt/user0

 

I've done an reiserfsck on every drive and had NO errors to correct.

It almost seems as though the share is a little wonky.  So how do I correct that?

 

--THOR!

What version of unRAID are you using?  If the current 5.0-rc11, then report this to lime-tech in the release thread so it can get fixed.

Sounds like handling extended attributes is causing the crash.

 

Joe L.

 

Hi Joe,

 

Thanks for your help.

I am running rc11, and I thought I was in the right place to put release candidate bugs.

Is there another thread / forum section I'm missing?

 

Thanks for the help.

[ThOr101]

If its not to late, can you check the syslogs in /var/log and search for 'shfs' errors.

I was getting them and mover had issues trying to move apple files. Parts of my logs can be seen here http://lime-technology.com/forum/index.php?topic=25689.0

 

I don't see any errors in the syslog, until I enable the "X" and then it all goes wonky.

[ThOr101]

I should add one more thing.

 

At one point, I had a bad set of memory in the box.  My fear is that somewhere along the way of moving files to the share, that something got corrupted in the SHFS, but not the individual REISERFS system.

 

But I don't really know how to go check the integrity of the SHFS, and this is also only a hypothesis.

 

--THOR!

[RobJ]

I know nothing about Apple support, so this may have nothing to do with your issue, but Tom released a special version (RC11a) that fixes an issue with extended attributes, see this post with rc11a download link.  This is a special test release, to confirm that the fix works.

[ThOr101]

Hi Rob,

 

Thanks for the pointer.

I downloaded the rc11a and tried it, it didn't solve my problem.

But I posted that in the other thread.  It seems to have Tom's attention.

 

Here's hoping!

 

--THOR!

[madburg]

Hey RobJ,

 

Yes, I went to that version RC11a to hopefully resolve the AFP issues I have (and the code snippet are from RC11a).

What I noticed was this, if you start from scratch and disable AFP, wipe all Apple files off cache and array. Start AFP back up it a total mess, mover bombs out trying to move apple file to the array (for some AFP shares but not others, but when it fails even for 1 you lose 'user0' and your out of commission).

 

1) If I create/set a "dbpath:" in AppleVolumes.default to a location off the array and cache drive so the DB is persistent across reboots (which is great and works really well, I enabled MAX debug to verify the shutdown and restart up of the DB)

 

2) Set "nocnidcache" in AppleVolumes.default ( this creates a ".AppleDesktop", ".AppleDouble", "Network Trash Folder" and "Temporary Items" for each AFP Share AND then manually copy them off the cache drive to a Disk of my choice to the root of the proper AFP share (the reason for this is mover still bombs out trying to move these files as well for some of the AFP shares)

 

With the following steps this (RC11a) then becomes the best AFP unRAID version so far from all others as it supports extended attributes. So hopefully it gets included going forward.

 

But it still remains that AFP support "as is" out of box from unRAID is not functional. I do recall Tom stating he was either looking into or was going to create a way to do this (setup DB) via unRAID GUI. It seems it would not be so easy as it should not be placed on a cache drive either (from what I see) as the DB needs to save and close:

 

See attachment 'Shutdown'

 

SMB gets shutdown while the DB is saving and closing out (BerkeleyDB environment recovery done. Removed BerkeleyDB environment), so it seems to me that it could not be located on the cache drive, thus I have it located off the array and cache.

 

Upon a reboot, the DB is opened and recover's:

 

See attachment 'Startup'

 

What I wonder is "CNID database version 1" should it not be version 2?

Shutdown.txt

Startup.txt

[RobJ]

Hey RobJ,

 

Yes, I went to that version RC11a to hopefully resolve the AFP issues I have (and the code snippet are from RC11a).

What I noticed was this, if you start from scratch and disable AFP, wipe all Apple files off cache and array. Start AFP back up it a total mess, mover bombs out trying to move apple file to the array (for some AFP shares but not others, but when it fails even for 1 you lose 'user0' and your out of commission).

 

I'd like to help, but I'm sorry, I have no experience with Apple products or software or even the support issues here.

 

It seems it would not be so easy as it should not be placed on a cache drive either (from what I see) as the DB needs to save and close:

 

SMB gets shutdown while the DB is saving and closing out (BerkeleyDB environment recovery done. Removed BerkeleyDB environment), so it seems to me that it could not be located on the cache drive, thus I have it located off the array and cache.

 

Upon a reboot, the DB is opened and recover's:

 

As far as I can see (and as far as I have any experience), I see nothing amiss in either syslog excerpt.  One comment (which you probably already know), log entries in the syslog are added 'simultaneously' by all parties, and don't always end up in the right order.  So be careful drawing any conclusions as to when SMB was stopped.  Also, the only relevant log entry is just a call to stop, not an indication of when it stopped.  You of course may have other info as to whether it stopped too soon.

[madburg]

Hey RobJ,

 

Yes, I went to that version RC11a to hopefully resolve the AFP issues I have (and the code snippet are from RC11a).

What I noticed was this, if you start from scratch and disable AFP, wipe all Apple files off cache and array. Start AFP back up it a total mess, mover bombs out trying to move apple file to the array (for some AFP shares but not others, but when it fails even for 1 you lose 'user0' and your out of commission).

 

I'd like to help, but I'm sorry, I have no experience with Apple products or software or even the support issues here.

 

No worries  I learn something from your second statement.

 

 

It seems it would not be so easy as it should not be placed on a cache drive either (from what I see) as the DB needs to save and close:

 

SMB gets shutdown while the DB is saving and closing out (BerkeleyDB environment recovery done. Removed BerkeleyDB environment), so it seems to me that it could not be located on the cache drive, thus I have it located off the array and cache.

 

Upon a reboot, the DB is opened and recover's:

 

As far as I can see (and as far as I have any experience), I see nothing amiss in either syslog excerpt.  One comment (which you probably already know), log entries in the syslog are added 'simultaneously' by all parties, and don't always end up in the right order.  So be careful drawing any conclusions as to when SMB was stopped.  Also, the only relevant log entry is just a call to stop, not an indication of when it stopped.  You of course may have other info as to whether it stopped too soon.

 

 

I should have been more clear. The syslog snippets I attached were of a good AFP setup (config with MAX debug enable to show its working properly). Once I placed the DB outside of unRAID's array and cache drive. So you are correct there is nothing amiss in those snippets, its to show how it should be (i guess), working properly. But would not be the case had I not SET the "DB" and "nocnidcache" value.

 

I cant image Tom tested Turning on AFP out of box (with having a cache drive) using a afp client and then trying to move the generated apple file to the array via mover and see that it doesn't work...

 

It's good to know about log entries write 'simultaneously' thanks for that, learn something every day 

 

Thought about it some more and believe I am wrong about one statement here. First SMB shutting down is not a problem too AFP shutting down its DB. Secondly I re-analyzed the log and see until the AFP DB completely shuts down ( BerkeleyDB environment recovery done. Removed BerkeleyDB environment, for each AFP Share) only then did emhttp start unmounting disks. So Tom would be able to add the ability to allow setting up the AFP DB on a cache drive.

 

I wonder if that should maybe be a pre-req to AFP. Whereby if a cache drive is set then AFP would have the option to be enabled and auto setup to the cache drive (Otherwise it seems pointless to lose it with each reboot). As an example ":DEFAULT: cnidscheme:dbd options:upriv,usedots,nodev,nocnidcache dbpath:/mnt/cache/.dbd/$v". Thereby setting up 'mnt/cache/.dbd/%AFP Share NAME%/.AppleDB' per AFP share.

 

But one issue would still remain (at this time) the ".AppleDesktop" and ".AppleDouble" directories and respective files still (like they should) get generated per each AFP share and mover fails to move them to the array (only for some, still cant understand why some succeed to move while other don't) but that for Tom to resolve.

 

Again for the time being I over came this by copying them out to the array myself and deleting them of the cache drive.

 

The mover issue may or not be related exactly to apple files, as ThOr101 ran into the fact that files with extended attributes are crashing with rsync in general as I understand it.

 

Not sure if speeding_ant has tried to move anything with his setup.