You can rebuild one of the 3TB xfs drives to 4TB and it would become a 4TB xfs drive. You can rebuild one of the 3TB reiserfs drives to 4TB and it would become a 4TB reiserfs drive. Just don't make the mistake of trying to change the format when rebuilding.

 

And you can rsync specific directories to one drive then rsync the other directories to another drive. unBalance might also help. Don't know exactly what issues were encountered with that but maybe if you just use it to move files between drives with the same format it will be OK.

 

thanks very much. I really appreciate all the actual help.

Hey this thread was just brought to my attention.  I am still in the process of trying to digest what the issue is here, and see if we can reproduce.  But there are a couple things that need to be said:

 

mikedpitt420 - I understand the panic expressed in your initial post.  But the way you talked about other members is unacceptable, please refrain from that kind of behavior in the future.  Once this issue is sorted out we will be deleting some of these posts, better yet, I ask that you go back and edit/delete them yourself.

 

mods and other members: I really appreciate you guys taking the time to help this guy out despite his attitude.

 

Now let me try and figure out wtf is going on here...

mikedpitt420 - I understand the panic expressed in your initial post.  But the way you talked about other members is unacceptable, please refrain from that kind of behavior in the future.  Once this issue is sorted out we will be deleting some of these posts, better yet, I ask that you go back and edit/delete them yourself.

 

mods and other members: I really appreciate you guys taking the time to help this guy out despite his attitude.

Just in mikedpitt420's defence, there was some sarcastic remarks given by other members which contributed to his responses, and wound up degrading the beginning of this thread to a flame war.

mikedpitt420 - I understand the panic expressed in your initial post.  But the way you talked about other members is unacceptable, please refrain from that kind of behavior in the future.  Once this issue is sorted out we will be deleting some of these posts, better yet, I ask that you go back and edit/delete them yourself.

 

mods and other members: I really appreciate you guys taking the time to help this guy out despite his attitude.

Just in mikedpitt420's defence, there was some sarcastic remarks given by other members which contributed to his responses, and wound up degrading the beginning of this thread to a flame war.

Maybe everyone involved would consider cleaning this up a little.

I over reacted, I was not trying to be a jerk, I will edit whatever anyone would like me to. Apologies, but I was provoked after losing 1TB worth of data and I over reacted. I've been bringing people to this OS in droves, and a bunch of people lost data and all came at me pretty hard. Again, apologies.

I over reacted, I was not trying to be a jerk, I will edit whatever anyone would like me to. Apologies, but I was provoked after losing 1TB worth of data and I over reacted. I've been bringing people to this OS in droves, and a bunch of people lost data and all came at me pretty hard. Again, apologies.

 

Mike, seeing as your issue looks pretty system specific rather than Unraid specific, I think they should post their own threads.  I don't think we've seen any evidence that anybody else's problems can be linked to this..

Again, they aren't going to do this. We all had the exact same thing happen when I said "I've upgraded to 6.2". They're not active on the forums here, and take all of their unRAID advice from me. I'm not just making this up. I'll give it a shot but I doubt they're going to pipe up in here. To clarify a bit more, I'm talking maybe 10-15 people. None of whom I've ever seen post anything here at all. magiin83 and fantax are the only ones I've ever seen post anything. Magiin had a totally different issue with his shares missing, as well as this one, and simply downgraded back to 6.1.9. As there were 10+ others all pretty upset with me, I posted here. We're a pretty tight knit community at Geek, and if you've ever been there, our Yoda chat bot says "unRAID, best it is" when anyone even mentions my name in there. I've been a huge advocate over there of this OS. Again, I am sorry for over reacting, but this was not an issue specific to me.

Again, they aren't going to do this. We all had the exact same thing happen when I said "I've upgraded to 6.2". They're not active on the forums here, and take all of their unRAID advice from me. I'm not just making this up. I'll give it a shot but I doubt they're going to pipe up in here.

 

I'm happy to see what others think, but I stand by my assessment.  It just seems unlikely that the only group of people that are experiencing this are from one specific sub-community, when we've not had any other reports from anyone else, coupled with the issues we've found so far.

The issue was specifically with the Sonarr docker, and the mover, moving files to a reiserFS disk. When the move completed, the files disappeared. I'm quite sure it could be replicated as even I can still. Now that I'm almost finished converting to XFS disks, the issue is gone. I was not even the 1st person to notice it. Someone else brought it to my attention, and then I checked my array, to find several seasons of blurays showed as downloaded, replacing other files, that were then just gone from the array.

Here is the problem.

 

The core issue is that something in linux kernel 4.4.x broke ACL support in ReiserFS.  I have confirmed that myself and also there is a mention of this on the Arch forum.  See last post of this thread:

https://bbs.archlinux.org/viewtopic.php?pid=1599558#p1599558

 

The kernel used in upcoming unRAID-OS 6.3 is version 4.7.6, and I have confirmed the problem does not occur there.  When we release 6.3.0-rc1 this will work again.

 

However, there is another problem.  The 'mover' uses 'rsync' to copy files from one file system to another.  One of the options included is called "--remove-source-files".  This tells 'rsync' to delete the source file after it has been successfully copied to the target.  Well this is also broken.  Apparently 'rsync' does something like this:

 

step 1: make a local copy of source file metadata (mode, attributes, acl info)

step 2: copy file from source to target

Did step 2 succeed? If yes, delete the source file and proceed to step 3, else exit with error

step 3: apply metadata to target file.

Did step 3 succeed? If yes, exit with "success", else exit with error

 

If the 'rsync' command exits with error, the mover assumes the source file did not get deleted, and in this case, it will delete the target file (because it might be partial/corrupted since rsync exited with error).

 

However, from above it's possible 'rsync' could copy the file, delete the source, and then fail changing target file attributes, resulting in both files getting deleted (which is what you see).

 

All this happens because apparently the sonarr app tags files with an ACL (specifically system.posix_acl_access).

 

I am going to put a change in 6.2.1 that will prevent the source file from being deleted if the full copy, including the metadata, did not complete successfully.  That will prevent losing files, but until a patch for 4.4.x kernel is created that fixes this, such files will be stuck on the cache.  I have sent email to the ReiserFS maintainers pointing out this problem.

 

 

So need Unraid v6.2 + Sonarr + ReiserFS for this to occur.... interesting stuff.

So need Unraid v6.2 + Sonarr + ReiserFS for this to occur.... interesting stuff.

unRAID 6.2 + Sonarr + target share on cache + cache not reiserfs + target share on array + reiserfs on array + mover

So need Unraid v6.2 + Sonarr + ReiserFS for this to occur.... interesting stuff.

unRAID 6.2 + Sonarr + target share on cache + cache not reiserfs + target share on array + reiserfs on array + mover

 

Wow, no wonder we haven't seen more of this...

So need Unraid v6.2 + Sonarr + ReiserFS for this to occur.... interesting stuff.

unRAID 6.2 + Sonarr + target share on cache + cache not reiserfs + target share on array + reiserfs on array + mover

 

Wow, no wonder we haven't seen more of this...

Yeah and who knows which kernel release reiserfs acl's got broken.  I couldn't find any patch request, or other indication whatsoever that reiserfs acl stuff was changed/broken/fixed.  It just is working in 4.7.6 and not 4.4.x.  Doing a 'diff' of reiserfs source files between the two, there are indeed lots of changes related to xattr/acl support.  Why these guys keep tinkering with a file system without active development for almost 10 years now, is beyond me. 

You have to ask on here why people tinker?  Really?! 

not cool that somebodies lost a lot of data...

but cool that my guess was somewhat on track.

Thanks for telling us what exactly the move binary does @LT

 

Thanks for telling us what exactly the move binary does @LT

Impressed with the speed of the investigation and the detail uncovered.  Thanks, Tom and team.

Also pleased to know that this problem has very specific causes which most of us wont trip over.

Thank you all very much again. I've finished converting everything to XFS anyway. But just in reading all of LT's and others who have a vastly superior knowledge of unRAID than myself, I just wanted to ask if the cache drive should be using a different FS than BTRFS? I have read on the forums of people using XFS for it, but hadn't really seen anyone using anything but BTRFS or XFS. Since unRAID defaults to BTRFS when you add a cache drive I just assumed this was the "standard" FS for a cache drive.

Thank you all very much again. I've finished converting everything to XFS anyway. But just in reading all of LT's and others who have a vastly superior knowledge of unRAID than myself, I just wanted to ask if the cache drive should be using a different FS than BTRFS? I have read on the forums of people using XFS for it, but hadn't really seen anyone using anything but BTRFS or XFS. Since unRAID defaults to BTRFS when you add a cache drive I just assumed this was the "standard" FS for a cache drive.

 

If you're not pooling the cache I personally would stick with XFS.

To follow up on this, we added this known issue to the 6.2.1 announcement post:

 

If a system extended attribute, eg, system.posix_acl_access, is set on a file on the cache, if that file is targeted by the mover for transfer to the array, and the target array disk file system is ReiserFS, there will be an error in the system log and the file will not be moved.  This is a kernel issue that will be resolved in a future 6.2 patch release.  unRAID 6.3 does not have this issue.  This issue normally does not come up but certain docker containers, e.g., sonarr, might create the conditions where this can occur. 

 

In the 6.2.1 release then, it will not delete a file off the cache disk/pool if transferring the extended attributes to the target disk fails.  Of course this is an incomplete solution because the file is stuck on the cache disk/pool.

 

We have been in direct email correspondence with the ReiserFS maintainers at SUSE.  There was a kernel patch in 4.4 series that caused this issue.  They are now working on a new patch to fix it.  But because of the way linux kernel development works, they have to put this patch in 'mainline', where it will then find it's way to 'stable' and 'long term support' kernels.  Well the next 'merge window' for mainline was just opened a couple days ago, so there might not be any movement on this for up to 2 weeks (until merge window closes).  When the patch is finally available we will issue a 6.2.2 release.

 

The current unRAID prerelease (our 'mainline') does not have this issue.

Thank you all very much again. I've finished converting everything to XFS anyway. But just in reading all of LT's and others who have a vastly superior knowledge of unRAID than myself, I just wanted to ask if the cache drive should be using a different FS than BTRFS? I have read on the forums of people using XFS for it, but hadn't really seen anyone using anything but BTRFS or XFS. Since unRAID defaults to BTRFS when you add a cache drive I just assumed this was the "standard" FS for a cache drive.

if you have a single cache drive then either format is OK although many would recommend XFS.  If you want a cache pool (I.e. More than one cache drive) then BTRFs is the only option that supports this.

Thank you all very much again. I've finished converting everything to XFS anyway. But just in reading all of LT's and others who have a vastly superior knowledge of unRAID than myself, I just wanted to ask if the cache drive should be using a different FS than BTRFS? I have read on the forums of people using XFS for it, but hadn't really seen anyone using anything but BTRFS or XFS. Since unRAID defaults to BTRFS when you add a cache drive I just assumed this was the "standard" FS for a cache drive.

 

If you're not pooling the cache I personally would stick with XFS.

 

With all due respect, there is no reason to not use btrfs for the cache.  Long gone are the days of btrfs instability in single-device and multi-device raid-1 pools.  Sure you can use xfs for cache, but you will not be able to make it redundant.  If you don't care about the risk of your cache disk failing and losing all the data stored on it, sure use xfs.  But I don't think it offers any advantage over btrfs in the way we use it.

 

Ok, I don't want this thread to now get polluted with OT btrfs vs. xfs discussion.  If someone wants to continue that discussion perhaps we should split it off.

Thank you all very much again. I've finished converting everything to XFS anyway. But just in reading all of LT's and others who have a vastly superior knowledge of unRAID than myself, I just wanted to ask if the cache drive should be using a different FS than BTRFS? I have read on the forums of people using XFS for it, but hadn't really seen anyone using anything but BTRFS or XFS. Since unRAID defaults to BTRFS when you add a cache drive I just assumed this was the "standard" FS for a cache drive.

 

If you're not pooling the cache I personally would stick with XFS.

 

With all due respect, there is no reason to not use btrfs for the cache.  Long gone are the days of btrfs instability in single-device and multi-device raid-1 pools.  Sure you can use xfs for cache, but you will not be able to make it redundant.  If you don't care about the risk of your cache disk failing and losing all the data stored on it, sure use xfs.  But I don't think it offers any advantage over btrfs in the way we use it.

 

Ok, I don't want this thread to now get polluted with OT btrfs vs. xfs discussion.  If someone wants to continue that discussion perhaps we should split it off.

 

Fair enough. I'd bow to your superior knowledge on that one.  And I was basing my opinion purely on the fact I don't pool, don't need redundancy and sporadic reports in the past of btrfs issues, but I don't keep abreast of these things so I'd recommend the OP listens to advice from Tom rather than me.

There is a patch for this but we are going to wait until it makes it's way to linux stable release before creating a 6.2.1 release.  In mean time, those who might be affected by this issue are encouraged to install 6.3.0-rc release.