Empty files (0kb) on cache drive

[alexricher]

Good day Unraid Community,

 

About 2 days ago, my server experienced a weird behavior which left it in a problematic state. Just a bit of information before:

- Server on Unraid 6.14b

- 12 disks (38TB)

- 1 cache drive (with 2 folders on cache only: /cache/downloads & /cache/appdata) All my dockers are on there (with their config in appdata)

 

I noticed in the morning that the LimeTech WebUI lost some of my usual settings, such as my black theme. I started trying to figure out what was wrong. Saw some bread() error. I noticed the following:

 

- All content of my Cache drive was empty (downloads and appdata folders gone) and all the content got transfered from the cache drive to my Unraid protected array (/mnt/user/downloads & /mnt/user/appdata) I assume during the mover process at night.

- This in turn rendered all my dockers unusable as they lost their configuration files (in cache/appdata)

- /boot was empty.

 

After research, I understood my USB key might have died. While still running, I've removed the USB key from my Unraid server, did a checkdisk on Windows and repaired the issue. In the meantime, I took everything under (/mnt/user/downloads & /mnt/user/appdata) and copied them (not moved) back to my cache drive. So far, so good.

 

I've rebooted the server and noticed that my cache drive was almost empty, even though it shows all my files. My dockers wouldn't start properly either. It's when I noticed everything in /mnt/user/downloads & /mnt/user/appdata was of size 0kb and same for /cache...

 

I'm now wondering if anyone could help me figuring out why all my files are of size 0kb. I've tried deleting a file from /mnt/user/downloads and see if the file would suddenly have a size on my cache. No dice. Did the opposite (delete from cache and check in /mnt/user/downloads) and still same issue. How can I retrieve my files?

 

Thanks and have a great day!

[trurl]

You probably can't get them back if they are zero length. Depending on exactly how you tried to do the copy, you may have encountered the user share copy bug. It is very important to not mix disk and user shares when doing copy/move operations. This can result in a file trying to overwrite itself, causing it to be zero length.

 

The main thing you need to take care of when recreating this is to set the shares to cache-only if you intend for them to stay on the cache drive.

 

Any folder at the top level of cache or an array drive is automatically a user share. If you don't configure the share it will have default configuration.

 

What do you get with?

v /mnt/user0

[JonathanM]

This is potentially a bug. From the description, I gather that the unraid USB stick was corrupted, and subsequently the mover ignored the cache only settings on the relevant shares.

 

This would be easy enough to test. Set up a cache only share, physically pull the USB stick, and allow the mover to run. I suspect the logic for the mover queries the share config file and if it doesn't find one, it assumes it should move everything. If this is the case, it needs to be fixed pronto. The default activity should be to leave the data alone, and only move it if the share config specifies it should be moved to the array. This was not the case when the cache drive was first introduced, but things have changed with docker and vm's.

 

As for the OP's current state, I suspect the data is now gone, given the 0kb files. If the correct actions had been taken immediately after the issue occurred this would likely not be the case, but in any event, the data shouldn't have been moved off the cache drive.

[trurl]

As far as I know, the default settings for shares have never included the "cache-only" setting.

 

You can examine the settings for any share in config/shares. Each share that has been configured will have its own .cfg file there. If the setting could not be saved to the flash then that probably explains what happened as far as mover is concerned, and the user share copy bug explains why the files were truncated.

 

In any case, if your flash has been fixed, you should be able to save the cache-only setting, and then examine the share's .cfg file in config/shares to see that the setting was in fact saved.

[JonathanM]

As far as I know, the default settings for shares have never included the "cache-only" setting.

Precisely. I believe it's time to address this before 6.0 goes final.

I don't want the mover assuming it's supposed to clear out my cache/apps drive if it can't read the share config file.

[alexricher]

Good day everyone,

 

Thanks for helping out! It's greatly appreciated.

 

v /mnt/user0

 

I'll give this a try as soon as I get back home.

 

I believe what happened is when my USB key "died", Unraid acted as if there were no more *.cfg for my share and assumed "appdata" & "downloads" were meant to be moved but I sadly catched this issue once it was too late. A nice feature would be to receive some kind of notification if an issue occurs with the USB key so we can be proactive instead of reactive. From what you're saying, having copied my files from the /mnt/user to /mnt/cache might have destroyed my files?!? That would s*ck if it's the case but at least it's not my array's content. Sadly, I had tons of useful stuff on my cache drive such as all of my docker's configurations... It'll take a while to setup once again. But the worst is the files that were in downloads. Some files will be hard to get back...

 

Is there any test I can perform to confirm the files are totally gone? I remember having a similar issue a few years back when files were moved manually from a /mnt/disk# manually and if the file was onto 2 disks at the same time, the filesize would be 0kb. Once I've deleted the duplicate, the filesize would come back > 0kb. That's how I had recovered a few files in the past. Could this process be similar? Also, is there a clever way to keep your appdata on the cache drive (for perfomance) but have regular backup?

 

Thanks a lot and have yourself a great day!

[alexricher]

You probably can't get them back if they are zero length. Depending on exactly how you tried to do the copy, you may have encountered the user share copy bug. It is very important to not mix disk and user shares when doing copy/move operations. This can result in a file trying to overwrite itself, causing it to be zero length.

 

The main thing you need to take care of when recreating this is to set the shares to cache-only if you intend for them to stay on the cache drive.

 

Any folder at the top level of cache or an array drive is automatically a user share. If you don't configure the share it will have default configuration.

 

What do you get with?

v /mnt/user0

 

I've executed the command and I see a listing of my array's main folder, including appdata & downloads.

 

I confirm my shares for the cache drive were set as "Cache-only" before the USB corruption.

[trurl]

/mnt/user0 are the user shares excluding any folders/files on cache. In other words, one or more of your array drives still have appdata and downloads folders on them. Check each array drive. Maybe you got lucky and some of your data is still there. The proper way to move them back to cache is to specify disks, not user shares.