Data loss on one of the disks

[dangitzin]

I happened to glance at the dashboard and noticed one of my disks only had ~3% utilization and the other ~30%. I have 2 6tb disks in my array and with high-water selected, they were supposed to be close in how much space was used up -- about 42% in total. I thought it was an error so I checked my plex for a random movie and it couldn't find the file. Then I decide to rescan my library and saw my movies library drop in count. I hopped on the computer to check radarr with to check the missing files and sure enough, the status' went from "Downloaded" to "Missing". Not just my movies folder, but I guess most of everything on disk 2 was deleted. Luckily my server is only 1 week old and I haven't accumulated that much and I have some of the media on my PC's hard drives still and all my important things are in another share/drive/extrernals/clouds (basically 3-2-1 method of backups). 

Not sure what happened. Then I noticed my Disk 1 went from 2.19TB down to 1.73TB so I stopped my array in the mean time so I can ask here if there's anything I can do or am I just SOL? Like I said, luckily it's all just media data that deleted but it's going to suck getting everything back that I didn't already have since I can only use public trackers. I had hardlinks setup properly too, but I guess all those disappeared as well.

Edited June 6, 2021 by dangitzin
accidental submission before finishing

[ChatNoir]

Files don't just disappear. Something is probably deleting the files.

 

Do you have something like Radarr that looks for a better version of a file ?

[dangitzin]

I had Radarr, Sonarr, and deluge running. Deluge was only seeding and I un-monitor any titles that have completed series or I’m satisfied with the quality but those too were deleted. Anything to that has to do with deletion I unchecked in the arr’s and plex. And even if I had something still monitored for upgrade, it wouldn’t get deleted until the replacement finished downloading then imported. 
In UFS Explorer, I see the data that wasn’t deleted so I’m running a scan right now. 
You’re right though, files don’t just disappear and something deleted those files. I just don’t know what caused it. Since my server is new and I’m only interested in having this as a media server, I don’t have much installed or running.

Bubuntex/Nordpvn, binhex-deluge, hotio-jackett/Radarr/Sonarr, Rebuild-DNDC and plex are the only things that were running at the time. 

[Squid]

Post ze diagnostics

[dangitzin]

Is it okay to post it as this zip? Or is there a specific thing I need to post?

miku-diagnostics-20210605-2250.zip

[Squid]

2 hours ago, dangitzin said:

UFS Explorer,

You really shouldn't be doing that on an active drive within the parity protected array.  Double check that parity is still ok.

 

But, there's nothing within the logs to explain what's happened.  Any chance you've got a folder within the share named "Movies" and another named "movies"  via SMB, only one would show up.

[dangitzin]

I've stopped my array and pulled the drive out to check. Is that still a no-no? 

As for my share, I have "data" then subfolder "media" and inside it are "anime" "movies" "tv" with no repeating subfolders with different capitalization in each respective subfolder, just the folders of titles.

[trurl]

1 hour ago, dangitzin said:

I've stopped my array and pulled the drive out to check. Is that still a no-no? 

Yes, a no-no. The disk can't be put back in the array without rebuilding it.

 

You should post new diagnostics

[dangitzin]

miku-diagnostics-20210607-1236.zip

[trurl]

Shouldn't cause any problems except wasted space, but your system share has files on disk1, probably duplicate docker or libvirt img created when the specified pool didn't exist for that share.

 

I agree it seems likely that something deleted those files.

 

I'm a little unclear about what exactly you have done while trying to fix this yourself though. You mention UFS Explorer. Was that used on the disk while it was still in the array? Then you mention removing the disk, but Diagnostics doesn't indicate any disk removed. It seems likely that you have invalidated parity at some point in all this.

[dangitzin]

The first diagnostic was when I was asked to post it and I haven't done anything yet. Then I removed the disk2 two to dock it into my PC to use UFS. The second diagnostic I started the array without my disk2 and I believe that's when I invalidated my array. 

I've spent the last couple days transferring my files back into the array and I'll just have to re-download the rest. Is there anything I can do to prevent this from happening again? Like a way to to clean up my docker with items that are no longer showing but are actually still in /appdata - would a simple smb delete do?

[dangitzin]

Ok so it's definitely either sonarr or radarr doing it. I'm back to loss I had on Saturday. Movies were the first thing I transferred over and I hit update all so the titles could catch the files. Everything was fine there and saw everything I had. Just now my I did the same with sonarr, except while it caught some titles at first, I saw my file size shrink back down again. I immediately stopped both containers, and well I'm back to the same loss I had on Saturday. I don't have anything that indicates deletion in either container's settings but when clicking on the info icon of a file it states Sonarr/Radarr was unable to find the file on disk so it was removed. Maybe something's wrong with my paths? But I haven't changed anything since I started and followed ibracorp/trash-guides structure ----> /data/media/tv, movie, etc. I've clicked update all before without issue, but this happens now? I'm guessing this is exactly what happened when I lost data on Saturday.