10tb+ gone

[Lassley7]

I’m clueless I woke up today, and all my tv shows for Plex have just been wiped from my array? No sings of a bad drive, no errors being shown just poof gone. Can anyone help me pin point what is going on and if my media is really just gone? Please any help would be much appreciated 

[JonathanM]

Are the episodes already watched? Check the show profiles to see if remove after watching is set.

[Lassley7]

The shows are gone from the server off of multiple hard drives just gone and no Plex is not set to remove after watched so I’m not sure what’s going on like I said I got six disc in my server and it went through and deleted all TV shows on all disk

[Squid]

Post your diagnoatics

[Lassley7]

Can u explain how to do this also my server was reset today, so I’m not even sure if the diag would show what happened now 

poseidon-diagnostics-20211223-1508.zip

Edited December 23, 2021 by Lassley7

[Lassley7]

Could this be from a corrupted file sys? 

[Lassley7]

23 hours ago, Squid said:

Post your diagnoatics

Do you have any clue what caused this? I did a app data restore and it restored all the data from sonarr and Plex but the media files were still gone

[Squid]

Nope.  There's nothing to signify any reason for loss of data.  TBH, the usual cause of loss of data is user error, especially if there's no corruption on any disk.

[JonathanM]

On 12/23/2021 at 3:45 PM, Lassley7 said:

The shows are gone from the server off of multiple hard drives just gone and no Plex is not set to remove after watched so I’m not sure what’s going on like I said I got six disc in my server and it went through and deleted all TV shows on all disk

https://forums.unraid.net/topic/117190-file-system-audit-to-detect-file-deletion/?do=findComment&comment=1071101

 

[Lassley7]

Please someone help.. I woke up today and all the rest of my media is gone. I’m server has completely wiped itself. Please someone tell me the diag shows something that can help pin point what is happening 

[itimpi]

2 minutes ago, Lassley7 said:

Please someone help.. I woke up today and all the rest of my media is gone. I’m server has completely wiped itself. Please someone tell me the diag shows something that can help pin point what is happening 

Nothing in the diagnostics indicating an issue.   This almost suggests you are running something on the server, or something on the LAN that is actively deleting files.

 

one thing I notice is that you have the CA Backup plug-in installed.   How is that configured?   You must be careful about where you tell it to store backups as anything existing at that location will be deleted.

[Lassley7]

Would the diagnosis show if someone outside the network did this? You don’t realize I’ve had this setup for years nothing new, nothing changed. Yesterday all my tv shows were gone, now movies did the same thing now. This is crazy 10+Tb gone within 2 days when this has been running this way for years 

[Lassley7]

Here is a new diagnosis please someone atleast look and see if they can tell anything. Server hasn’t been rebooted yet last time I rebooted before doing a diag 

poseidon-diagnostics-20211225-1203.zip

[itimpi]

7 minutes ago, Lassley7 said:

Would the diagnosis show if someone outside the network did this?

No.   If the files were deleted using standard SMB calks this would not show up in the logs.

 

Do you have your server exposed to the internet in any way?     There have been reports in the past of users who do not have their servers properly secured against access from the internet being hacked and all their files deleted.   I guess another (albeit.not one I have heard happening before) is a docker container being hacked and if it has one that has volume mapping that allows write access to the array it could also delete files with nothing showing up in any log.

[Lassley7]

I posted a new log, can u please take a look when u can. Nothing has changed, I do have Plex, sonarr, radarr, sabnzbd so yes a lot of dockers connected to the internet 

[Lassley7]

Also just like tv shows not everything got deleted. It kept 3-4 shows. Now today my movies did the same thing 2,000+ movies gone 3-4 remaining 

[Lassley7]

22 minutes ago, itimpi said:

No.   If the files were deleted using standard SMB calks this would not show up in the logs.

 

Do you have your server exposed to the internet in any way?     There have been reports in the past of users who do not have their servers properly secured against access from the internet being hacked and all their files deleted.   I guess another (albeit.not one I have heard happening before) is a docker container being hacked and if it has one that has volume mapping that allows write access to the array it could also delete files with nothing showing up in any log.

File activity shows all the deleted files, started this morning. But it does not show why, plus I have recycling bin setup but it didn’t catch any of them? Please someone look at my new diagnosis and see if u can help me figure this out 

poseidon-diagnostics-20211225-1203.zip

[itimpi]

5 minutes ago, Lassley7 said:

Please someone look at my new diagnosis and see if u can help me figure this out 

I did not see anything in the diagnostics (although that does not mean nobody else will spot something).

 

5 minutes ago, Lassley7 said:

plus I have recycling bin setup but it didn’t catch any of them?

The recycle bin will only catch files that are deleted via SMB (I.e. over the network).   That suggests it is something local to the Unraid server.

[Lassley7]

5 minutes ago, itimpi said:

I did not see anything in the diagnostics (although that does not mean nobody else will spot something).

 

The recycle bin will only catch files that are deleted via SMB (I.e. over the network).   That suggests it is something local to the Unraid server.

Witch is why I’m trying to pin point what has happened, is there any docker that does file recovery? Or will I have to fry one disk at a time in a windows machine? Also as I said I posted a updated diag to see if u can atleast see what could of happened file activity shows everything being deleted just doesn’t say why 

poseidon-diagnostics-20211225-1203.zip

[JonathanM]

In your plex settings, under library, what is "allow media deletion" set to?

[Lassley7]

4 hours ago, JonathanM said:

In your plex settings, under library, what is "allow media deletion" set to?

This is set to no, I don’t allow deleting. I posted a more recent diagnosis after the movies disappeared, my first diagnosis was from when my tv shows did the previous day BUT I also shut my server down the first time before doing a diagnosis that’s why I’m hoping someone can look into the most recent one I posted above just too see if anything sticks out 

[Lassley7]

On 12/25/2021 at 12:31 PM, itimpi said:

No.   If the files were deleted using standard SMB calks this would not show up in the logs.

 

Do you have your server exposed to the internet in any way?     There have been reports in the past of users who do not have their servers properly secured against access from the internet being hacked and all their files deleted.   I guess another (albeit.not one I have heard happening before) is a docker container being hacked and if it has one that has volume mapping that allows write access to the array it could also delete files with nothing showing up in any log.

So I did some digging today, and I found that my root paths in sonarr and radarr were changed to data/movies and not set to data/Movies when u changed this back it asked if I wanted to “delete files” so I’m wondering if I’m unfortunately another victim and someone got into my containers as I did have them all running through a duckdns domain. I have disabled all remote access to my server, I am going to add a few movies tonight, and see if they get deleted if they do. Do you or anyone else suggest what else to check? Ps I also changed server password, and container passwords.