30tb of files disappeared/deleted!

[pras1011]

HELP!

 

I have one share and inside it I have folders and video files. All the video files outside of the folders have been deleted!!!!

 

Is there a way to get them back???

 

I am not sure how this happened but I have lost 30tb of data. 

 

Where do I start to workout what went wrong?

Edited April 13, 2023 by pras1011

[apandey]

Post diagnostics to start with

[pras1011]

Please see attached. 

unraid-diagnostics-20230414-0642.zip

[Frank1940]

Look at your MAIN Tab on the GUI.   In the Array Devices section, look at the totals on the bottom line ("Array of three devices").   Does this total reflect the loss of 30TB of data?

[pras1011]

Yes it does. 

[Frank1940]

22 minutes ago, pras1011 said:

Yes it does. 

That eliminates the possibility that the files were moved inside of another folder.

 

I observed that you had the only share (F---s) set to Secure.  This implies that you allow everyone read access to that share.  Who has read-write access to the share?  Are these persons trustworthy?  Are they prone to occasional 'OH, shit' moments?   (As you are aware, it takes many hours to upload 30TB of data.  What you may not realize that deleting those 30TB takes a fraction of a second to a couple minutes depending on the number of files involved!)   IF the files were deleted, What OS might have been used?  (Windows10 will ask if you mean to delete the files from the server but the default choice is 'Yes'.)

[JonathanM]

Media downloaders and players have been known to delete files in certain circumstances. Misconfiguration and exposure to WAN without proper security come to mind. For instance, Plex has a setting to delete files after they have been viewed. If that is set, all files with a viewed status will be deleted during routine file maintenance.

[pras1011]

I had the Film share on Public and I then changed to Secure after this happened. I have not added any User profiles as I will just manually switch from Secure to Public when I want to write to the server. 

 

I am using a Windows 11 to write to the share. 

 

The server is at home and I trust everyone. Today, I have changed passwords, disable upnp, disable remote access for router. 

 

I have a Zidoo player but I have never had this problem in 10 plus years I have had Unraid. 

 

I assume there are no clues in the diagnostic file regarding this?

 

An odd thing was that one film had been copied from the server to the desktop yesterday evening. I did not do this. 

[Frank1940]

23 minutes ago, pras1011 said:

An odd thing was that one film had been copied from the server to the desktop yesterday evening. I did not do this. 

 

Did you have the folder (that you found this film in) shared on your Windows computer?  Did you (or any other program) access this file for any reason?  (The question is not a requisitioning of "Did you move it?"  More on the lines of "Was it played?  Etc, etc..")   Any grandchildren visited lately?  (I have a five year old great G-Kid and he loves all things electronic!!!)

 

I checked and you have a 2.5Gb NIC and it is connecting at that speed.  To transfer 30TB of data would have taken a minimum of 33 hours---if I didn't screw up the exponents. 

 

EDIT:  Just realized you said Desktop.  I don't think you can share the desktop unless you allow remote access to your computer. 

Edited April 14, 2023 by Frank1940

[pras1011]

No one touches my stuff but me. Lol. 

 

I didn't backup the 30tb. Obviously. Lol. 

 

I am just wondering if my computer/router has been hacked.

 

I am scratching my head on this one.

 

Surely I could have not deleted 30tb of data so easily. Its not quick to delete one file on the Zidoo let alone 1000.

[Frank1940]

56 minutes ago, pras1011 said:

I had the Film share on Public and I then changed to Secure after this happened. I have not added any User profiles as I will just manually switch from Secure to Public when I want to write to the server. 

 

Have a look here for another way to do this:

 

     https://forums.unraid.net/topic/58374-secure-writing-strategy-for-unraid-server-using-write-once-read-many-mode

 

 

Also here is a link to securing things with much more control:

 

     https://forums.unraid.net/topic/110580-security-is-not-a-dirty-word-unraid-windows-10-smb-setup/

 

25 minutes ago, pras1011 said:

Surely I could have not deleted 30tb of data so easily. Its not quick to delete one file on the Zidoo let alone 1000.

 

I don't know about the Zidoo but Win10 can delete 21GB consisting of 628 folders and 10,227 files in less than two minutes.  It is the number of files not their size that determines how long it takes.  Deleting a file only requires changing a few bytes in the file allocation tables.  The data itself is not touched until the disk space is reallocated for use by another file.  You might be able to recover a portion of the files (to possibly all) with an undelete program designed to work with the files system you have on your data disks.  Google for details.  (My guess is that you would to get/make a linux bootable OS USB drive and install the undelete program on that disk.  Then booting that USB in your Unraid server, you would mount the data disks and see what you could recover with the undelete program.)  

Edited April 14, 2023 by Frank1940

[pras1011]

Thanks for this.

 

I just remembered that to transfer files from the PC to the Zidoo, the PC needs to have SMB1 enabled.

[pras1011]

If I set the security to Private and create a user with R/W access and map the share on Windows 11, I assume this still isnt safe?

[itimpi]

2 hours ago, pras1011 said:

If I set the security to Private and create a user with R/W access and map the share on Windows 11, I assume this still isnt safe?

You are not safe if the Windows 11 system got compromised as it has full access to the data.   You WOULD be safe against other systems who do not have this username/password being able to change files unless the Unraid server itself got compromised

[Frank1940]

1 hour ago, pras1011 said:

If I set the security to Private and create a user with R/W access and map the share on Windows 11, I assume this still isnt safe?

Remember that a 'user' is a set of rules that are defined on the server.  Anyone who knows the password and has access to a client computer on the network can gain the privileges those rules allow by logging in.  Plus, I would assume that more than 90% of all Windows client computers automatically log unto the server when the user signs onto the client computer.

 

Any user (and the computer associated with that user) has read-write access to your Private share with R/W access.    If the person who logged into the computer walks away (without logging out) and another person comes by, that person sit down at the computer and can do as he wants to the share and its files. 

 

If the person using that computer downloads a malware to that computer, that malware has the same privileges as the user.  (It is actually the computer and any process running on that computer that has the privileges...)  (This is the one that gives most system IT people nightmares.  Today, one wrong click of the mouse can bring down entire organizations for days and days!)  Your mapped share is probably the first thing that malware would be looking for.  Encrypting and locking out the receptionist in the front lobby from her own files on just her computer would be a trivial event in the life of an IT person.  It is the encryption of data on those mapped drives that is the real nightmare!

 

You can not avoid having some files being read-write if a computer is going to be useful.  You don't want those files to be the only copies that exist on your system.  You want backups of those files that are read-only.  If you can't make backups of only copies (thinking of your videos), you want those to be read only.  You have to have a backup scheme that makes sense from a data loss standpoint--- Is once month a enough protection, once a week, or daily. 

 

If you have files that are files that are irreplaceable, you need to provide for off-site storage of those files.   You never want only a single copy of anything that is irreplaceable regardless of how much parity protection that only copy has. 

 

Any storage scheme is 'Safe' until there is an 'OH, shit' event that a tired someone has at 3:00AM.  Or a thoughtless click on a link in an E-mail from an old friend.  Or the latest update to a program has a data loosing bug in it.  (Think this doesn't happen, MS did this with a WIN10 semi-annual update that luckily was caught early in a staged release.  Luckily, I say if you weren't a victim!)

[pras1011]

The big question is, has my server/pc been compromised and how do I tell and how to remove the compromise?!?!

[Frank1940]

Do you access your server over the Internet?

 

Have you opened any ports on your router?   Google    GRC shields up   and you should find the Gibson scanning site to see what access the outside world has to your LAN. 

 

Google   malware detection software   for detection software to run on your client computers. 

 

Google about possible issues with that video player if it has write access to your server. 

 

From your Diagnostics file, Docker was disabled so there can't be a problem from that standpoint.   I don't believe there are any plugins left that access the general Internet. 

 

What is surprising to me is that you have a single share.  From your description, you had all your video files in the root of that share.  These files all disappeared.  Apparently, there are other folders in that same share.  Yet, they were untouched???.       This does not seem like malware...

[pras1011]

And the folders themselves have videos in them. 3 empty folders were deleted as well. 

[Frank1940]

Have a look at this reply in another thread for a possible way to recover your lost files:

 

        https://forums.unraid.net/topic/137933-help-i-accidentally-deleted-a-turbo-tax-file/#comment-1252417

 

If you have questions you want to ask of JonathanM, you can 'ping' him in a reply in this thread by typing an '@' and then the letters of 'JonathanM' until you see his name in the list.  Click on the list name and that will setup a ping to him. 

[pras1011]

I still have no idea how I lost all of that data but I have strengthened the server security settings!