My Unraid was running perfectly. I updated to v7 earlier this week and had no issues. Today, I came home from work, and all the data on all my array devices was gone. I don't know what to do. There was 60-70Tb of data loss. The little bit of data that is there is probably from sonarr and radarr redownloading.

can someone advise what I should do?
 

unraid-diagnostics-20250116-2146.zip

Do you have your server exposed to the internet?   If so there is a good chance your system got hacked and the disks reformatted (this has happened in the past) as all array drives look like they might have been freshly formatted.     The Unraid GUI is not hardened against such attacks so its should never be directly exposed to the internet. 

 

There is also a possibility that a badly configured docker application went rogue (such as Plex) if you gave them write access to the data on the array, although I would think this less likely.

 

In terms of getting the data back a file recovery program such as UFS Explorer on Windows might be able to recover data.   It is not a free app, but there is a free option that will show you what it could recover before you have to buy a licence.

@itimpi I have a bunch of dockers that use a reverse proxy to access the Internet. Is this what you mean by exposed to the Internet? The only why I've been able to access to Unraid GUI outside of my network has been through Unraid Connect.

I'll try to recover the data according to your recommendation.

Just now, JEZBRO said:

 I have a bunch of dockers that use a reverse proxy to access the Internet. Is this what you mean by exposed to the Internet? The only why I've been able to access to Unraid GUI outside of my network has been through Unraid Connect.

I was thinking about inbound connections from the internet - not outbound ones.   Unraid Connect should be a safe way to access your server GUI from the internet side.  

 

There are 2 ways that I have seen that expose the Unraid GUI directly to the internet.

• There is a port forward in the router for the Unraid GUI or SSH port.
• The Unraid Server is in the DMZ zone of the router so that there is no barrier to accessing it directly from the internet.

It does not sound as if either of these apply to you so I a m a bit mystified    

 

Do any of your dockers have full read/write access to the main array in case one of them got compromised.

 

Another possibility is that a machine on the local LAN got compromised and that caused the damage - any chance of that having happened?   Really want to try and work out what could have caused the problem to avoid it happening again.

 

10 minutes ago, itimpi said:

I was thinking about inbound connections from the internet - not outbound ones.   Unraid Connect should be a safe way to access your server GUI from the internet side.  

 

There are 2 ways that I have seen that expose the Unraid GUI directly to the internet.

• There is a port forward in the router for the Unraid GUI or SSH port.
• The Unraid Server is in the DMZ zone of the router so that there is no barrier to accessing it directly from the internet.

It does not sound as if either of these apply to you so I a m a bit mystified    

 

Do any of your dockers have full read/write access to the main array in case one of them got compromised.

 

Another possibility is that a machine on the local LAN got compromised and that caused the damage - any chance of that having happened?   Really want to try and work out what could have caused the problem to avoid it happening again.

 

I setup my array so that there was one share called "Data" and nested in that folder were media, downloads, etc. Plex only had access to the media folder, qbit only had access to downloads, and the "arrs" had access to both because they moved completed downloads from one to the other. I did this so that the copying was instant and easy on the processor.  I followed this same method for all the other dockers I was running.

13 minutes ago, JEZBRO said:

I setup my array so that there was one share called "Data" and nested in that folder were media, downloads, etc. Plex only had access to the media folder, qbit only had access to downloads, and the "arrs" had access to both because they moved completed downloads from one to the other. I did this so that the copying was instant and easy on the processor.  I followed this same method for all the other dockers I was running.

Nothing obviously wrong with that as far as I can see.

 

Still a bit worried that no obvious cause for your problem that I can see.   Concerned that it could occur again.

1 hour ago, itimpi said:

Nothing obviously wrong with that as far as I can see.

 

Still a bit worried that no obvious cause for your problem that I can see.   Concerned that it could occur again.

Yeah, I'm gonna try and recover as much as I can and tighten security the best I can I guess. Thanks for the help though. It's gonna take 15 hours to scan for lost data. 

 Is it by chance a Plex type program where a user could have access to delete media files?

Did you forward any ports from your unraid private IP to 80 and 443?

6 hours ago, MowMdown said:

Did you forward any ports from your unraid private IP to 80 and 443?

From memory, Nginx was using port 80 and 443 and was on br0 network so had a different IP address to the GUI, Nextcloud also used port 80 and had different IP.

10 hours ago, Veah said:

 Is it by chance a Plex type program where a user could have access to delete media files?

I set it up so only I could delete media through the Plex app, but not other users.

 

I also set up Nextcloud in a way using, "adapted storage" so that it could share the media folder content with a public or private link if I wanted, but other than that Nextcloud had its own folder on the share and couldn't access anything else. 

It does look as though your drives were re-formatted, not that someone got in and just deleted the data. The reason I suspect a reformat is look at the number of reads and writes, if someone was manually deleting the data, those numbers would be in the millions/billions. It's also possible you rebooted/restarted the array and cleared the read/writes just before taking the screenshot.

 

This makes me suspect either:

 

1. Your unraid server was directly accessible to the internet by an accidental port forward.

 

2. You have a compromised machine on your network that someone as remote access to and gained access to your LAN through that device.

 

First thing I suggest, if you haven't already, is locking down your network and remove all port forwards and delete any VPNs/Reverse Proxies.

Edited January 17, 20251 yr by MowMdown

13 hours ago, MowMdown said:

It does look as though your drives were re-formatted, not that someone got in and just deleted the data. The reason I suspect a reformat is look at the number of reads and writes, if someone was manually deleting the data, those numbers would be in the millions/billions. It's also possible you rebooted/restarted the array and cleared the read/writes just before taking the screenshot.

 

This makes me suspect either:

 

1. Your unraid server was directly accessible to the internet by an accidental port forward.

 

2. You have a compromised machine on your network that someone as remote access to and gained access to your LAN through that device.

 

First thing I suggest, if you haven't already, is locking down your network and remove all port forwards and delete any VPNs/Reverse Proxies.

Yeah, that's my plan. what a pain though, it looks like it's gonna take 2-3 days to recover all the data and I'm gonna need to do some weird juggling act with the HDDs as well. I have a 1 18TB drive that I've reformatted to NTFS, then I'm going to have to move any recoverable data from 1 drive onto that, then reformat the drive I just move data from and reformat that, to move the next drive. Then at the end, I should be left with one empty HDD and 6 NTFS HDDs with all the data on it, which I'll have to rebuild the array with, one HDD at a time, adding an array drive then copying one HDD over. I'm hoping that I don't require a parity disk, because the last empty drive will become the parity disk at the end.