January 3, 20251 yr So I was watching a movie, then 1 hour later come back and video was missing. so I logged into unraid and all my drives are basically blank. I think I have 16 with about 8 different shares. Almost all the shares are empty. Appdata mostly empty. So I am wondering where to go from here. No one has access to shares even remotely. The only thing that has changed in the past week was I mounted an external share with plugins, and then had a docker use it. I thought I read somewhere that my shares could have been mounted wrong, like wrong mount point? I have been using unraid for about 5 years or so, I am familiar with it, but not super technical. don't know if this is a hack of some sort or a technical problem. I have rebooted the server but was able to look as some logs prior. I have since ran xfs_repair. kernel: xmrig[63657]: segfault at 1d93c0 ip 000014fbaeb1a266 sp 000014fbb0d68118 error 4 in libc.so.6[14fbaeb09000+15b000] likely on CPU 48 (core 0, socket 1) kernel: xmrig[1269]: segfault at 1d93c0 ip 000014b25ca10266 sp 000014b25ec5e118 error 4 in libc.so.6[14b25c9ff000+15b000] likely on CPU 38 (core 6, socket 0) Jan 2 22:15:37 kernel: XFS (md3p1): Metadata corruption detected at xfs_dinode_verify+0xa0/0x732 [xfs], inode 0x104c17872 dinode Jan 2 22:15:37 kernel: XFS (md3p1): Unmount and run xfs_repair Jan 2 22:15:37 kernel: XFS (md3p1): First 128 bytes of corrupted metadata buffer: Jan 2 22:15:37 kernel: 00000000: 49 4e 41 ff 03 01 00 00 00 00 00 63 00 00 00 64 INA........c...d Jan 2 22:15:37 kernel: 00000010: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 2 22:15:37 kernel: 00000020: 61 1d 9d 97 00 74 b7 9a 61 1d e2 15 0f 9d cb 5f a....t..a......_ Jan 2 22:15:37 kernel: 00000030: 64 4e ab 35 2c f4 23 23 00 00 00 00 00 00 00 0a dN.5,.##........ Jan 2 22:15:37 kernel: 00000040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 2 22:15:37 kernel: 00000050: 00 00 24 01 00 00 00 00 00 00 00 00 4f 40 66 bc ..$.........O@f. Jan 2 22:15:37 kernel: 00000060: ff ff ff ff f2 6b aa 6a 00 00 00 00 00 00 00 10 .....k.j........ Jan 2 22:15:37 kernel: 00000070: 00 00 00 01 00 3d b4 57 00 00 00 00 00 00 00 00 .....=.W........ Jan 2 22:17:35 shfs: /usr/sbin/zfs unmount 'cache/Temp' 2>&1 jpserver-diagnostics-20250103-0133.zip
January 3, 20251 yr Author I rebooted in safe mode and xfs_repair on all drives. Still the same issue. what seems odd is from the gui all drives are basically empty. If I open each drive to see contents, they also look empty. A delete would take hours to occur and it’s like I can’t find a trail. Edited January 3, 20251 yr by joshrp5
January 3, 20251 yr Community Expert 1 hour ago, joshrp5 said: I rebooted in safe mode and xfs_repair on all drives. Still the same issue. Post new diags.
January 3, 20251 yr Community Expert 2 hours ago, joshrp5 said: A delete would take hours to occur But format wouldn't
January 3, 20251 yr Community Expert 1 hour ago, trurl said: Did you install xmrig? (crypto miner) If you didn't install this crypto miner then 8 hours ago, joshrp5 said: a hack of some sort seems likely
January 3, 20251 yr Community Expert 8 hours ago, joshrp5 said: I have rebooted the server but was able to look as some logs prior Post those complete logs.
January 3, 20251 yr Author so this is diag as of now, but it shouldn't be much different than before. No xmrig installed or crypto miner, I mean I may have 3 years ago, but I don't think so. Definitely nothing in plugins or dockers or anything as I keep those pretty clean. jpserver-diagnostics-20250103-0133 2.zip
January 3, 20251 yr Community Expert 35 minutes ago, joshrp5 said: diag as of now Current diagnostics only contains syslogs from rebooting today. Do you have earlier syslogs? 35 minutes ago, joshrp5 said: No xmrig installed 10 hours ago, joshrp5 said: kernel: xmrig[63657]: segfault at 1d93c0 ip 000014fbaeb1a266 sp 000014fbb0d68118 error 4 in libc.so.6[14fbaeb09000+15b000] likely on CPU 48 (core 0, socket 1) kernel: xmrig[1269]: segfault at 1d93c0 ip 000014b25ca10266 sp 000014b25ec5e118 error 4 in libc.so.6[14b25c9ff000+15b000] likely on CPU 38 (core 6, socket 0) As you can see from those logs in your first post, xmrig was running. Are you sure your server isn't accessible from the internet? https://docs.unraid.net/unraid-os/manual/security/good-practices/
January 3, 20251 yr Author 5 minutes ago, trurl said: Current diagnostics only contains syslogs from rebooting today. Do you have earlier syslogs? As you can see from those logs in your first post, xmrig was running. Are you sure your server isn't accessible from the internet? https://docs.unraid.net/unraid-os/manual/security/good-practices/ Yes. The server is available from the internet. It is connect to Unraid connect plugin and has ngnix installed for reverse proxy. using unifi network for my firewall, etc. Crafty was installed and ngnix are about the only ports open. my the most I have prior to rebooting is what I posted originally. I wish I didn’t reboot, as I assumed it was being backed up, which is was to the array. Edited January 3, 20251 yr by joshrp5
January 3, 20251 yr Community Expert Your diagnostics shows some go.save files in the config folder of your flash drive. These were larger than the standard go file, perhaps they would give a clue.
January 3, 20251 yr Community Expert All disks are mounted now but mostly empty, someone/something deleted all the data, options basically are restoring from a backup or using a file recovery app, like UFS explorer.
January 3, 20251 yr Author 7 minutes ago, JorgeB said: All disks are mounted now but mostly empty, someone/something deleted all the data, options basically are restoring from a backup or using a file recovery app, like UFS explorer. When you say restoring from a backup, you don’t mean the flash drive backup do you? I am assuming all the data is gone otherwise. Had about 120TB of data
January 3, 20251 yr Author 6 minutes ago, JorgeB said: I mean a backup of the data. Right, like a duplicate, not some sort of snapshot
January 3, 20251 yr Community Expert Snapshots could have helped if they existed, depending on how exactly the data was deleted, they are read-only, good protection against accidental deletions or ransomware, but won't help you now.
January 3, 20251 yr Author So I will be starting from scratch and would like to halfway identify how it happened. I assume my root pw wasn’t strong enough and they ssh into it? also, should I build the array with zfs so I can do snapshots? Edited January 3, 20251 yr by joshrp5
January 3, 20251 yr Community Expert 1 hour ago, joshrp5 said: they ssh into it? DId you have that port open?
January 4, 20251 yr Community Expert 13 hours ago, joshrp5 said: also, should I build the array with zfs so I can do snapshots? You can, or btrfs, note that zfs currently has a known performance write issue when used with the array only
January 4, 20251 yr Community Expert Since you may have been hacked, and are starting over anyway, might be a good idea to reformat and reinstall flash. Make sure you have a copy of your license .key file from the config folder on flash.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.