Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Dreaded error: Out Of Memory errors detected on your server

Featured Replies

image.thumb.png.6830b18bafdd3a28263ea97f9b9f1919.png

 

I haven't logged into my Unraid server in a month or two but I went in today and found this error above ^^. Clicking "More Information" led me to this forum post which essentially says to capture diagnostics BEFORE REBOOTING, which I did download and have attached here, and make a post under General Support.

 

My first instinct is to review my RAM memory usage, screenshot below.

image.png.88387312fbda3544d547bdcb1db2d825.png

 

And my overall storage array still have 18T of space remaining.

image.thumb.png.e476da5ff43a096a9eac0a3c87f9bf6f.png

 

Any help would be greatly appreciated as I have not read through any diagnostics before and am not sure if I would have noticed any warning signs leading up to this. I have not rebooted my server yet in case anyone who can help needs more information.

 

Thanks in advance internet friends.

bespin-diagnostics-20240201-1446.zip

  • Community Expert
Feb  1 14:38:48 Bespin root: Fix Common Problems: Warning: Possible mining software running

Is this expected?

  • Community Expert

Have you been hacked?

Jan 28 04:30:08 Bespin root: FCP Debug Log: nobody   25491 69.9 14.6 3231888 2401512 ?     Sl   Jan18 9934:36      \_ ./xmrig -o randomxmonero.eu.nicehash.com:3380 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 -cpu-max-threads-hint=100 --cpu-freq=4000
Jan 28 04:30:08 Bespin root: FCP Debug Log: nobody   14347 63.3 14.6 3231888 2399984 ?     Sl   Jan21 6200:16      \_ ./xmrig -o randomxmonero.eu.nicehash.com:3380 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 -cpu-max-threads-hint=100 --cpu-freq=4000
Jan 28 04:30:08 Bespin root: FCP Debug Log: nobody   16019 62.6 14.6 3231892 2399976 ?     Sl   Jan22 5157:40      \_ ./xmrig -o randomxmonero.eu.nicehash.com:3380 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 -cpu-max-threads-hint=100 --cpu-freq=4000
Jan 28 04:30:08 Bespin root: FCP Debug Log: nobody    6083 53.6 14.6 3231888 2399940 ?     Sl   Jan23 3868:27      \_ ./xmrig -o randomxmonero.eu.nicehash.com:3380 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 -cpu-max-threads-hint=100 --cpu-freq=4000
Jan 28 04:30:08 Bespin root: FCP Debug Log: nobody   10950 61.6 14.6 3231888 2399628 ?     Sl   Jan25 2809:17      \_ ./xmrig -o randomxmonero.eu.nicehash.com:3380 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 -cpu-max-threads-hint=100 --cpu-freq=4000
Jan 28 04:30:08 Bespin root: Fix Common Problems: Warning: Possible mining software running

 

  • Author
Quote

Have you been hacked?


I saw that error too and was planning to make a separate post about it. 
 

but any direction/suggestions on how to remove a hacker/unknown would be greatly appreciated as well. 

  • Community Expert
26 minutes ago, MissMagdalene said:

remove a hacker/unknown

Should be your first priority.

  • Author

So I search in the forums for the "Possible mining software running" but nothing came up but my post here. Broadening my search to just "mining software" I ran into this forum post. Which basically directed me to a few files to review if they were compromised or had stuff I didn't put in them, which did not seem to be the case for me.

 

I'm still unsure how or if I was indeed hacked, no other users were visibly present via the GUI or looking via console/terminal.

 

I have restarted my server and will monitor if either of these errors come up again. I have also strengthened my Unraid server user password and reviewed my port forwards on my router and those are secure to be sure, they align with the suggestions referenced by @trurl here https://docs.unraid.net/unraid-os/manual/security/.

  • Community Expert
23 minutes ago, MissMagdalene said:

still unsure how or if I was indeed hacked

If you didn't intend to have mining software on your server, then someone else put it there.

 

Do you have any docker containers you didn't install?

  • Author

 

28 minutes ago, trurl said:

Do you have any docker containers you didn't install?


No, there were no docker containers that I didn’t install myself. Unless they could be hidden from the GUI view?

 

same with VMs, none that were not installed by me. 

  • Community Expert
2 minutes ago, MissMagdalene said:

no docker containers that I didn’t install myself. Unless they could be hidden from the GUI view?

On Dockers page, move slider at upper right to Advanced View.

Run the following command and give us the output:

 

ps -auxf | grep -v grep | grep -i xmrig

 

This is what Fix Common Problems is looking for. Kudos @Squid for thinking to include this.

 

We need to go into damage control mode and figure out if they've established persistence and how.

 

Did you ever expose your Unraid server to the internet? Ever port forwarded to SSH?

  • Author
36 minutes ago, trurl said:

On Dockers page, move slider at upper right to Advanced View.

No additional containers showing under Advanced View.

 

10 minutes ago, T0rqueWr3nch said:

Run the following command and give us the output:

 

ps -auxf | grep -v grep | grep -i xmrig

 

Output reveals nothing:

root@Bespin:~# ps -auxf | grep -v grep | grep -i xmrig
root@Bespin:~# 

 

11 minutes ago, T0rqueWr3nch said:

Did you ever expose your Unraid server to the internet? Ever port forwarded to SSH?

I have never exposed SSH port forward. I have a Plex server port forwarded with the Plex default (32400) and 3 other port forwards in the 8xxx range.

 

I have a static IP if that makes any difference.

On one hand, good that the output reveals nothing, which is probably to be expected since you currently aren't running out of memory...on the other hand, now we're in this ambiguous state on if we're still compromised since persistence is always a concern.

 

Good that you've never exposed SSH. And you've never exposed your Unraid Web GUI to the internet correct? What are the other forwards to?

 

The logs show this happened on the 28th- did you have anything (Docker containers, plugins, etc.) then that you don't have now?

  • Author
12 minutes ago, T0rqueWr3nch said:

And you've never exposed your Unraid Web GUI to the internet correct? What are the other forwards to?

No I have my GUI exposed to the internet (my static IP) but I have beefed up my password at the start of this. I like to check things from anywhere is the reason.

 

The other port forwards are to Tautulli and Deluge dockers, the GUIs. EDIT: And Krusader, I forgot.

 

12 minutes ago, T0rqueWr3nch said:

The logs show this happened on the 28th- did you have anything (Docker containers, plugins, etc.) then that you don't have now?

None that were added or removed by me, again I hadn't logged into my Unraid until authoring this post, so Feb 1st. Not sure if the logs would provide entries for anything added or removed in that window of time?

 

There was a lot of scanning happening on my Plex server coming from Deluge downloads recently (and from me moving download locations via Deluge GUI), but I can't remember how far back that was, it feels like well before 01/28 but maybe I'm wrong. Reviewing my Deluge activity I'm seeing now that the 'Date Added' in my Deluge GUI is incorrect. Some entries date added shows 12/01/2024...So I'm not sure if I can identify what was being downloaded/seeded/moved on that date.

Edited by MissMagdalene
Forgot about Krusader port forward. That totals the 3 I mentioned

  • Community Expert
4 minutes ago, MissMagdalene said:

I have my GUI exposed to the internet (my static IP)

Looks like your server has a LAN IP address so not clear what you mean.

 

In any case, you should use Wireguard (builtin) or other VPN, or Unraid Connect, to access your server remotely.

  • Author

Woke up to the mining software error again this morning. Ran the grep for xmrig and found something this time.

 

root@Bespin:~# ps -auxf | grep -v grep | grep -i xmrig
nobody   18199  292 14.7 3231768 2404120 ?     Sl   Feb03 4047:15      \_ ./xmrig -o 167.88.169.6:3333 -u NHbW7fJpRFQxQU8nFHP72FC6zCoDA714Xgg6.15 -k --nicehash --coin monero -a rx/0 --cpu-max-threads-hint=100 --cpu-freq=4000

 

No unexpected dockers running in the advanced view via GUI.

Edited by MissMagdalene
added details about docker

  • Community Expert

Diagnostics shows stock go file, and no /boot/extra folder. So either the OS has been hacked, or one of your plugins, dockers, or VMs had been hacked.

 

Disable Docker and VM Manager in Settings, then reboot in SAFE mode. This will insure no dockers, VMs, or plugins are running. Then check for xmrig.

 

Let it run like that for a while and keep checking for xmrig.

 

If it doesn't come back, then one of your plugins, dockers, or VMs has been hacked and is causing it.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.