[Solved] Anyway to diagnose Running out of Memory?


Recommended Posts

My server seems to run completely out of ram randomly and isn't accessible even from the console.

 

Is there a way to have the logs write to the Flash or Cache so I can do a proper post-mortem?

 

Server went 163 Days uptime on Unraid 5.0.5 without issues.

 

Unraid 6.0.1

BIOSTAR NM70I-847 1.1g Celeron

4G DDR3 1333

1x SYBA SI-PEX40064

1x 3tb cache

1x 2tb data

2x 1tb data

3x 500g data

1x 500g cache

 

Dockers:

Sabnzbd

Sickbeard

Couchpotato

Apache

Smokeping

Crashplan

HTPC-Manager

Transmission

 

VMs... after I buy more RAM.

 

Link to comment

RobJ...

 

Again... can't access device in any way. So SSH, No HTTP, No SAMBA, No NFS, No Dockers...

 

I'll I've got is a bunch of gibbish on my monitor... and no keyboard commands will do anything.

 

Any change of getting logs to write to a disk, that way I'd have something to look at when it comes back up?

Link to comment

Just trying to understand, you said above that you saw the Dashboard?  How did you get there?  And at some point you saw memory info, including 'out-of-memory' errors or something like that?  How did you see that?

 

I can't give you any instructions when in one sentence you say you saw the DashBoard and something that indicated out of memory, but in other sentences you indicate all you see is gibberish and you can't access the machine in any way.  Help!?!?  Do you mean it used to work but now does not, perhaps hardware is failing?

 

A last question, when you updated to v6 did you use the Upgrading to UnRAID v6 guide?

Link to comment

I don't see any memory issues in this syslog, just the GPF which *could* be memory related, but no direct clue.  You have 4GB which should be enough, even for that many heavy duty dockers.  What amazes me is trying to run all that on a 1.1GHz CPU!  Memory usage is tricky in Linux.  Often what looks like all RAM used is just the caching using every available byte, but it freely gives it up to any program needing it.  What error messages did you get that indicated out of memory (and where were they)?

 

The GPF happens 5 seconds into the Mover, so probably related to *something* there, but no syslog clue as to why it happened.

 

I still think a long memory test is a good idea.  The board seems fairly modern, with 8 6gbps SATA ports, but the BIOS is from 2012.  You might check for an update.

Link to comment

Found it.

 

Ran MEMTEST and it threw a TON of errors. I opened my case and found a TINY paint chip wedged into the contacts, Cleaned it out, ran a 12hour MEMTEST overnight with 0 errors.

 

Robj:

CPU is Dual Core Sandy Bridge Based. No other BIOS available either.

 

Those apps are not CPU intensive except for the unrar when sab is done downloading. Since it's all automated it doesn't matter if it takes longer to do that operation as I'm not sitting waiting for it.

The vast majority of my UnRaid box is to serve files which it does using very little CPU. Parity checks average 100 megs a second and with the cache drive most writes are limited by WIFI during the day.

 

The great benefit to that CPU is that my unraid box draws 20watts with the disks spun down and around 80watts during a parity check.

 

 

Link to comment

Found it.

 

Ran MEMTEST and it threw a TON of errors. I opened my case and found a TINY paint chip wedged into the contacts, Cleaned it out, ran a 12hour MEMTEST overnight with 0 errors.

Excellent!

 

Robj:

CPU is Dual Core Sandy Bridge Based. No other BIOS available either.

 

Those apps are not CPU intensive except for the unrar when sab is done downloading. Since it's all automated it doesn't matter if it takes longer to do that operation as I'm not sitting waiting for it.

The vast majority of my UnRaid box is to serve files which it does using very little CPU. Parity checks average 100 megs a second and with the cache drive most writes are limited by WIFI during the day.

 

The great benefit to that CPU is that my unraid box draws 20watts with the disks spun down and around 80watts during a parity check.

Very good!  Because I don't know the user I'm responding to, or their experience level, I try to point out anything that appears anomalous, unusual, just in case ...  In your case, this was a studied decision, to take advantage of the power savings.  Great!  Wish I had better power usage...

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.