Kernel Panic when newznab starts running

[mp328]

Running 5.0.5 along with the usual plugins (sab,sb and newznab) for years here, without issues.

two weeks ago i received a kernel panic alarm with unraid crashing instantly. after several restarts i have narrowed it down to newznab being the culprit.

if i execute the update scripts, it runs for a bit and goes haywire with the telnet session being killed and unraid coming up with kernel panic alarm.

finally figured out how to tail the log and record what was happening. 

 

i have backups of everything going months/years back. but not sure what to restore. tried some of the files on the flash dive, without success.

if i leave newznab update scripts off, there are no issues.

 

ive searched the forums and kernel panic seems to be related to drives and memory.. i did the memtest overnight, without any errors. my monthly parity check also completed without issues, while newznab is off.

 

here is the syslog, hopefully someone sees something that i am missing.

 

http://pastebin.com/raw.php?i=rBuW6WKC

 

kernel panic happens from

Oct 11 14:44:55 in the syslog above

[Joe L.]

Quite easy to explain...

 

You have

Oct 11 14:35:20 unRaid kernel: 8068MB HIGHMEM available. (System)

Oct 11 14:35:20 unRaid kernel: 891MB LOWMEM available. (System)

 

You have too many processes running for the amount of memory. (most likely, it is the "low-memory" that you have exhausted.)

When this occurs, the kernel attempts to free memory by killing off processes it thinks are idle the longest.  Eventually, even that does not work to free enough.  Best solution, upgrade to the 64 bit unRAID where you do not have a limit on "low-memory" (although it will require you to migrate to all 64 bit add-ons, as none of your existing addons will be compatible) Good news, most popular plugins have equivalents as 64 bit versions.

 

Since you've run for years, it is possible you've recently installed a new version of newsnab that either needs more memory, or has a memory leak (allocating memory, but not freeing it) or you are downloading a HUGE media file and it is using more memory than usual.

 

 

Excerpts from your syslog:

Oct 11 14:44:55 unRaid kernel: smartctl invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=-1000

Oct 11 14:44:55 unRaid kernel: Pid: 9650, comm: smartctl Tainted: G O 3.9.11p-unRAID #5

Oct 11 14:44:55 unRaid kernel: Call Trace:

Oct 11 14:44:55 unRaid kernel: [] dump_header+0x5a/0x199

Oct 11 14:44:55 unRaid kernel: [] ? ___ratelimit+0xb4/0xc8

Oct 11 14:44:55 unRaid kernel: [] oom_kill_process+0x4f/0x2c2

Oct 11 14:44:55 unRaid kernel: [] ? has_ns_capability_noaudit+0x18/0x1f

Oct 11 14:44:55 unRaid kernel: [] ? oom_badness+0xd3/0xdd

Oct 11 14:44:55 unRaid kernel: [] out_of_memory+0x1f1/0x235

Oct 11 14:44:55 unRaid kernel: [] __alloc_pages_nodemask+0x491/0x52f

Oct 11 14:44:55 unRaid kernel: [] do_anonymous_page+0x107/0x220

Oct 11 14:44:55 unRaid kernel: [] handle_pte_fault+0xa5/0x2a9

Oct 11 14:44:55 unRaid kernel: [] handle_mm_fault+0x129/0x138

Oct 11 14:44:55 unRaid kernel: [] __do_page_fault+0x34b/0x366

Oct 11 14:44:55 unRaid kernel: [] ? sys_brk+0xfb/0x12e

Oct 11 14:44:55 unRaid kernel: [] ? __do_page_fault+0x366/0x366

Oct 11 14:44:55 unRaid kernel: [] do_page_fault+0x8/0xd

Oct 11 14:44:55 unRaid kernel: [] error_code+0x5a/0x60

Oct 11 14:44:55 unRaid avahi-daemon[952]: Disconnected from D-Bus, exiting.

 

Oct 11 14:44:55 unRaid kernel: Out of memory: Kill process 1209 (dbus-daemon) score 0 or sacrifice child

Oct 11 14:44:55 unRaid kernel: Killed process 1209 (dbus-daemon) total-vm:3368kB, anon-rss:152kB, file-rss:464kB

Oct 11 14:44:55 unRaid kernel: smartctl invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=-1000

Oct 11 14:44:55 unRaid kernel: Pid: 9650, comm: smartctl Tainted: G O 3.9.11p-unRAID #5

 

Oct 11 14:44:55 unRaid kernel: Out of memory: Kill process 1171 (rpc.portmap) score 0 or sacrifice child

Oct 11 14:44:55 unRaid kernel: Killed process 1171 (rpc.portmap) total-vm:2852kB, anon-rss:80kB, file-rss:400kB

Oct 11 14:44:56 unRaid kernel: smartctl invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=-1000

Oct 11 14:44:56 unRaid kernel: Pid: 9645, comm: smartctl Tainted: G O 3.9.11p-unRAID #5

 

[mp328]

Tnx Joe as usual 

 

i've seen the warnings and saw those errors, so i kinda figured thats what was hapenning. and i think it did all start with the latest newznab svn update... if i only knew how to go back.

 

i'm looking t the 6 beta... but with all this talk about virtualization, i am lost... i think i need to read up on it a bit first... before switching.

[sashasuman]

I believe I had the same issue. Everything showed running but not accessible from web or clients. Tried various steps but in the end only deleting everything and restarting fixed it. I hope it's not every reboot.

[mp328]

It was a corrupt newznab database in my case. Had to go back 2-3 months from my backups until it ran stable again.

Now I'm not updating anything that has to do with the dynamix stuff and I'm good for the last month or so. Making daily backups of the database. As a safety.