Hi all,

 

Server has been rock solid for 3 years, never crashed once.  In the span of about 4 weeks, it has now crashed 4 times.  Once crashed webGUI becomes unresponsive, unable to access shares, but dockers are still running, and can SSH into box.  All attempts to shut down the system via command line fail and I have to power cycle the server, although it's entirely possible I'm not doing it correctly via cmd line.

 

It just crashed again this afternoon, and here is the relevant portion of the syslog which I pulled before power cycling the server.

 

Nov 16 14:09:28 Landfill kernel: general protection fault: 0000 [#1] PREEMPT SMP 
Nov 16 14:09:28 Landfill kernel: Modules linked in: xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 ebtable_filter ebtables kvm_intel kvm vhost_net vhost macvtap macvlan tun xt_nat veth ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_nat_ipv4 iptable_filter ip_tables nf_nat md_mod i2c_i801 ahci libahci r8169 mii [last unloaded: md_mod]
Nov 16 14:09:28 Landfill kernel: CPU: 1 PID: 13845 Comm: shfs Not tainted 4.1.7-unRAID #3
Nov 16 14:09:28 Landfill kernel: Hardware name: System manufacturer System Product Name/P8H77-I, BIOS 0904 10/15/2012
Nov 16 14:09:28 Landfill kernel: task: ffff88002ce3baa0 ti: ffff880006fd0000 task.ti: ffff880006fd0000
Nov 16 14:09:28 Landfill kernel: RIP: 0010:[<ffffffff811535ac>]  [<ffffffff811535ac>] __discard_prealloc+0x98/0xb3
Nov 16 14:09:28 Landfill kernel: RSP: 0018:ffff880006fd3bb8  EFLAGS: 00010246
Nov 16 14:09:28 Landfill kernel: RAX: ffff88011a2af468 RBX: ffff88011a2af440 RCX: d1e528b18ebe2731
Nov 16 14:09:28 Landfill kernel: RDX: 1b1530a073d5c084 RSI: ffff88011a2af440 RDI: ffff880006fd3d00
Nov 16 14:09:28 Landfill kernel: RBP: ffff880006fd3be8 R08: 0000000000001199 R09: 000000000003ad6e
Nov 16 14:09:28 Landfill kernel: R10: 00000000ffffffff R11: ffff880153d82e38 R12: ffff880006fd3d00
Nov 16 14:09:28 Landfill kernel: R13: ffff88011a2af4e0 R14: ffff880006fd3d00 R15: 00000000aec273c8
Nov 16 14:09:28 Landfill kernel: FS:  00002b7615618700(0000) GS:ffff88021fa80000(0000) knlGS:0000000000000000
Nov 16 14:09:28 Landfill kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 16 14:09:28 Landfill kernel: CR2: 00000000006fb808 CR3: 0000000208e3a000 CR4: 00000000001406e0
Nov 16 14:09:28 Landfill kernel: Stack:
Nov 16 14:09:28 Landfill kernel: ffff88021416b200 ffff880006fd3d00 ffffc90001599000 ffffc900015b91e8
Nov 16 14:09:28 Landfill kernel: ffff880006fd3d00 ffff880214148800 ffff880006fd3c18 ffffffff8115362b
Nov 16 14:09:28 Landfill kernel: ffff880006fd3d00 ffff88002ce3baa0 ffffc90001599000 ffffc90001599000
Nov 16 14:09:28 Landfill kernel: Call Trace:
Nov 16 14:09:28 Landfill kernel: [<ffffffff8115362b>] reiserfs_discard_all_prealloc+0x44/0x4e
Nov 16 14:09:28 Landfill kernel: [<ffffffff8116fe6c>] do_journal_end+0x4e7/0xc78
Nov 16 14:09:28 Landfill kernel: [<ffffffff81170b5c>] journal_end+0xae/0xb6
Nov 16 14:09:28 Landfill kernel: [<ffffffff81161249>] reiserfs_dirty_inode+0x6c/0x7c
Nov 16 14:09:28 Landfill kernel: [<ffffffff8104d390>] ? ns_capable+0x3a/0x4f
Nov 16 14:09:28 Landfill kernel: [<ffffffff8111cb25>] __mark_inode_dirty+0x2f/0x265
Nov 16 14:09:28 Landfill kernel: [<ffffffff8115cfd7>] reiserfs_setattr+0x262/0x297
Nov 16 14:09:28 Landfill kernel: [<ffffffff810ff408>] ? __sb_start_write+0x9a/0xce
Nov 16 14:09:28 Landfill kernel: [<ffffffff8111367a>] notify_change+0x1db/0x2dc
Nov 16 14:09:28 Landfill kernel: [<ffffffff811164d8>] ? __mnt_want_write+0x4a/0x61
Nov 16 14:09:28 Landfill kernel: [<ffffffff810fbe72>] chmod_common+0x7a/0x102
Nov 16 14:09:28 Landfill kernel: [<ffffffff810fcd75>] SyS_fchmodat+0x3f/0x77
Nov 16 14:09:28 Landfill kernel: [<ffffffff810fcdc1>] SyS_chmod+0x14/0x16
Nov 16 14:09:28 Landfill kernel: [<ffffffff815f562e>] system_call_fastpath+0x12/0x71
Nov 16 14:09:28 Landfill kernel: Code: 1c 75 bb 0f 0b 85 c0 74 12 48 8b 93 e8 00 00 00 4c 89 ee 4c 89 e7 e8 be 6e 00 00 48 8b 4b 28 44 89 7b 1c 48 8d 43 28 48 8b 53 30 <48> 89 51 08 48 89 0a 48 89 43 28 48 89 43 30 58 5b 41 5c 41 5d 
Nov 16 14:09:28 Landfill kernel: RIP  [<ffffffff811535ac>] __discard_prealloc+0x98/0xb3
Nov 16 14:09:28 Landfill kernel: RSP <ffff880006fd3bb8>
Nov 16 14:09:28 Landfill kernel: ---[ end trace 9726fa9b66052d85 ]---

 

There is nothing in the log prior that appears to be related to the issue.  Nothing with the server has changed recently, hardware-wise, and software-wise the only differences in the last few months would be updated plugins and/or dockers.  No new plugins or dockers installed in at least 3 months.

 

Possible RAM issue?  I'll run a memtest overnight.  Beyond that I'm at a loss as far as troubleshooting Linux kernel panics.  Any help would be appreciated.

Hello,

 

I have been having similar issues but failed to get the syslog before hard rebooting (no other method to reboot, just sits at command line not doing anything). I do remember seeing the "PREEMPT SMP" message though in the syslog at some point (idiotic that i didn't save it before power cycling). I have also been using unraid successfully for a number of years without issue. Since moving to 6.x a month or so ago I have had multiple crashes, once about every week or 1.5 weeks. Last was Dec 3rd, then again today.

 

I was a little hesitant in upgrading to 6.x as I really prefer my file server be simply be a file server and do one and one thing really well, which unraid has for years. I upgraded anyway with absolutely no intention of using docker or any of the virtual machine stuff as I have a separate xen server for that.

 

FYI, server is on UPS and has no power issues.

 

Anyone have any thoughts on the crashes? Sanchez have you figured anything out or resolved the issue?

 

Thanks

Hello,

 

I have been having similar issues but failed to get the syslog before hard rebooting (no other method to reboot, just sits at command line not doing anything). I do remember seeing the "PREEMPT SMP" message though in the syslog at some point (idiotic that i didn't save it before power cycling). I have also been using unraid successfully for a number of years without issue. Since moving to 6.x a month or so ago I have had multiple crashes, once about every week or 1.5 weeks. Last was Dec 3rd, then again today.

 

I was a little hesitant in upgrading to 6.x as I really prefer my file server be simply be a file server and do one and one thing really well, which unraid has for years. I upgraded anyway with absolutely no intention of using docker or any of the virtual machine stuff as I have a separate xen server for that.

 

FYI, server is on UPS and has no power issues.

 

Anyone have any thoughts on the crashes? Sanchez have you figured anything out or resolved the issue?

 

Thanks

Memtest

Memtest

 

Hello, just completed 4 total passes (2 normal, 2 force smp) with 0 errors on all 4 tests. Any other thoughts?

 

Thanks!

I haven't had the crash since I removed the cachedirs plugin.  It looks like that's what was causing it for me.

Unfortunately I don't have that plugin installed, just the UPS "Powerdown Package".

I've been experiencing the same crashes. Same type of error in the syslog just before things start to hang. Same hardware that was running fine on 5.05, once I upgraded to 6.1.3 these crashes tend to happen every couple of weeks.

I've got Sabnzbd, sickrage and transmission dockers running. Also running a SAS2LP card in my system, been reading that it might have other issues besides the slow parity check speeds under 6.x, not sure if it's related to this though.

 

Already done several passes of memtest with no errors.

What filesystems does everyone on this thread have? I know there have been some who have reported they stopped having these issues after they converted all ReiserFS to XFS. And others who report they never have these problem despite keeping ReiserFS. I personally converted everything to XFS except for btrfs cache pretty early and my experience has been that v6 is more stable than v5 ever was.

In my system all array drives are ReiserFS, cache drive is XFS.