I am using unraid to offload some other servers.   I am moving bulk data over to it via rsync to a mounted NFS export on the unraid side.   3 data disks, single parity.   twice now I have had some sort of kernel panic and all file services stop.   I can't do a clean shutdown, so I reboot into a parity rebuild.   

 

this looks like the start of the failures.   segfault, followed by a bunch of the FPDMA QUEUED errors.

 

Aug  4 02:15:01 unraid01 kernel: monitor[17608]: segfault at 16573bc5f258 ip 000014574019ba8d sp 00007ffd40268920 error 4 in ld-2.29.so[145740190000+20000]
Aug  4 02:15:01 unraid01 kernel: Code: 49 8b 43 68 49 83 bb f8 00 00 00 00 48 8b 40 08 48 89 85 40 ff ff ff 0f 84 b0 0c 00 00 45 85 ed 74 5b 49 8b 43 58 48 8b 40 08 <48> 8b 50 08 48 85 d2 0f 85 36 0d 00 00 44 8b 8d 3c ff ff ff 4c 89
Aug  4 02:15:01 unraid01 crond[1603]: exit status 139 from user root /usr/local/emhttp/plugins/dynamix/scripts/monitor &> /dev/null
Aug  4 03:00:56 unraid01 kernel: ata8.00: exception Emask 0x0 SAct 0x7ff00000 SErr 0x0 action 0x6 frozen
Aug  4 03:00:56 unraid01 kernel: ata8.00: failed command: SEND FPDMA QUEUED
Aug  4 03:00:56 unraid01 kernel: ata8.00: cmd 64/01:a0:00:00:00/00:00:00:00:00/a0 tag 20 ncq dma 512 out
Aug  4 03:00:56 unraid01 kernel:         res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)

Aug  4 03:00:56 unraid01 kernel: ata8.00: status: { DRDY }

.

.

.

Aug  4 03:02:30 unraid01 kernel: ata8: hard resetting link
Aug  4 03:02:30 unraid01 kernel: ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Aug  4 03:02:30 unraid01 kernel: ata8.00: supports DRM functions and may not be fully accessible
Aug  4 03:02:30 unraid01 kernel: ata8.00: supports DRM functions and may not be fully accessible
Aug  4 03:02:30 unraid01 kernel: ata8.00: configured for UDMA/133
Aug  4 03:02:30 unraid01 kernel: ata8.00: device reported invalid CHS sector 0
Aug  4 03:02:30 unraid01 kernel: ata8.00: device reported invalid CHS sector 0
Aug  4 03:02:30 unraid01 kernel: ata8: EH complete
Aug  4 03:02:30 unraid01 kernel: ata8.00: Enabling discard_zeroes_data

Aug  4 04:10:12 unraid01 kernel: BUG: unable to handle kernel paging request at 0000020000000010
Aug  4 04:10:12 unraid01 kernel: PGD 0 P4D 0 
Aug  4 04:10:12 unraid01 kernel: Oops: 0000 [#1] SMP PTI
Aug  4 04:10:12 unraid01 kernel: CPU: 2 PID: 4590 Comm: shfs Not tainted 4.19.56-Unraid #1
Aug  4 04:10:12 unraid01 kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77 Extreme4, BIOS P2.90 07/11/2013
Aug  4 04:10:12 unraid01 kernel: RIP: 0010:__vma_rb_erase+0xa3/0x190
Aug  4 04:10:12 unraid01 kernel: Code: e9 f4 00 00 00 48 89 06 e9 07 01 00 00 4d 8b 41 10 4d 89 cb 4d 85 c0 75 11 48 8b 47 38 4c 89 c9 49 8b 69 08 49 89 41 18 eb 58 <49> 8b 40 10 48 85 c0 74 08 4d 89 c3 49 89 c0 eb ef 49 8b 68 08 4c
Aug  4 04:10:12 unraid01 kernel: RSP: 0018:ffffc90008b23e88 EFLAGS: 00010206
Aug  4 04:10:12 unraid01 kernel: RAX: ffff888641daa6e0 RBX: ffff888407f2bb40 RCX: ffff888641daa6d0
Aug  4 04:10:12 unraid01 kernel: RDX: ffffffff00000001 RSI: ffff8887f7543748 RDI: ffff888407f2bb40
Aug  4 04:10:12 unraid01 kernel: RBP: 00001515bd140000 R08: 0000020000000000 R09: ffff888407f2bc20
Aug  4 04:10:12 unraid01 kernel: R10: ffff888407f2bb60 R11: ffff888407f2bc20 R12: ffff888641daa6c0
Aug  4 04:10:12 unraid01 kernel: R13: 00001515bd11e000 R14: ffff888407f2bb40 R15: ffff888407f2bb40
Aug  4 04:10:12 unraid01 kernel: FS:  00001515bda6a700(0000) GS:ffff8887ff500000(0000) knlGS:0000000000000000
Aug  4 04:10:12 unraid01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug  4 04:10:12 unraid01 kernel: CR2: 0000020000000010 CR3: 00000007d0578002 CR4: 00000000001606e0
Aug  4 04:10:12 unraid01 kernel: Call Trace:
Aug  4 04:10:12 unraid01 kernel: do_munmap+0x1ef/0x2b8
Aug  4 04:10:12 unraid01 kernel: vm_munmap+0x62/0x90
Aug  4 04:10:12 unraid01 kernel: __x64_sys_munmap+0xd/0x10
Aug  4 04:10:12 unraid01 kernel: do_syscall_64+0x57/0xf2
Aug  4 04:10:12 unraid01 kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Aug  4 04:10:12 unraid01 kernel: RIP: 0033:0x1515be628107
Aug  4 04:10:12 unraid01 kernel: Code: 10 e9 67 ff ff ff 0f 1f 44 00 00 48 8b 15 81 ed 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff e9 6b ff ff ff b8 0b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 59 ed 0c 00 f7 d8 64 89 01 48
Aug  4 04:10:12 unraid01 kernel: RSP: 002b:00001515bda69b08 EFLAGS: 00000202 ORIG_RAX: 000000000000000b
Aug  4 04:10:12 unraid01 kernel: RAX: ffffffffffffffda RBX: 00001515a8007b60 RCX: 00001515be628107
Aug  4 04:10:12 unraid01 kernel: RDX: 0000000000000000 RSI: 0000000000022000 RDI: 00001515bd11e000
Aug  4 04:10:12 unraid01 kernel: RBP: 000000000044af60 R08: 00001515bd11e000 R09: 0000000000000000
Aug  4 04:10:12 unraid01 kernel: R10: 0000000093300000 R11: 0000000000000202 R12: 00001515bda69c50
Aug  4 04:10:12 unraid01 kernel: R13: 0000000000000001 R14: 00001515bd11f000 R15: 000000000044d4e0
Aug  4 04:10:12 unraid01 kernel: Modules linked in: xt_nat veth xt_CHECKSUM ipt_REJECT ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat nfsd lockd grace sunrpc md_mod bonding e1000e tg3 x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd i2c_i801 glue_helper video i2c_core backlight mxm_wmi intel_cstate intel_uncore ahci intel_rapl_perf wmi libahci ie31200_edac button pcc_cpufreq [last unloaded: e1000e]
Aug  4 04:10:12 unraid01 kernel: CR2: 0000020000000010
Aug  4 04:10:12 unraid01 kernel: ---[ end trace 91867f88e09af37a ]---
Aug  4 04:10:12 unraid01 kernel: RIP: 0010:__vma_rb_erase+0xa3/0x190
Aug  4 04:10:12 unraid01 kernel: Code: e9 f4 00 00 00 48 89 06 e9 07 01 00 00 4d 8b 41 10 4d 89 cb 4d 85 c0 75 11 48 8b 47 38 4c 89 c9 49 8b 69 08 49 89 41 18 eb 58 <49> 8b 40 10 48 85 c0 74 08 4d 89 c3 49 89 c0 eb ef 49 8b 68 08 4c
Aug  4 04:10:12 unraid01 kernel: RSP: 0018:ffffc90008b23e88 EFLAGS: 00010206
Aug  4 04:10:12 unraid01 kernel: RAX: ffff888641daa6e0 RBX: ffff888407f2bb40 RCX: ffff888641daa6d0
Aug  4 04:10:12 unraid01 kernel: RDX: ffffffff00000001 RSI: ffff8887f7543748 RDI: ffff888407f2bb40
Aug  4 04:10:12 unraid01 kernel: RBP: 00001515bd140000 R08: 0000020000000000 R09: ffff888407f2bc20
Aug  4 04:10:12 unraid01 kernel: R10: ffff888407f2bb60 R11: ffff888407f2bc20 R12: ffff888641daa6c0
Aug  4 04:10:12 unraid01 kernel: R13: 00001515bd11e000 R14: ffff888407f2bb40 R15: ffff888407f2bb40
Aug  4 04:10:12 unraid01 kernel: FS:  00001515bda6a700(0000) GS:ffff8887ff500000(0000) knlGS:0000000000000000
Aug  4 04:10:12 unraid01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug  4 04:10:12 unraid01 kernel: CR2: 0000020000000010 CR3: 00000007d0578002 CR4: 00000000001606e0
Aug  4 04:16:16 unraid01 emhttpd: shcmd (218): /usr/sbin/hdparm -y /dev/sdg
Aug  4 04:16:16 unraid01 root: 
Aug  4 04:16:16 unraid01 root: /dev/sdg:
Aug  4 04:16:16 unraid01 root:  issuing standby command
Aug  4 04:16:16 unraid01 emhttpd: shcmd (219): /usr/sbin/hdparm -y /dev/sdc
Aug  4 04:16:16 unraid01 root: 
Aug  4 04:16:16 unraid01 root: /dev/sdc:
Aug  4 04:16:16 unraid01 root:  issuing standby command
Aug  4 11:05:26 unraid01 kernel: md: sync done. time=43867sec
Aug  4 11:05:26 unraid01 kernel: md: recovery thread: exit status: 0
 

Go to Tools - Diagnostics and attach the complete diagnostics zip file to your next post.

after I crash it again?   I can do that.   I am just waiting for the parity to stabilize at this point. 

23 minutes ago, Quartz said:

after I crash it again?   I can do that.   I am just waiting for the parity to stabilize at this point. 

Ideally diagnostics should be taken before rebooting. This is because syslog from before the reboot will be lost, unless you have one of the latest versions of Unraid that includes the syslog server feature and you have that setup.

 

So diagnostics taken immediately after a crash might tell us less than diagnostics taken before a crash. Possibly it won't matter either way.

 

Syslog snippets are seldom sufficient. In fact I would say they are a waste of time.

I did save the syslog.    that is most of it though.   I'll just crash it again when the parity checks finish.