Problem?

 

unRaid 6.2.4

 

Lately I've had my unassigned drives stop mounting and the mount button is darkened. Can only be remounted by stopping and starting the array. But I guess that has nothing to do with this.

 

Jun  3 08:49:16 Brahms1 kernel: WARNING: CPU: 2 PID: 16 at net/core/skbuff.c:4177 skb_try_coalesce+0x22c/0x338()
Jun  3 08:49:16 Brahms1 kernel: Modules linked in: xt_nat veth xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 ebtable_filter ebtables vhost_net tun vhost macvtap macvlan ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_nat_ipv4 iptable_filter ip_tables nf_nat md_mod ipmi_devintf bonding mlx4_en mlx4_core vxlan udp_tunnel ptp pps_core bnx2 coretemp kvm_intel kvm mpt3sas raid_class scsi_transport_sas ipmi_si acpi_cpufreq [last unloaded: mlx4_core]
Jun  3 08:49:16 Brahms1 kernel: CPU: 2 PID: 16 Comm: ksoftirqd/2 Tainted: G          I     4.4.30-unRAID #2
Jun  3 08:49:16 Brahms1 kernel: Hardware name: HP ProLiant DL380 G6, BIOS P62 08/16/2010
Jun  3 08:49:16 Brahms1 kernel: 0000000000000000 ffff8811eaf07990 ffffffff8136f79f 0000000000000000
Jun  3 08:49:16 Brahms1 kernel: 0000000000001051 ffff8811eaf079c8 ffffffff8104a4ab ffffffff8152f85e
Jun  3 08:49:16 Brahms1 kernel: ffff88011b2f4100 ffff88011b2f5200 0000000000000ac0 ffff8811eaf07a34
Jun  3 08:49:16 Brahms1 kernel: Call Trace:
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8136f79f>] dump_stack+0x61/0x7e
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8104a4ab>] warn_slowpath_common+0x8f/0xa8
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8152f85e>] ? skb_try_coalesce+0x22c/0x338
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8104a568>] warn_slowpath_null+0x15/0x17
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8152f85e>] skb_try_coalesce+0x22c/0x338
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81597820>] tcp_try_coalesce+0x38/0x97
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81597cd0>] tcp_queue_rcv+0x5c/0x101
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81598964>] tcp_data_queue+0x1d1/0xa4b
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8159babc>] tcp_rcv_established+0x2c4/0x5e5
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff815a2ded>] tcp_v4_do_rcv+0x98/0x1c8
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff815a52b9>] tcp_v4_rcv+0x733/0xad1
Jun  3 08:49:16 Brahms1 kernel: [<ffffffffa059f213>] ? ipv4_confirm+0x7a/0xd0 [nf_conntrack_ipv4]
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81586da8>] ip_local_deliver_finish+0xf4/0x1c3
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff815872c3>] ip_local_deliver+0xae/0xb7
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81586cb4>] ? inet_del_offload+0x40/0x40
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81587110>] ip_rcv_finish+0x299/0x2a2
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8158759d>] ip_rcv+0x2d1/0x32f
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81586e77>] ? ip_local_deliver_finish+0x1c3/0x1c3
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8153c6cf>] __netif_receive_skb_core+0x5a5/0x644
Jun  3 08:49:16 Brahms1 kernel: [<ffffffffa0521794>] ? mlx4_en_process_rx_cq+0x72e/0x96d [mlx4_en]
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8106d584>] ? dequeue_entity+0x69b/0x775
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8153cc2f>] __netif_receive_skb+0x13/0x55
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8153d98e>] process_backlog+0xac/0x143
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8153d794>] net_rx_action+0xd8/0x226
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8104d6d3>] __do_softirq+0xc3/0x1b6
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8104d7e1>] run_ksoftirqd+0x1b/0x3c
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81061e6c>] smpboot_thread_fn+0x16c/0x1ac
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81061d00>] ? sort_range+0x1d/0x1d
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8105fb24>] kthread+0xcd/0xd5
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8105fa57>] ? kthread_worker_fn+0x137/0x137
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff81629f7f>] ret_from_fork+0x3f/0x70
Jun  3 08:49:16 Brahms1 kernel: [<ffffffff8105fa57>] ? kthread_worker_fn+0x137/0x137
Jun  3 08:49:16 Brahms1 kernel: ---[ end trace 7a4c1f5132bbb293 ]---

 

I would suggest posting up the entire diagnostics log...

SHAZAM!

brahms1-diagnostics-20170604-0950.zip

Edited June 4, 20179 yr by 1812

The first trace (the one you posted is network related).  Beyond me though.  I'd say that if your 10G network is working fine to leave well enough alone.

 

You've got these warnings from Unassigned Devices

Jun  3 19:25:49 Brahms1 kernel: FAT-fs (sdg1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
Jun  3 19:25:49 Brahms1 kernel: hfsplus: invalid secondary volume header
Jun  3 19:25:49 Brahms1 kernel: hfsplus: unable to find HFS+ superblock
Jun  3 19:25:49 Brahms1 kernel: hfsplus: Filesystem was not cleanly unmounted, running fsck.hfsplus is recommended.  mounting read-only.
Jun  3 19:25:50 Brahms1 kernel: FAT-fs (sdh1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
Jun  3 19:25:50 Brahms1 kernel: hfsplus: Filesystem was not cleanly unmounted, running fsck.hfsplus is recommended.  mounting read-only.

And then the following from your cache drive (sdb)

Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 842878, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 842879, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843094, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843123, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843328, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843329, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843330, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843408, lost async page write
Jun  4 07:38:01 Brahms1 kernel: Buffer I/O error on dev sdb1, logical block 843409, lost async page write

Which then leads to errors in your docker.img file

Jun  4 07:38:02 Brahms1 kernel: blk_update_request: I/O error, dev loop0, sector 8520248
Jun  4 07:38:02 Brahms1 kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 0, rd 1, flush 0, corrupt 0, gen 0
Jun  4 07:38:02 Brahms1 kernel: blk_update_request: I/O error, dev loop0, sector 8520408
Jun  4 07:38:02 Brahms1 kernel: BTRFS error (device loop0): bdev /dev/loop0 errs: wr 0, rd 2, flush 0, corrupt 0, gen 0
Jun  4 07:38:02 Brahms1 kernel: blk_update_request: I/O error, dev loop0, sector 8520488

Followed by btrfs having a heartattack trying to figure out the docker.img and posting up its own call trace and a ton of other errors

 

I'd Check Disk Filesystem on the cache drive, and delete the docker.img and go from there.

6 hours ago, Squid said:

The first trace (the one you posted is network related).  Beyond me though.  I'd say that if your 10G network is working fine to leave well enough alone.

 

 It's either dying, or unRaid is being wonky. The same card was acting up in a near identical server a few weeks ago. Another user was also recently reporting issues with one. But a few others using them are just fine. Maybe mine are just getting too hot in their 2u cases crowded with other cards...

 

 

6 hours ago, Squid said:

You've got these warnings from Unassigned Devices

 

the hfs/fat-fs disc issues are 2 discs that get passed to an osx vm, which are then combined in software raid through the os. I don't think unRaid is sure what to make of them since it doesn't expect to see 2 raided disc just hanging out, but they are fine as far as I can tell. If I stop the array w/out un-mounting them, I get the error. If I remember to unmount them before stopping the array, no errors.

 

 

The cache drive/docker image errosr makes sense as plex has been acting up lately. It's a cheap SSD and I need an excuse to upgrade!

 

Many thanks, @Squid!