Cannot get system stable again

BigWorm · November 30, 2018

This weekend I had the server become unresponsive. I had to do a hard reset. Ever since then, I cannot get past the parity check. It's showing errors on parity drive and disk 1. Currently I am running it in safe mode to see what happens during parity check. But it's over 200 million errors.

Any thoughts?

thor-diagnostics-20181129-1959.zip

trurl · November 30, 2018

Both parity and disk 1 are not responding and no SMART for them in your diagnostics. Check connections, try again, and post new diagnostics.

BigWorm · November 30, 2018

11 minutes ago, trurl said:

Both parity and disk 1 are not responding and no SMART for them in your diagnostics. Check connections, try again, and post new diagnostics.

I will try that. These are the only 2 drives connected to the Marvell Controller on the motherboard.

JorgeB · November 30, 2018

You should never use the 4 Marvell ports on those Asrock boards, they are notoriously problematic and keep dropping disks.

BigWorm · December 1, 2018

18 hours ago, johnnie.black said:

You should never use the 4 Marvell ports on those Asrock boards, they are notoriously problematic and keep dropping disks.

Weird thing is that I have been using those ports for about 2 years. I just received a new LSI controller that I am going to put into the server and change the drives to use that one.

xanian · December 1, 2018

I have the Asrock X79 extreme9 with 9220 marvell and i cant confirm that the marvell is shity, like what johnnie.black said
i moved it to the other controller worked fine
just look here

Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: exception Emask 0x0 SAct 0xfffe002f SErr 0x0 action 0x6 frozen
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/88:00:c8:98:fe/04:00:02:00:00/40 tag 0 ncq dma 593920 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/08:08:68:77:ff/00:00:02:00:00/40 tag 1 ncq dma 4096 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/48:10:40:04:10/00:00:03:00:00/40 tag 2 ncq dma 36864 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/38:18:68:ab:9e/00:00:03:00:00/40 tag 3 ncq dma 28672 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:01:01:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/00:28:e8:70:bd/02:00:03:00:00/40 tag 5 ncq dma 262144 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: READ FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 60/08:88:a8:99:a9/00:00:01:00:00/40 tag 17 ncq dma 4096 in
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/40:90:d8:f1:20/00:00:00:00:00/40 tag 18 ncq dma 32768 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:01:01:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/20:98:90:f2:20/00:00:00:00:00/40 tag 19 ncq dma 16384 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/b8:a0:a8:94:83/03:00:00:00:00/40 tag 20 ncq dma 487424 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:01:01:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/08:a8:30:63:38/00:00:01:00:00/40 tag 21 ncq dma 4096 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/08:b0:70:49:8e/00:00:00:00:00/40 tag 22 ncq dma 4096 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/60:b8:40:c3:d9/06:00:00:00:00/40 tag 23 ncq dma 835584 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/a0:c0:08:7d:bc/09:00:01:00:00/40 tag 24 ncq dma 1261568 ou
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/10:c8:98:ef:d6/00:00:01:00:00/40 tag 25 ncq dma 8192 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/10:d0:18:f1:d6/00:00:01:00:00/40 tag 26 ncq dma 8192 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/10:d8:78:f1:d6/00:00:01:00:00/40 tag 27 ncq dma 8192 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/10:e0:f8:f1:d6/00:00:01:00:00/40 tag 28 ncq dma 8192 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:01:01:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/10:e8:38:f2:d6/00:00:01:00:00/40 tag 29 ncq dma 8192 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/28:f0:c8:81:3f/00:00:02:00:00/40 tag 30 ncq dma 20480 out
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: failed command: WRITE FPDMA QUEUED
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: cmd 61/78:f8:50:3e:4b/09:00:02:00:00/40 tag 31 ncq dma 1241088 ou
Nov 30 20:05:35 Hansen-The-Boss kernel: res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x4 (timeout)
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14.00: status: { DRDY }
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14: hard resetting link
Nov 30 20:05:35 Hansen-The-Boss kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Nov 30 20:05:41 Hansen-The-Boss kernel: ata14.00: qc timeout (cmd 0xec)
Nov 30 20:05:41 Hansen-The-Boss kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 30 20:05:41 Hansen-The-Boss kernel: ata14.00: revalidation failed (errno=-5)
Nov 30 20:05:41 Hansen-The-Boss kernel: ata14: hard resetting link
Nov 30 20:05:41 Hansen-The-Boss kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Nov 30 20:05:51 Hansen-The-Boss kernel: ata14.00: qc timeout (cmd 0xec)
Nov 30 20:05:51 Hansen-The-Boss kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 30 20:05:51 Hansen-The-Boss kernel: ata14.00: revalidation failed (errno=-5)
Nov 30 20:05:51 Hansen-The-Boss kernel: ata14: limiting SATA link speed to 1.5 Gbps
Nov 30 20:05:51 Hansen-The-Boss kernel: ata14: hard resetting link
Nov 30 20:05:52 Hansen-The-Boss kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 310)
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14.00: qc timeout (cmd 0xec)
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14.00: revalidation failed (errno=-5)
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14.00: disabled
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 310)
Nov 30 20:06:22 Hansen-The-Boss kernel: ata14: EH complete
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#20 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#20 CDB: opcode=0x2a 2a 00 00 20 f1 d8 00 00 40 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 2159064
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#6 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#6 CDB: opcode=0x2a 2a 00 02 4b 3e 50 00 09 78 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 38485584
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#22 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#22 CDB: opcode=0x2a 2a 00 03 bd 70 e8 00 02 00 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 62746856
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#24 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#24 CDB: opcode=0x2a 2a 00 03 10 04 40 00 00 48 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 51381312
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#26 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#26 CDB: opcode=0x2a 2a 00 02 fe 98 c8 00 04 88 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 50239688
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 5, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 6, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#7 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#7 CDB: opcode=0x2a 2a 00 02 3f 81 c8 00 00 28 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 37716424
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 7, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#8 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#8 CDB: opcode=0x2a 2a 00 01 d6 f2 38 00 00 10 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 30863928
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 8, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#9 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#9 CDB: opcode=0x2a 2a 00 01 d6 f1 f8 00 00 10 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 30863864
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 9, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#10 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#10 CDB: opcode=0x2a 2a 00 01 d6 f1 78 00 00 10 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 30863736
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 10, rd 0, flush 0, corrupt 0, gen 0
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#11 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:06:22 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#11 CDB: opcode=0x2a 2a 00 01 d6 f1 18 00 00 10 00
Nov 30 20:06:22 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 30863640
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS: error (device sdg1) in btrfs_commit_transaction:2237: errno=-5 IO failure (Error while writing out transaction)
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS info (device sdg1): forced readonly
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS warning (device sdg1): Skipping commit of aborted transaction.
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS: error (device sdg1) in cleanup_transaction:1847: errno=-5 IO failure
Nov 30 20:06:22 Hansen-The-Boss kernel: BTRFS info (device sdg1): delayed_refs has NO entry
Nov 30 20:07:52 Hansen-The-Boss kernel: scsi_io_completion: 25 callbacks suppressed
Nov 30 20:07:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#11 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:07:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#11 CDB: opcode=0x28 28 00 01 3b da d8 00 00 08 00
Nov 30 20:07:52 Hansen-The-Boss kernel: print_req_error: 25 callbacks suppressed
Nov 30 20:07:52 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 20699864
Nov 30 20:07:52 Hansen-The-Boss kernel: btrfs_dev_stat_print_on_error: 26 callbacks suppressed
Nov 30 20:07:52 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 35, rd 2, flush 0, corrupt 0, gen 0
Nov 30 20:07:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#12 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:07:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#12 CDB: opcode=0x28 28 00 01 3b da d8 00 00 08 00
Nov 30 20:07:52 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 20699864
Nov 30 20:07:52 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 35, rd 3, flush 0, corrupt 0, gen 0
Nov 30 20:08:23 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#14 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:08:23 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#14 CDB: opcode=0x28 28 00 01 3b da d8 00 00 08 00
Nov 30 20:08:23 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 20699864
Nov 30 20:08:23 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 35, rd 4, flush 0, corrupt 0, gen 0
Nov 30 20:08:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#15 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:08:52 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] tag#15 CDB: opcode=0x28 28 00 01 3b da d8 00 00 08 00
Nov 30 20:08:52 Hansen-The-Boss kernel: print_req_error: I/O error, dev sdg, sector 20699864
Nov 30 20:08:52 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 35, rd 5, flush 0, corrupt 0, gen 0
Nov 30 20:09:57 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:09:57 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:09:57 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:12:02 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:12:02 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:12:02 Hansen-The-Boss kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Nov 30 20:13:28 Hansen-The-Boss kernel: ata14: SATA link down (SStatus 0 SControl 300)
Nov 30 20:13:28 Hansen-The-Boss kernel: ata14.00: detaching (SCSI 14:0:0:0)
Nov 30 20:13:28 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] Synchronizing SCSI cache
Nov 30 20:13:28 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:13:28 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] Stopping disk
Nov 30 20:13:28 Hansen-The-Boss kernel: sd 14:0:0:0: [sdg] Start/Stop Unit failed: Result: hostbyte=0x04 driverbyte=0x00
Nov 30 20:14:52 Hansen-The-Boss kernel: BTRFS error (device sdg1): bdev /dev/sdg1 errs: wr 35, rd 6, flush 0, corrupt 0, gen 0

JorgeB · December 1, 2018

7 hours ago, BigWorm said:

Weird thing is that I have been using those ports for about 2 years.

You've been extremely lucky, that's very uncommon.

John_M · December 1, 2018

Is the issue with the Marvell 9220 broken hardware or is it the same (or related) driver issue that affects the Marvell-based SASLP HBAs? What driver does the 9220 use anyway? The reason I ask is because I still use a Marvell 9235 in one server and it has never been any trouble at all. However, I don't have IOMMU enabled (the CPU doesn't support it) and it does use the standard AHCI driver module.

07:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller (rev 11) (prog-if 01 [AHCI 1.0])
        Subsystem: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller
        Flags: bus master, fast devsel, latency 0, IRQ 28, NUMA node 0
        I/O ports at 5000 [size=8]
        I/O ports at 5008 [size=4]
        I/O ports at 5010 [size=8]
        I/O ports at 5018 [size=4]
        I/O ports at 5020 [size=32]
        Memory at fbff0000 (32-bit, non-prefetchable) [size=2K]
        [virtual] Expansion ROM at fbf00000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Capabilities: [70] Express Legacy Endpoint, MSI 00
        Capabilities: [e0] SATA HBA v0.0
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: ahci
        Kernel modules: ahci

I notice that @BigWorm's server has the Marvell 9230, which also uses the standard AHCI driver. According to the data sheet, my 9235 seems to be a reduced-feature version (no RAID or "HyperDuo") of the 9230 so it's strange that the latter causes so much trouble while the former just seems to work.

Interesting that both the 9220 and 9230 feature in Rob's original list but the 9235 doesn't.

JorgeB · December 1, 2018

9 minutes ago, John_M said:

Is the issue with the Marvell 9220 broken hardware or is it the same (or related) driver issue that affects the Marvell-based SASLP HBAs?

No, Marvell 9215, 9230 and 9235 all use the standard AHCI driver, and some work well, the 9215 and 9235 are likely more stable since there's no raid support, even though it's not being used, the 9230 seems to be the worst offender, but it can vary from manufacturer to manufacturer, the ones used on that series of Asrock boards are usually very problematic, they can have issues every day of the week that ends in Y.

BigWorm · December 1, 2018

Switched the drives to the new controller. Here is the updated diagnostics files.

thor-diagnostics-20181201-1314.zip

Cannot get system stable again

Recommended Posts

BigWorm

Link to comment

trurl

Link to comment

BigWorm

Link to comment

JorgeB

Link to comment

BigWorm

Link to comment

xanian

Link to comment

JorgeB

Link to comment

John_M

Link to comment

JorgeB

Link to comment

BigWorm

Link to comment

Join the conversation