BTRFS warnings / errors


Recommended Posts

Out of nowhere (no change to system) these errors popped up on all drives in the array (see attached logs).

Jul 31 12:27:31 NAS kernel: BTRFS warning (device sdb1): direct IO failed ino 11586 rw 0,0 sector 0x34d0ca8 len 0 err no 10
Jul 31 12:27:32 NAS kernel: BTRFS error (device sdb1): bdev /dev/sdb1 errs: wr 1, rd 4913238, flush 0, corrupt 0, gen 0
Jul 31 12:27:35 NAS kernel: blk_update_request: I/O error, dev sdb, sector 6139136 op 0x0:(READ) flags 0x1000 phys_seg 4 prio class 0


Any hint what could be a probable cause for this? Should I be worried?

nas-diagnostics-20210731-1221.zip

Link to comment

Issue starts with this:

 

Jul 31 11:43:47 NAS kernel: ahci 0000:01:00.1: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 address=0xce464000 flags=0x0000]

 

Device 01:00.1 is the AMD SATA controller, after these errors the disks drop offline one by one:

 

Jul 31 11:44:53 NAS kernel: ata1: reset failed, giving up
Jul 31 11:44:53 NAS kernel: ata1.00: disabled

 

Jul 31 11:45:19 NAS kernel: ata6: reset failed, giving up
Jul 31 11:45:19 NAS kernel: ata6.00: disabled

 

etc

Link to comment

Well, another day, another issue, sigh. 

Array went into parity sync after a clean reboot, took 15 hours or so, finished with no errors. 

This morning I woke up to another error:

01-08-2021 06:10	Unraid array errors	Warning [NAS] - array has errors	Array has 1 disk with read errors	warning	
01-08-2021 06:10	Unraid Disk 2 error	Alert [NAS] - Disk 2 in error state (disk dsbl)	ST6000DM003-2CY186_ZF2032KA (sde)	alert


 

I've attached the logs again. There is a new SATA controller on the way. 
This seems to be the relevant section:
Aug  1 06:00:54 NAS emhttpd: read SMART /dev/sdd
Aug  1 06:04:32 NAS kernel: ata6.00: exception Emask 0x0 SAct 0x10000 SErr 0x0 action 0x6 frozen
Aug  1 06:04:32 NAS kernel: ata6.00: failed command: READ FPDMA QUEUED
Aug  1 06:04:32 NAS kernel: ata6.00: cmd 60/08:80:b8:de:a4/00:00:9b:02:00/40 tag 16 ncq dma 4096 in
Aug  1 06:04:32 NAS kernel:         res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Aug  1 06:04:32 NAS kernel: ata6.00: status: { DRDY }
Aug  1 06:04:32 NAS kernel: ata6: hard resetting link
Aug  1 06:04:42 NAS kernel: ata6: softreset failed (1st FIS failed)
Aug  1 06:04:42 NAS kernel: ata6: hard resetting link
Aug  1 06:04:52 NAS kernel: ata6: softreset failed (1st FIS failed)
Aug  1 06:04:52 NAS kernel: ata6: hard resetting link
...
Aug  1 06:05:27 NAS kernel: ata6: softreset failed (1st FIS failed)
Aug  1 06:08:18 NAS kernel: ata6.00: disabled
Aug  1 06:08:28 NAS kernel: ata6: softreset failed (1st FIS failed)
### [PREVIOUS LINE REPEATED 2 TIMES] ###
Aug  1 06:09:19 NAS kernel: ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Aug  1 06:09:19 NAS kernel: ata6.00: link online but device misclassified
Aug  1 06:09:19 NAS kernel: ata6: EH complete
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#31 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=307s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#31 CDB: opcode=0x35 35 00 00 00 00 00 00 00 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 a1 68 6e b8 00 00 00 20 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 2707975864 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 0
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=2707975800
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=2707975808
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=2707975816
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=2707975824
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#1 CDB: opcode=0x88 88 00 00 00 00 02 84 01 28 98 00 00 00 20 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 10804603032 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 0
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=10804602968
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=10804602976
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=10804602984
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=10804602992
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#2 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#2 CDB: opcode=0x88 88 00 00 00 00 00 46 c7 18 80 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 1187453056 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=1187452992
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#3 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=317s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#3 CDB: opcode=0x88 88 00 00 00 00 02 9b a4 de b8 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 11201207992 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=11201207928
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#4 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#4 CDB: opcode=0x35 35 00 00 00 00 00 00 00 00 00
Aug  1 06:09:19 NAS kernel: blk_update_request: I/O error, dev sde, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
### [PREVIOUS LINE REPEATED 4 TIMES] ###
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#18 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#18 CDB: opcode=0x8a 8a 00 00 00 00 01 80 39 4b 90 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205776
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#19 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#19 CDB: opcode=0x88 88 00 00 00 00 01 80 39 4b d0 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: md: disk2 read error, sector=6446205840
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#21 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#21 CDB: opcode=0x8a 8a 00 00 00 00 01 80 39 4b 98 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205784
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#22 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00 cmd_age=0s
Aug  1 06:09:19 NAS kernel: sd 6:0:0:0: [sde] tag#22 CDB: opcode=0x8a 8a 00 00 00 00 01 80 39 4b a0 00 00 00 08 00 00
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205792
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205800
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205808
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=6446205816
...
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=10804602976
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=10804602984
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=10804602992
Aug  1 06:09:19 NAS kernel: md: disk2 write error, sector=11201207928

Is this again related to the controller, or is this a genuine disk error (disk is somewhat old)?

nas-diagnostics-20210801-1051.zip

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.