Something weird is happening to my cache system.

I have ran one drive for a long time just added in my second SSD about 2 weeks ago.

 

I am getting these errors.

ov 19 20:35:21 Vault kernel: sd 4:0:0:0: [sdk] tag#17 CDB: opcode=0x28 28 00 1d 54 f4 28 00 00 08 00
Nov 19 20:35:21 Vault kernel: blk_update_request: I/O error, dev sdk, sector 492106792
Nov 19 20:35:21 Vault kernel: sd 4:0:0:0: [sdk] tag#18 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 19 20:35:21 Vault kernel: sd 4:0:0:0: [sdk] tag#18 CDB: opcode=0x28 28 00 1d 54 f4 30 00 00 08 00
Nov 19 20:35:21 Vault kernel: blk_update_request: I/O error, dev sdk, sector 492106800
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1
Nov 19 20:35:22 Vault kernel: BTRFS warning (device sdl1): lost page write due to IO error on /dev/sdk1

 

When i Smart the drive its all fine and when i do preclears it is error free.

Can anyone help with this, when i reboot it seems to be repairing it.

 

Model family:	Indilinx Barefoot_2/Everest/Martini based SSDs
Device model:	OCZ-VERTEX4
Serial number:	OCZ-O71N80GX4YB2BAX3
LU WWN device id:	5 e83a97 79a2515c2
Firmware version:	1.5.1
User capacity:	256,060,514,304 bytes [256 GB]
Sector size:	512 bytes logical/physical
Rotation rate:	Solid State Device
Device:	In smartctl database [for details use: -P show]
ATA version:	ACS-2 (minor revision not indicated)
SATA version:	SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local time:	Sun Nov 19 20:55:43 2017 AEDT
SMART support:	Available - device has SMART capability.
SMART support:	Enabled
SMART overall-health:	Passed

 

 

scrub status for 8e82d82d-f6f6-45d7-9e5a-d389fb0e0bb3 scrub started at Sun Nov 19 21:16:33 2017 and finished after 00:02:37 total bytes

scrubbed: 27.56GiB with 1208172 errors error details: verify=3447 csum=1204725 corrected errors: 1208172, uncorrectable errors: 0, unverified errors: 0

 

Edited November 19, 20178 yr by Maticks

It's a hardware problem, with SSDs usually a bad cable, try replacing or swapping both cables, then run a correcting scrub on the pool.

 

In the future post the complete diagnostics.

Remember that SSD normally uses a higher bandwidth on the cable - in this case 6 Gbit/s, while most HDD only uses 3 Gbit/s and the first generation SATA disks used 1,5 Gbit/s.

 

You need a SATA cable that is designed to handle 6 Gbit/s or you may get lots of issues with the drive.

I have them both plugged into my onboard Sata controller on my Z87 motherboard.

I do have an LSI 9210-8i is that going to be better to run them off of?
the first cache drive seems to be fine though and its running off the same Sata controller as the second one.

5 minutes ago, Maticks said:

I do have an LSI 9210-8i is that going to be better to run them off of?

No, it doesn't trim support on most SSDs, just replace the cables.

Could it be the OCZ Firmware on the drive that is causing this?

usually with SATA cables a stress test shows the problem, 11 hours of heavy data read and writes and its still fine..

 

8 minutes ago, Maticks said:

Could it be the OCZ Firmware on the drive that is causing this?

Possible but IMO not very likely.

 

9 minutes ago, Maticks said:

usually with SATA cables a stress test shows the problem

Not necessarily, most likely culprits are the cables (it can also be the power cable, specially if using an adapter), the SSD itself or the SATA port/controller.

Ok changed out the SATA Power cable and changed out the SATA Data cable.

ran for 10 hours no issues reading and writing then this happened in the syslog.

 

Nov 21 10:42:36 Vault kernel: ata3.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x6 frozen
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:00:c0:42:7c/05:00:00:00:00/40 tag 0 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:08:c0:47:7c/05:00:00:00:00/40 tag 1 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:10:c0:4c:7c/05:00:00:00:00/40 tag 2 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:18:c0:51:7c/06:00:00:00:00/40 tag 3 ncq dma 786432 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:20:c0:57:7c/07:00:00:00:00/40 tag 4 ncq dma 917504 out
Nov 21 10:42:36 Vault kernel:         res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:28:c0:5e:7c/05:00:00:00:00/40 tag 5 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/08:30:50:10:62/04:00:00:00:00/40 tag 6 ncq dma 528384 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3: hard resetting link
Nov 21 10:42:42 Vault kernel: ata3: link is slow to respond, please be patient (ready=0)
Nov 21 10:42:46 Vault kernel: ata3: COMRESET failed (errno=-16)
Nov 21 10:42:46 Vault kernel: ata3: hard resetting link
Nov 21 10:42:52 Vault kernel: ata3: link is slow to respond, please be patient (ready=0)
Nov 21 10:42:56 Vault kernel: ata3: COMRESET failed (errno=-16)
Nov 21 10:42:56 Vault kernel: ata3: hard resetting link
Nov 21 10:43:02 Vault kernel: ata3: link is slow to respond, please be patient (ready=0)
Nov 21 10:43:31 Vault kernel: ata3: COMRESET failed (errno=-16)
Nov 21 10:43:31 Vault kernel: ata3: limiting SATA link speed to 3.0 Gbps
Nov 21 10:43:31 Vault kernel: ata3: hard resetting link
Nov 21 10:43:36 Vault kernel: ata3: COMRESET failed (errno=-16)
Nov 21 10:43:36 Vault kernel: ata3: reset failed, giving up
Nov 21 10:43:36 Vault kernel: ata3.00: disabled
Nov 21 10:43:36 Vault kernel: ata3: EH complete
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#7 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#7 CDB: opcode=0x2a 2a 00 00 7c 3d c0 00 05 00 00
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 8142272
Nov 21 10:43:36 Vault kernel: btrfs_dev_stat_print_on_error: 47087 callbacks suppressed
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752897, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752898, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752899, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752900, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752901, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752902, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752903, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752904, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752905, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14752906, rd 13109798, flush 62525, corrupt 2409818, gen 6908
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#8 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#8 CDB: opcode=0x2a 2a 00 00 7c 38 c0 00 05 00 00
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 8140992
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#9 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#9 CDB: opcode=0x2a 2a 00 00 7c 31 c0 00 07 00 00
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 8139200
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#10 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#10 CDB: opcode=0x2a 2a 00 00 7c 28 40 00 09 80 00
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 8136768
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#11 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:36 Vault kernel: sd 4:0:0:0: [sdg] tag#11 CDB: opcode=0x28 28 00 05 eb 8c c0 00 02 60 00
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 99323072
Nov 21 10:43:36 Vault kernel: blk_update_request: I/O error, dev sdg, sector 159799760
Nov 21 10:43:36 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:36 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:36 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:36 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:37 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:41 Vault kernel: scsi_io_completion: 123230 callbacks suppressed
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#0 CDB: opcode=0x28 28 00 03 34 a1 b0 00 02 10 00
Nov 21 10:43:41 Vault kernel: blk_update_request: 123285 callbacks suppressed
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780912
Nov 21 10:43:41 Vault kernel: btrfs_dev_stat_print_on_error: 241929 callbacks suppressed
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226455, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226456, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226457, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226458, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226459, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#1 CDB: opcode=0x28 28 00 03 34 a3 c8 00 00 68 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53781448
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226460, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#2 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#2 CDB: opcode=0x28 28 00 03 34 a1 b0 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780912
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226461, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226463, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#5 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#5 CDB: opcode=0x28 28 00 03 34 a1 c8 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780936
Nov 21 10:43:41 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14878386, rd 13226464, flush 62535, corrupt 2409818, gen 6908
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#6 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#6 CDB: opcode=0x28 28 00 03 34 a1 d0 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780944
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#7 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#7 CDB: opcode=0x28 28 00 03 34 a1 d8 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780952
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#8 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#8 CDB: opcode=0x28 28 00 03 34 a1 e0 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780960
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#9 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:41 Vault kernel: sd 4:0:0:0: [sdg] tag#9 CDB: opcode=0x28 28 00 03 34 a1 e8 00 00 08 00
Nov 21 10:43:41 Vault kernel: blk_update_request: I/O error, dev sdg, sector 53780968
Nov 21 10:43:42 Vault kernel: btrfs_end_buffer_write_sync: 1 callbacks suppressed
Nov 21 10:43:42 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:42 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:43 Vault kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdg1
Nov 21 10:43:46 Vault kernel: scsi_io_completion: 42067 callbacks suppressed
Nov 21 10:43:46 Vault kernel: sd 4:0:0:0: [sdg] tag#3 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
Nov 21 10:43:46 Vault kernel: sd 4:0:0:0: [sdg] tag#3 CDB: opcode=0x28 28 00 04 32 cd f8 00 03 f8 00
Nov 21 10:43:46 Vault kernel: blk_update_request: 42086 callbacks suppressed
Nov 21 10:43:46 Vault kernel: blk_update_request: I/O error, dev sdg, sector 70438392
Nov 21 10:43:46 Vault kernel: btrfs_dev_stat_print_on_error: 79975 callbacks suppressed
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257903, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257904, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257905, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257906, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257907, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257908, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257909, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257910, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: BTRFS error (device sdh1): bdev /dev/sdg1 errs: wr 14927015, rd 13257911, flush 62538, corrupt 2409818, gen 6908
Nov 21 10:43:46 Vault kernel: sd 4:0:0:0: [sdg] tag#4 UNKNOWN(0x2003) Result: hostbyte=0x04 driverbyte=0x00
 Smart says everything is ok...

 

Then it's likely the SSD itself.

got to the bottom of the issue... its BUGFS.

 

btrfs filesystem show:

Label: none  uuid: 8e82d82d-f6f6-45d7-9e5a-d389fb0e0bb3
    Total devices 2 FS bytes used 31.45GiB
    devid    1 size 238.47GiB used 201.06GiB path /dev/sdl1
    devid    3 size 238.47GiB used 201.06GiB path /dev/sdo1
 

The filesystem show is filling up it was at 201GB when it hits 235GB it starts throwing those errors.

Given my average data patterns on the cache it makes sense why it happens on write times.

My filesystem shows 31GB is usage though.

 

Manually forcing a balance 

btrfs balance start -dusage=80 /mnt/cache

Done, had to relocate 107 out of 143 chunks
 

btrfs filesystem show:

Label: none  uuid: 8e82d82d-f6f6-45d7-9e5a-d389fb0e0bb3
    Total devices 2 FS bytes used 31.41GiB
    devid    1 size 238.47GiB used 35.06GiB path /dev/sdl1
    devid    3 size 238.47GiB used 35.06GiB path /dev/sdo1

 

No more cache errors

I think ill have to convert this to XFS or something..

maybe we just need balance to run after mover runs..

These errors are hardware errors, nothing to do with a balance:

 

Nov 21 10:42:36 Vault kernel: ata3.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x6 frozen
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:00:c0:42:7c/05:00:00:00:00/40 tag 0 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:08:c0:47:7c/05:00:00:00:00/40 tag 1 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED
Nov 21 10:42:36 Vault kernel: ata3.00: cmd 61/00:10:c0:4c:7c/05:00:00:00:00/40 tag 2 ncq dma 655360 out
Nov 21 10:42:36 Vault kernel:         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 21 10:42:36 Vault kernel: ata3.00: status: { DRDY }
Nov 21 10:42:36 Vault kernel: ata3.00: failed command: WRITE FPDMA QUEUED 

 

Exactly - the error output is not caused by unRAID software but represents issues with the actual hardware.

 

It's just that btrfs balance affects how much of the physical flash space that gets used inside the SSD, and the storage pattern. So running balance can affect if you see the hardware errors or not.

 

But in the end, it's time to consider a different drive.