I currently have 12 drives connected to motherboard SATA ports and port multipliers.  Everything runs pretty smoothly with no surprises except when I write directly to the cache drive from a mounted NTFS drive.  That's a pretty fast transfer and you can see by watching drive access lights that other operations (like a parity rebuild or mover running) that the writes to cache are significantly interrupting things.

 

After a few minutes of this, syslog errors start showing up:

 

Sep 25 03:14:58 Bench kernel: ata6.00: status: { DRDY ERR }

Sep 25 03:14:58 Bench kernel: ata6.00: error: { ABRT }

Sep 25 03:14:58 Bench kernel: ata6.00: configured for UDMA/133

Sep 25 03:14:58 Bench kernel: ata6: EH complete

Sep 25 03:14:58 Bench kernel: sd 6:0:0:0: [sdf] 976773168 512-byte hardware sectors: (500GB/465 GiB)  this is the NTFS drive

Sep 25 03:14:58 Bench kernel: sd 6:0:0:0: [sdf] Write Protect is off

Sep 25 03:14:58 Bench kernel: sd 6:0:0:0: [sdf] Mode Sense: 00 3a 00 00

Sep 25 03:14:58 Bench kernel: sd 6:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:14:58 Bench kernel: sd 6:1:0:0: [sdg] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:14:58 Bench kernel: sd 6:1:0:0: [sdg] Write Protect is off

Sep 25 03:14:58 Bench kernel: sd 6:1:0:0: [sdg] Mode Sense: 00 3a 00 00

Sep 25 03:14:58 Bench kernel: sd 6:1:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:14:58 Bench kernel: sd 6:2:0:0: [sdh] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:14:58 Bench kernel: sd 6:2:0:0: [sdh] Write Protect is off

Sep 25 03:14:58 Bench kernel: sd 6:2:0:0: [sdh] Mode Sense: 00 3a 00 00

Sep 25 03:14:58 Bench kernel: sd 6:2:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:14:58 Bench kernel: sd 6:3:0:0: [sdi] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:14:58 Bench kernel: sd 6:3:0:0: [sdi] Write Protect is off

Sep 25 03:14:58 Bench kernel: sd 6:3:0:0: [sdi] Mode Sense: 00 3a 00 00

Sep 25 03:14:58 Bench kernel: sd 6:3:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:15:09 Bench kernel: ata6.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0

Sep 25 03:15:09 Bench kernel: ata6.00: cmd c8/00:a8:27:27:0c/00:00:00:00:00/ec tag 0 dma 86016 in

Sep 25 03:15:09 Bench kernel:          res 51/04:a8:27:27:0c/00:00:00:00:00/ec Emask 0x1 (device error)

Sep 25 03:15:09 Bench kernel: ata6.00: status: { DRDY ERR }

Sep 25 03:15:09 Bench kernel: ata6.00: error: { ABRT }

Sep 25 03:15:09 Bench kernel: ata6.00: configured for UDMA/133

Sep 25 03:15:09 Bench kernel: ata6: EH complete

Sep 25 03:15:09 Bench kernel: sd 6:0:0:0: [sdf] 976773168 512-byte hardware sectors: (500GB/465 GiB)

Sep 25 03:15:09 Bench kernel: sd 6:0:0:0: [sdf] Write Protect is off

Sep 25 03:15:09 Bench kernel: sd 6:0:0:0: [sdf] Mode Sense: 00 3a 00 00

Sep 25 03:15:09 Bench kernel: sd 6:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:15:09 Bench kernel: sd 6:1:0:0: [sdg] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:15:09 Bench kernel: sd 6:1:0:0: [sdg] Write Protect is off

Sep 25 03:15:09 Bench kernel: sd 6:1:0:0: [sdg] Mode Sense: 00 3a 00 00

Sep 25 03:15:09 Bench kernel: sd 6:1:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:15:09 Bench kernel: sd 6:2:0:0: [sdh] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:15:09 Bench kernel: sd 6:2:0:0: [sdh] Write Protect is off

Sep 25 03:15:09 Bench kernel: sd 6:2:0:0: [sdh] Mode Sense: 00 3a 00 00

Sep 25 03:15:09 Bench kernel: sd 6:2:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:15:09 Bench kernel: sd 6:3:0:0: [sdi] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)

Sep 25 03:15:09 Bench kernel: sd 6:3:0:0: [sdi] Write Protect is off

Sep 25 03:15:09 Bench kernel: sd 6:3:0:0: [sdi] Mode Sense: 00 3a 00 00

Sep 25 03:15:09 Bench kernel: sd 6:3:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Sep 25 03:15:21 Bench kernel: ata6.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0

Sep 25 03:15:21 Bench kernel: ata6.00: cmd c8/00:a8:27:0f:1c/00:00:00:00:00/ec tag 0 dma 86016 in

Sep 25 03:15:21 Bench kernel:          res 51/04:a8:27:0f:1c/00:00:00:00:00/ec Emask 0x1 (device error)

 

And this sequence repeats over and over until I stop the writes to the cache drive.

 

It looks like the NTFS drive itself is getting reset by the controller.  All the other drives referenced are data drives.  The cache drive is /dev/sde and is not referenced.

 

Anyone recognize this behavior?  I've transferred several TB of data without this problem, but not necessarily with the same controller/PM configuration.

 

--Bill

This turns out to be caused by using a port multiplier on the motherboard AHCI SATA ports (ICH10 chipset).  It only shows up when a local drive is really pushing fast transfers to the cache drive.  Even if the cache drive is NOT on a PM or motherboard SATA port, it still happens.  Move all PM's from MB to sil3132 PCI-e controllers and the behavior stops.

 

So either there are still Intel bugs in this implementation of AHCI, or (more likely) the drivers in our unRaid kernel is not correctly handling the ICH10 chipset when PM's are involved.

 

Shot down again.

 

--Bill