Recently upgraded parity drive to a 16TB refurb EXOS...having major issues

[Original_Vecna]

Purchased two 16TB exos factory refurbs to replace my 14TB parity drive so I could add more space to my array. Pre-cleared the disk, no issues. Added as my parity 1 drive, no issues. Added the second 16TB drive after a pre-clear as my second parity drive, and I woke up to this this morning.

 

Dec 15 07:01:30   kernel: critical medium error, dev sdd, sector 10484684520 op 0x0:(READ) flags 0x0 phys_seg 16 prio class 0
Dec 15 07:01:40   kernel: sd 1:0:2:0: [sdd] tag#1068 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=18s
Dec 15 07:01:40   kernel: sd 1:0:2:0: [sdd] tag#1068 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:01:40   kernel: sd 1:0:2:0: [sdd] tag#1068 ASC=0x11 ASCQ=0x0 
Dec 15 07:01:40   kernel: sd 1:0:2:0: [sdd] tag#1068 CDB: opcode=0x88 88 00 00 00 00 02 70 ef a5 e0 00 00 01 20 00 00
Dec 15 07:01:40   kernel: critical medium error, dev sdd, sector 10484688432 op 0x0:(READ) flags 0x0 phys_seg 26 prio class 0
Dec 15 07:01:50   kernel: sd 1:0:2:0: [sdd] tag#1077 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=9s
Dec 15 07:01:50   kernel: sd 1:0:2:0: [sdd] tag#1077 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:01:50   kernel: sd 1:0:2:0: [sdd] tag#1077 ASC=0x11 ASCQ=0x0 
Dec 15 07:01:50   kernel: sd 1:0:2:0: [sdd] tag#1077 CDB: opcode=0x88 88 00 00 00 00 02 70 ef d2 08 00 00 00 d0 00 00
Dec 15 07:01:50   kernel: critical medium error, dev sdd, sector 10484699736 op 0x0:(READ) flags 0x0 phys_seg 16 prio class 0
Dec 15 07:02:00   kernel: sd 1:0:2:0: [sdd] tag#1047 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=13s
Dec 15 07:02:00   kernel: sd 1:0:2:0: [sdd] tag#1047 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:02:00   kernel: sd 1:0:2:0: [sdd] tag#1047 ASC=0x11 ASCQ=0x0 
Dec 15 07:02:00   kernel: sd 1:0:2:0: [sdd] tag#1047 CDB: opcode=0x88 88 00 00 00 00 02 70 ef e1 90 00 00 02 08 00 00
Dec 15 07:02:00   kernel: critical medium error, dev sdd, sector 10484703840 op 0x0:(READ) flags 0x0 phys_seg 39 prio class 0
Dec 15 07:02:09   kernel: sd 1:0:2:0: [sdd] tag#1033 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=23s
Dec 15 07:02:09   kernel: sd 1:0:2:0: [sdd] tag#1033 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:02:09   kernel: sd 1:0:2:0: [sdd] tag#1033 ASC=0x11 ASCQ=0x0 
Dec 15 07:02:09   kernel: sd 1:0:2:0: [sdd] tag#1033 CDB: opcode=0x88 88 00 00 00 00 02 70 ef e0 40 00 00 01 50 00 00
Dec 15 07:02:09   kernel: critical medium error, dev sdd, sector 10484703584 op 0x0:(READ) flags 0x0 phys_seg 6 prio class 0
Dec 15 07:02:24   kernel: sd 1:0:2:0: [sdd] tag#1047 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=9s
Dec 15 07:02:24   kernel: sd 1:0:2:0: [sdd] tag#1047 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:02:24   kernel: sd 1:0:2:0: [sdd] tag#1047 ASC=0x11 ASCQ=0x0 
Dec 15 07:02:24   kernel: sd 1:0:2:0: [sdd] tag#1047 CDB: opcode=0x88 88 00 00 00 00 02 70 f0 3a a0 00 00 01 00 00 00
Dec 15 07:02:24   kernel: critical medium error, dev sdd, sector 10484726472 op 0x0:(READ) flags 0x0 phys_seg 27 prio class 0
Dec 15 07:02:41   kernel: sd 1:0:2:0: [sdd] tag#1076 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=9s
Dec 15 07:02:41   kernel: sd 1:0:2:0: [sdd] tag#1076 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:02:41   kernel: sd 1:0:2:0: [sdd] tag#1076 ASC=0x11 ASCQ=0x0 
Dec 15 07:02:41   kernel: sd 1:0:2:0: [sdd] tag#1076 CDB: opcode=0x88 88 00 00 00 00 02 70 f0 9f 40 00 00 04 00 00 00
Dec 15 07:02:41   kernel: critical medium error, dev sdd, sector 10484752576 op 0x0:(READ) flags 0x0 phys_seg 80 prio class 0
Dec 15 07:02:51   kernel: sd 1:0:2:0: [sdd] tag#1034 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=9s
Dec 15 07:02:51   kernel: sd 1:0:2:0: [sdd] tag#1034 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:02:51   kernel: sd 1:0:2:0: [sdd] tag#1034 ASC=0x11 ASCQ=0x0 
Dec 15 07:02:51   kernel: sd 1:0:2:0: [sdd] tag#1034 CDB: opcode=0x88 88 00 00 00 00 02 70 f0 cd 38 00 00 01 30 00 00
Dec 15 07:02:51   kernel: critical medium error, dev sdd, sector 10484764112 op 0x0:(READ) flags 0x0 phys_seg 19 prio class 0
Dec 15 07:03:00   kernel: sd 1:0:2:0: [sdd] tag#1057 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=17s
Dec 15 07:03:00   kernel: sd 1:0:2:0: [sdd] tag#1057 Sense Key : 0x3 [current] [descriptor] 
Dec 15 07:03:00   kernel: sd 1:0:2:0: [sdd] tag#1057 ASC=0x11 ASCQ=0x0 
Dec 15 07:03:00   kernel: sd 1:0:2:0: [sdd] tag#1057 CDB: opcode=0x88 88 00 00 00 00 02 70 f0 d9 c8 00 00 03 18 00 00
Dec 15 07:03:00   kernel: critical medium error, dev sdd, sector 10484767728 op 0x0:(READ) flags 0x0 phys_seg 30 prio class 0

 

It goes on for hours. On the main screen that drive now has over 30,000 errors.

Now, I know this drive is crap. But I'm in the process of building parity on the second 16TB drive I've added.

My question is, it's going at a snails pace, and says it'll take 200 days to complete, and is only running at 200k/sec.

 

I've reached out to the sellers of the drives for a replacement, but what should be my course of remediation at this point?

Should I stop the array and yank the drive, and start the parity rebuild on the other drive?

[Original_Vecna]

I brought the array off line. I was getting an alert email every second. 

Removed both parity drives, brought the array back online, then shut down the server.

 

Pulled the bad drive, replaced the good 16TB into the parity drive bay and restarted. Shut the array down, added the "good" 16TB drive as my primary parity drive and restarted the array. It's rebuilding now. This is my first time buying refurb exos drives, everyone says it's the best way to get large capacity drives for your server, but this has me incredibly worried. The last 16TB drive had no errors during preclear, went the entire way of becoming a full parity drive, ran for 48 hrs, then when I added another parity drive, 5 hrs in decides to shit itself?

 

That's incredibly worrying. There were no red flags in the SMART data of the drive either...just unlucky?

Edited December 15, 2022 by Original_Vecna

[JorgeB]

Post a SMART report for that drive but it's logged as disk problem.

 

3 minutes ago, Original_Vecna said:

This is my first time buying refurb exos

I would never buy a refurbish drive, refurbish drives I got before from warranty replacements have at best a 50/50 chance of lasting more than a week.