Long story short - I've had a power outage while not adequately prepared. I wasn't able to shutdown my server in time and the UPS ran out of juice. The server and the NetApp DS4246 disk shelf got their power cut. The disk in question was in the disk shelf.

When I powered on the server after power outage was over - I found one disk (located in the NetApp disk shelf) was missing from the array.

That same disk was showing up in the Disk Devices list with format button. And it was not reporting any SMART data.

Here's a snippet from the syslog related to this disk:

Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3393 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3393 Sense Key : 0x3 [current] 
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3393 ASC=0x11 ASCQ=0x0 
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3393 CDB: opcode=0x88 88 00 00 00 00 04 8c 3f ff 80 00 00 00 08 00 00
Jul 25 23:13:01 thePit kernel: blk_update_request: critical medium error, dev sdaa, sector 19532873600 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3394 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3394 Sense Key : 0x3 [current] 
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3394 ASC=0x11 ASCQ=0x0 
Jul 25 23:13:01 thePit kernel: sd 8:0:24:0: [sdaa] tag#3394 CDB: opcode=0x88 88 00 00 00 00 04 8c 3f ff 80 00 00 00 08 00 00
Jul 25 23:13:01 thePit kernel: blk_update_request: critical medium error, dev sdaa, sector 19532873600 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
Jul 25 23:13:01 thePit kernel: Buffer I/O error on dev sdaa, logical block 2441609200, async page read
Jul 25 23:14:02 thePit kernel: sd 8:0:24:0: [sdaa] tag#7063 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s

I tried only a couple of simple things - I restarted both the server and the disk shelf and tried swapping the disk to another slot in the shelf. That did not help. I did nothing else - I powered everything off until I had time to deal with it.

I do not have/use parity on the array. I understand I have lost all the files on that disk if it's dead/borked for good.

Questions:

1. What happened to that disk? Why is it showing up like it's not dead, while not even reporting SMART data?

2. If its only a borked filesystem problem, is there any XFS magic that can be used recover the files?

Here's the diagnostics file:

thepit-diagnostics-20260725-2206.zip

Thanks in advance for any help.

Edited yesterday at 12:04 AM1 day by shEiD

That disk is not even giving a valid SMART report, that's typically a bad disk, but you can swap cables/slots with another disk to confirm.

@JorgeB The disk is in a NetApp DS4246 disk shelf, so no cables involved (per single disk) and I have already tried changing to another slot - did not help (same result/error and no SMART data).

That confirms the disk has failed, and it needs to be replaced.