Hi all, I'll keep this brief and share what I know but I'm at a loss for next steps. I tried googling this error for my ZFS drives but couldn't find a thread that didn't end with the drive being corrupted and needed rebuilding

 

Long story short, Disk2 in my array is now reporting "Unmountable: Unsupported or no file system", all other disks mount correctly. This was after a hard shutdown after my server became unresponsive and crashed.

 

I attached my diagnostics, would appreciate any help on next steps. Totally prepared to rebuild array as of right now if needed.

milkmaid-diagnostics-20231111-1630.zip

Edited November 12, 20232 yr by Ribby

First thing look here, Ryzen with overclocked RAM live you have is known to corrupt data, zfs is detecting data corruption on multiple disks, so correct that, may also be a good idea to run memtest.

 

As for disk2, and after fixing the above, you can try to force import it but note that "Approximately 18 seconds of data must be discarded, irreversibly", if you want to try type:

zpool import -F disk2

then

zpool export disk2

 

Then restart the array and scrub all 3 disks, once done check for any corrupt files by typing:

zpool status -v disk#

 

15 hours ago, JorgeB said:

First thing look here, Ryzen with overclocked RAM live you have is known to corrupt data, zfs is detecting data corruption on multiple disks, so correct that, may also be a good idea to run memtest.

 

As for disk2, and after fixing the above, you can try to force import it but note that "Approximately 18 seconds of data must be discarded, irreversibly", if you want to try type:

zpool import -F disk2

then

zpool export disk2

 

Then restart the array and scrub all 3 disks, once done check for any corrupt files by typing:

zpool status -v disk#

 

ah, thank you for the heads up on the Ryzen stuff. I have done a memtest when I first built my server earlier in the year with 0 errors but I'll try it again, might find something. I'll fix that right away and try those commands since I'm facing either that or rebuilding the whole disk.

Edited November 13, 20232 yr by Ribby

17 hours ago, JorgeB said:

First thing look here, Ryzen with overclocked RAM live you have is known to corrupt data, zfs is detecting data corruption on multiple disks, so correct that, may also be a good idea to run memtest.

 

As for disk2, and after fixing the above, you can try to force import it but note that "Approximately 18 seconds of data must be discarded, irreversibly", if you want to try type:

zpool import -F disk2

then

zpool export disk2

 

Then restart the array and scrub all 3 disks, once done check for any corrupt files by typing:

zpool status -v disk#

 

Hm. zpool import -F doesn't seem to do anything or is taking forever, there's no response in terminal and it just hangs. How long should it take before I see a response?

It can take some time, but I would assume not hours, do you see any disk activity?

3 hours ago, JorgeB said:

It can take some time, but I would assume not hours, do you see any disk activity?

I let it sit for a few hours and after not seeing anything (although there was read activity) I thought I did something wrong so I rebooted the server so I could try again with a blank slate, now the server is stuck in limbo trying to mount disk2 and never continues. I can't collect diagnostics, that too also hangs and never finishes.

 

The last log I can see is "PANIC at dmu.c:1123:dmu_write()"
 

Edited November 13, 20232 yr by Ribby

Disable array auto start, reboot and try to import disk2 manually again, if still nothing you will need to wipe the fs and restore from a backup.

40 minutes ago, JorgeB said:

Disable array auto start, reboot and try to import disk2 manually again, if still nothing you will need to wipe the fs and restore from a backup.

Yeah, looks like it's gone. No activity at all anywhere. How do I go about reformatting the disk?

Click on the disk and change the filesystem to a different one, then you can start the array and reformat the disk, all data on that disk will be lost.