I can't believe what I have done, at least what I think I have done.

 

I was adding a new parity disk (that was not precleared) so I used the "New Config" tool to both add a parity drive and reorder the drives.

 

When I started the array, two of my 10TB disks completely error-ed out (block level I/O). I repeatedly tried to cancel the Parity process which froze the machine. I did a hard shutdown, rebooted, checked the SATA cables, and tried again. Same thing happened, except the machine didn't fully freeze. I got the logs (attached) and then did another hard shutdown.

 

I removed the parity drive, thinking it was the causes of these weird issues, used the new config tool, and restarted the array.

After removing the parity drive and restarting, my 10TB do not have any block-level IO problems now. However, accessing my user shares, it seems that I have lost a lot of data (most of my photos, some video, etc). Not sure how extensive the loss it.

The only thing I can think of is that when I started the array it tried to rebuild the array using the invalid parity drive!?!? I definitely did not think this was the default behavior and I did not select any such options.

 

I'm hyperventilating. Any advice?

mastertower-diagnostics-20220717-1227.zip

24 minutes ago, ZipsServer said:

only thing I can think of is that when I started the array it tried to rebuild the array using the invalid parity drive

No it didn't do that.

 

Rebuild only writes to the replaced drive. New Config only writes to the parity drive. All other array disks are only read, and all other array disks are independent filesystems that can be read without any of the other array disks.

 

Looks like you have 9 data disks in the array, all are mounted and have contents. And you have 3 pools, all mounted and with contents.

 

Something has just broken user shares. For example

 

Connection problems on multiple disks. Looks like some of your drives are external. USB connections are not recommended for array or pools.

 

Also, you should always double check all connections when you are mucking about inside.

 

Post a screenshot of Main - Array Devices, and another screenshot of Main - Pool Devices.

 

 

 

 

It is amazing how adrenaline can narrow one's focus. Thanks for bringing me back up to the surface.

 

I forgot to adjust the User Share setting to include the new disk I added to the array. So when I reordered the drives, a drive with data on it was excluded from being included in the User Shares. That's embarrassing, yikes. It has been awhile since I have performed any real maintained on the server....

 

But yes, I know that USB drives are not optimal, unfortunately I have run out of room and SATA ports with my current enclosure and setup. Need to move to a rack mount system or add a new SAS JBOD card and some 3D printed drive mounts... but that takes more time than I have atm

 

Thanks trurl!

The drive errors when adding the parity to the array is another problem, for which I can create a less dramatic thead topic..

6 minutes ago, ZipsServer said:

USB drives are not optimal

I would only use them outside the parity array, either unassigned or in single disk pools. That way each USB drive has no effect on the array or other pools. Array drives need to be in sync with parity, and if they disconnect you have to rebuild. Similar for multidrive pools.