Lost motherboard...maybe array?

[wildwolf]

Last week, due to unfortunate circumstances, my unRAID server went down hard. Long story short, was on UPS, wasn't aware power was gone from wall, wasn't aware UPS was already drained, and power was pulled. Server would never post after that.

Prayed hard it was power supply. It wasn't. Tried new motherboard - and it booted. Now, I'm afraid I have, again through stupidity, trashed my array.

 

I have 4 8TB drives, 2 of which are parity, and I have 7 5TB drives, and roughly 51TB  array with about 30TB used, give or take.

 

I put all the SAS cables back in the same controllers, in the same PCIe slots on the motherboard (and the replacement motherboard is exact same model as the one that died). I noticed upon boot that 2 of my 8TB drives were in parity slots up top (normal view). I saw the other 2 8TB at the bottom of my array configuration - and don't think that was normal.

 

Unfortunately, I started the array. It took 20 minutes to come up. There is no data in the array and none of my saved shares were showing. I nearly immediately shut down after that, thinking about the lost data. My fault. Yes.

 

I'm fairly sure that my 2 parity drives listed are correct.

 

parity: sdb  (8TB)

parity 2: sdf (8TB)

I'm not convinced the other drives listed as disk 1 - disk 9 are in the correct order. In fact, after the week I've had, whatever order they come up in may or may not be wrong.

Am I SOL?

 

[JonathanM]

Did you, at any point, alter the USB drive, or set a new config?

 

Unraid records the position of drives based on serial number, so as long as you are using a plain HBA or a controller that has been flashed to work as such, you can connect drives to any physical position and they will be properly connected and the array will work fine.

 

Ideally, we would need to see diagnostics collected when the array was working properly on the old setup, and also diagnostics currently. Failing that, the current diagnostics may shed some light. Attach the intact diagnostics zip file to your post.

[chigaimaro]

I agree with jonathanm, you might want to check the USB drive you used. I recently went from an older AMD (non-FX) cpu and motherboard to an Intel Coffee Lake CPU and motherboard; everything came up just fine.  Just moved over the hardware and usb drive.

 

Also, you should be able to check all your hard drives are OK using an external USB drive and another Linux machine. Someone else in the forum can correct me if this is wrong. But you should be able to just mount the non-parity drives and see if the data is still intact and the drive works.

 

Also, are your SAS controllers ok?  I know on newer motherboards with UEFI BIOS (even though the compatibility mode is turned on in the motherboard's BIOS)  HBA cards can take a long to boot unless you disable the BIOS on the SAS card themselves and maybe something is timing out in the process?
See link: LSI-9211 boots extremely slow  

I don't think you're completely out of luck.  If you can't bring up the original array, your data drives are all OK and you have at least one or two empty data drives, you could do the following:

1. Create a new array with the two parity drives and one or two of the empty data hard drives
2. After the parity sync completes, setup your shares again
3. Take one full hard drive and copy the data back to the array (either through your network shares or attaching the hard drive to the unRaid server via USB and the unassigned devices plugin with the Krusader docker container)
4. Once the full hard drive has all of its contents copied to the array, you can add the now empty drive to the array
5. Repeat steps 3 and 4 until all the drives are back into the array

 

Edited October 6, 2018 by chigaimaro

[wildwolf]

Thanks, all. I had made no changes to my USB thumb drive/configuration.

I have encrypted array, 1st attempt, I entered wrong password, then correct one. Still not sure why it took so long to mount the array. Boot time was still pretty fast.

 

unRAID indicated 3 drives were unencrypted now, all 3 were on 2nd controller card. Both controller cards are an LSI-9211. I may have put controller card in different slot than before. I cannot recall. In troubleshooting why server would not post, I had removed all cards from system. I know which controller card was in top PCI-e slot (main 1 with 8 drives), but I may have transposed the 2nd card with network card upon mobo swap.

 

Unraid told me when I mounted the array this time that the 3 drives on 2nd card were unmountable, volume not encrypted or unsupported partition layout. It's currently doing a parity-sync / data-rebuild (and writing at a whopping 150-180 MB/s, so that's great I think!)

 

Another question - should I have mounted in maintenance mode & not mounted file systems to do this instead?

After about 5-8 minutes, rebuild is at 2.1%, so I think that's moving quickly at least.

[wildwolf]

Currently at about 69% for data rebuild of 2 of the 3 drives. Guess I'll figure out what's up with the 3rd drive when it stops. It says, "Unmountable: Unsupported partition layout" right now.

Wanted to thank everybody for the information that data was likely still there, giving me courage to start the array again. Not sure what happened, most likely scenarios are:

1. controller card not fully seated, or the SAS cable wasn't fully seated, for that controller card. It's the one that has the 3 drives in question connected.

2. maybe hard drop/power outages caused drive/controller corruption/breakage - I mean it at least took out a motherboard also, so could be possible.

 

Maybe another question for my learning? Is it because I have 8 other drives & 2 parity drives, or just 2 parity drives, that allows me to rebuild these 2 drives (with a 3rd one also not in the current array?). I thought I would be limited to 2 drive failures at 1 time, but essentially, I had 3, right?

 

Thanks again!