[SOLVED] Unmountable: Unsupported partition layout after data rebuild

[GTP]

Hi Everyone

 

I just lost a 4tb drive. The original drive would not boot or be recognized but it was only drive 15 of my 15 drive array. I figured since it was only one drive I could follow the replacing a failed drive directions and it would rebuild the data. Well it ran the parity sync/ data rebuild for two days and then after it was finished said "Unmountable: Unsupported partition layout" I am happy to provide what ever logs you require. Thanks you so much for your help! I just lost one drive I thought it would be ok. but I have no idea whats going on.

 

Thank you

 

James

[JorgeB]

We need the diagnostics: tools -> diagnostics

[GTP]

Thank you no problem, I have attached the log files to my reply. Thanks again I just didn't even know where to start!

 

Jamestower-diagnostics-20200427-1028.zip

[JorgeB]

Syslog doesn't show the rebuild, but both disks 15 and 16 are unmountable due to an invalid partition, that error is usually reserved for disks that were never formatted, since they are presumably the last disks you added to the array are you sure they were ever formatted?

[GTP]

You are correct I did not format them. The directions said when you are replacing a drive that was not necessary, so since I messed up should I just pull the new drives and conduct a pre-clear on the non-mounted drives and then after that is finished do the parity-sync/data rebuild with those drives?

 

Thanks for your help!

 

James

[JorgeB]

16 minutes ago, GTP said:

The directions said when you are replacing a drive that was not necessary,

It's not, and you don't want ever to do that, formatting is never part of a rebuild, what I was asking was if it was ever formatted before replacing it, the one that was replace, assuming it was only one, since you have 2 unmountable disks.

[GTP]

No they were both fresh disks from another machine. In the directions it said it was going to write over all the data so I assumed it would rebuild the file system at the same time. Let me get into a little more detail. I had a 4tb hard drive as disk 15. I restarted my array the other day and it could not find the disk no matter which power source or sata port I tried. At this point I am assuming I have a failed drive. I have 8tb of Parity so I thought that this was no big deal. So I got on the Unraid "replacing a drive" webpage and followed the directions or so I thought. I pulled the non-functional drive and plugged in two 4tb drives in that were recognized and I could assign to drive 15 and 16. I then let it go through the data rebuild process and when it finished, that's when it gave me the "Unmountable: Unsupported partition layout" error.  Thank you again for taking the time out of your day to assist me. What do you recommend I should do next?

 

Thank you,

 

James

[JorgeB]

Something is missing here, Unraid wouldn't let you replace one disk and assign a new one one at the same time.

[GTP]

So then what you are saying is I should pull disk 16 (unmount it) then rebuild the data again on just drive 15 the replacement 4tb drive?

 

thank you,

 

James

[JorgeB]

No, what I'm saying is that something is missing from your description, what you say happened is not possible, and without a complete description of what was done or the logs it's very difficult to help.

[GTP]

I apologize I will provide you with anything you need to help. I don't really know what else to say I pulled the bad disk and plugged in two 4tb disks that I had in another machine. I think I may have messed up by plugging in two 4tb disks at once instead of just plugging in one disk and rebuilding the lost disk. I thought it would rebuild it on the new replacement disk 15 and add the other 4tb disk for more space. Please feel free to ask any other questions that will help explain the issues and I am happy to provide what ever you need. Thank you so much for your help I am getting really worried now.

 

James

tower-syslog-20200427-1752.zip

Edited April 27, 2020 by GTP

[JorgeB]

9 minutes ago, GTP said:

I don't really know what else to say I pulled the bad disk and plugged in two 4tb disks that I had in another machine.

Like mentioned, that's not possible, you wouldn't be able to start the array.

 

You could try this without much risk (but array will be unprotected):

-stop the array

-unassign disk15 (if this is the one supposedly rebuilt)

-start the array

 

See if the emulated disk 15 mounts, if it doesn't post new diags.

 

 

 

 

[GTP]

I followed the steps you wrote and I am still getting the same error on disk 15 "Unmountable: Unsupported partition layout" it also said it was missing disk 16 and to add a replacement as soon as possible. I unplugged disk 16 so that makes sense. but it did not give me the option to rebuild the data on disk 15 or recognize the partition. I am so sorry to be taking so much of your time today thank you again. I have attached the latest syslog and diagnostics.

 

James

tower-syslog-20200427-1825.zip

tower-diagnostics-20200427-1331.zip

Edited April 27, 2020 by GTP
added diagnostics

[JorgeB]

I told you to unassign disk15, you unassigned disk16...

[JorgeB]

Disk16 should be clear but since I don't want to assume anything in this case, re-assign disk16 and rebuild it (it will still show unmountable), when the rebuild finishes unassign disk15 and start the array, then post new diags.

[GTP]

1 hour ago, johnnie.black said:

Something is missing here, Unraid wouldn't let you replace one disk and assign a new one one at the same time.

Ok I am just plugged in both drives again and I am at the drive mount screen I see both drives that were mounted in the 15 and 16 slots in the image can I send you a picture? If I try to change drive 16  or 15 to no device it says "too many wrong or missing disks"? The start button is also not active.

 

Its like I has to have both drives mounted or it wont start the array but it also won't mount the partition once it does start one of those drives.

 

Thank you,

 

James

[GTP]

I also tried the same on drive 16 and received the same error.

 

Thank you,

 

James

[GTP]

Ok when I tried assigning drive 15 to the array it started  but it is still throwing that "Unmountable: Unsupported partition layout" error   on the drive. It does not give me the option to rebuild.

tower-diagnostics-20200427-1405.zip

Edited April 27, 2020 by GTP
added diagnostics

[GTP]

I don't want to do anything wrong and lose my data if possible so I have not touched it in any other ways other than the ways I have described. I am willing to try anything even if I have to lose the 3TB on drive 15 I am just worried about the other 14 drives of data in the array. I just want it to work again. I am stumped that the drive replacement procedure did not work. Thank again for your help and I will wait for your expertise before I try anything else. 

 

Thank you,

 

James

tower-diagnostics-20200427-1622.zip

Edited April 27, 2020 by GTP
added diagnostics

[JorgeB]

I very much doubt there's any data to recover, but please try to follow the instructions:

 

13 hours ago, johnnie.black said:

re-assign disk16 and rebuild it (it will still show unmountable)

 

[jonp]

Hi there,

 

While johnnie has been doing an amazing job guiding you, I wanted to chime in considering the email you sent to support.  From reading through this thread, the main issue here is that something happened between your disk failing and the replacement process that shouldn't have.  When replacing a failed disk, the procedure is simply to stop the array, remove the failed device from the system, install a replacement device, select the replacement from the Main tab (on the disk slot that needs replacing), and start the array.

 

The only other option you can try would be to start the array in maintenance mode and attempt to run a filesystem check/repair.  It is possible that while the disk was failing, some data got corrupted and caused a filesystem corruption.  While highly unlikely, it can happen.  If so, a filesystem check/repair may resolve it.  To do this, stop the array and start it in maintenance mode (there is a checkbox for this option on the Main tab).  Then click the device on the main tab and you'll see an option for doing a filesystem check.  Click the help button in the top right so the help text displays and you can see what options you have to use to try the repair.

 

If that doesn't work, there is a good chance you may have lost the data on that device.

[GTP]

I ordered a new replacement 8tb drive, should I wait to try to start the array in maintenance mode and pull the two 4tb drives and then do a data rebuild on the new drive or should I try starting the array in maintenance mode with the two 4tb and try to check/repair drive 15. Also when you say I might have lost the data on that device does that include the entire array or just that drive? Please say its just that drive..... 

 

Thank you for your help,

 

James

[JorgeB]

1 hour ago, GTP said:

pull the two 4tb drives and then do a data rebuild on the new drive

That won't work, you can't remove two drives with a single parity.

 

1 hour ago, GTP said:

should I try starting the array in maintenance mode with the two 4tb and try to check/repair drive 15.

You can do that, but the invalid partition error can't be fixed by a filesystem check, why I suggested unassigning disk 15, Unraid would recreate the partition and we could see if a valid filesystem exists or not on the emulated disk, I suspect it doesn't.

 

1 hour ago, GTP said:

Also when you say I might have lost the data on that device does that include the entire array or just that drive? Please say its just that drive..... 

Only that drive's data would be lost, assuming the old disk is really dead.

 

 

 

My best guess of what happened here:

 

-disk15 failed (assuming it got disable, even if it didn't it wouldn't change the below)

-you replaced disk15 with a new disk and at the same time added disk16

-Unraid wouldn't let you start the array, it's impossible to replace one disk (disabled or not) and add another disk and start the array, you'd get the error "Invalid expansion.  - You may not add new disk(s) and also remove existing disk(s).", this error isn't very helpful, it's an old bug.

-you did a new config, maintained all assignments, including the new disks and started the array, a parity sync/data rebuild would begin (does this ring any bells?)

-Parity sync finished and both disks 15 and 16 are unmoutable with an invalid partition since they were never formatted or rebuilt.

 

The only other explanation I can think of, though it seems less likely to me, would be some flash drive trouble that would allow you to do something that is not possible on a normally functioning Unraid server.

 

 

[GTP]

14 minutes ago, johnnie.black said:

-you did a new config, maintained all assignments, including the new disks and started the array, a parity sync/data rebuild would begin (does this ring any bells?)

-Parity sync finished and both disks 15 and 16 are unmoutable with an invalid partition since they were never formatted or rebuilt.

Yes, I think you called the ball. This sounds exactly what happened. Is there anything I can do now? I am fine with losing 3 tb of data if I messed something up during the rebuild and I have a new 8tb drive being shipped to me that will be fresh out of the box. I will do whatever you fine gentleman suggest to save the other 93tb of data even if I have to lose both those 4 tb drives or the one that it had "rebuilt". Should I try the check file system on drive 15 again because last time I tried it said "Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!! attempting to find secondary superblock........."

 

Thanks again guys you are saving my life! 

 

James 

Edited April 28, 2020 by GTP
Punctuation

[GTP]

17 minutes ago, johnnie.black said:

Only that drive's data would be lost, assuming the old disk is really dead.

One question about this assuming I can't get drive 15 to be recognized or repaired and I get the new 8tb drive Thursday what do I do with drive 15 and 16. How do I go about adding the new 8tb drive and then just losing the 3 tb that I had on drive 15. Should I just pull drive 15 replace it with the new 8 tb and start the array and let it rebuild again or should I put it as an unassigned drive and format it first. I of course plan on pre-clearing it as an un-assigned drive as well. I am sorry I am just gun shy after this experience.

 

Thank you,

 

James