Disks formatted with Ubuntu report unsupported partition layout

[jsclayton]

A few years ago I got started with unRAID, but switched to vanilla Ubuntu after 2 flash drives fried themselves in a short timespan. Thankfully the disks are using a standard LUKS container, so I was able to easily mount them under Ubuntu. Since then I've added a number of disks, but kept the same formatting strategy in case I ever wanted to move back.

 

Well, today is that day. I popped in a fresh USB stick, setup the disks in the same order that they are under Ubuntu, entered the password, and find that 10 of the 24 drives are reporting:

Unmountable: Unsupported partition layout

I'm 98% certain that the drives reporting this error are new drives I'd added since moving to Ubuntu. lsblk correctly identifiers them as crypto_LUKS, same as the ones that work. I can unlock and mount them fine using the console and cryptSetup. Even using the Unassigned Devices plugin since they all use the same password I can click Mount and they unlock and mount fine!

 

If I print the partition layout with fdisk the only difference that I can see is that the start sector is different. On unRAID formatted drives (ie /dev/sdm) the partition starts at sector 64 and from Ubuntu (ie /dev/sdaa) it starts at sector 2048:

Disk /dev/sdm: 9.1 TiB, 10000831348736 bytes, 19532873728 sectors
Disk model: WDC WD100EMAZ-00
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: C75B0021-E883-4040-AE3E-28D6668AF385

Device     Start         End     Sectors  Size Type
/dev/sdm1     64 19532873694 19532873631  9.1T Linux filesystem


Disk /dev/sdaa: 9.1 TiB, 10000831348736 bytes, 19532873728 sectors
Disk model: WDC WD100EZAZ-11
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3CFA9EA6-E779-9742-9C36-5CCF361A52FE

Device     Start         End     Sectors  Size Type
/dev/sdaa1  2048 19532873694 19532871647  9.1T Linux filesystem

Clearly the Ubuntu-formatted drives are somehow different from what unRAID expects. The question is, is there anything I can do to get them to be recognized short of painstakingly mounting each one, copying to the little free space on the array that works, reformatting in unRAID, rinsing, and repeating? Is there some xattr or label or something hidden on the drive of file system or device somewhere that will indicate to unRAID that this drive is formatted as expected?

tower-diagnostics-20200704-0905.zip

[itimpi]

The problem is likely to be that UnRAID expects a very specific partition layout and it sounds as if these drives partitions do not conform.   

[jsclayton]

Any idea what specifically unRAID would be looking for?

[itimpi]

7 minutes ago, jsclayton said:

Any idea what specifically unRAID would be looking for?

I am afraid not.     I assume it is the partition type and the sector it starts on but that is just a guess. 

[trurl]

Let's see if @johnnie.black has anything.

[Lev]

1 hour ago, itimpi said:

The problem is likely to be that UnRAID expects a very specific partition layout and it sounds as if these drives partitions do not conform.   

This is correct. UnRAID's starting sector is 64, it used to be 63. Never 2048.

 

I know it sucks to hear. But consider it a good excuse to buy one more Easystore to shuck and make the long copy process a little less painful 😀

[JorgeB]

Unraid requires a specif partition layout and more importantly a specific MBR signature, if you have or add parity, and since partition info is outside parity and recreated by Unraid, you should be able to rebuild one disk at at time (or two with dual parity), this is for example what most people who change from RAID to non RAID controllers also must do, and it always worked AFAIK, but you can check the emulated disk first and make sure it is mounting and data looks correct before rebuilding on top, other than that you'd need to backup each disk and have Unradi re-formatted them, then restore the data.

[jsclayton]

Thanks @johnnie.black - I'll do some experimenting with the disk I've been using for parity - i'll see if partitioning with an explicit start sector makes the difference to portability.

[JonathanM]

18 hours ago, jsclayton said:

I'll do some experimenting with the disk I've been using for parity - i'll see if partitioning with an explicit start sector makes the difference to portability.

What he was trying to tell you is that if you have parity built using the full set of disks you intend to use with unraid, you should be able to force fail a single drive at a time and unraid will move the partition for you.

 

non-correcting parity check, zero errors, one or more drives showing unmountable because of partition layout

physically disconnect one of the unmountable drives with the array powered down

start the array, the disconnected disk should now be mountable and accessible, if so, you can safely rebuild to the same physical disk

 

Repeat that sequence until all drives are corrected.

 

Now, having said all that, it sounds like you may not have full backups. If that's the case, you are playing with fire. Recovery of encrypted volumes that get corrupted is very difficult or impossible sometimes, so I don't recommend experimenting until you have current backups.

[jsclayton]

@jonathanm no fire to play with, not even smoke.  

 

To clarify, since switching to Ubuntu I've been formatting new drives the same way. I have a single parity disk I've been using for SnapRaid, which is the one I was going to attempt to "fix" the partition on and see if it is recognized by unRAID. If that doesn't work (and I don't have high hopes that changing the start sector by itself works) there's nothing critical on it and I can try repartitioning it with an explicit start sector. If that works, I can play shell games with the other 10 data disks shuffling data off, repartitioning, and repeat. 

 

Not 100% that I'll be switching back to unRAID right now, but I very much like the option to be able to easily hence trying to figure out the partitioning. 

[JorgeB]

3 minutes ago, jsclayton said:

and I don't have high hopes that changing the start sector by itself works

It won't, it needs a special signature.

[JorgeB]

I guess you could copy the MBR of a working disk with dd and write it back to one of the new ones, if you feel comfortable with dd, after correcting the partition start size, I never tried that but it should work.