Jump to content

Backup and restore of LUKS headers on ARRAY data drives


Recommended Posts

A few month ago I tried writing a script in order to backup and restore LUKS headers on encrypted drives. Some interesting issues were raised by @Kilrah here, highlighting in particular the fact that the first 32kB of every drive is "reserved" by Unraid on array data drives, which could cause issues.

 

Has anyone input to provide on the following ?

  1. should the "cryptsetup luksHeaderBackup" command be run on "/dev/sdX" (the raw drive) or on "/dev/mdX" (the mounted volume) on array data drives?
  2. should the "cryptsetup luksHeaderBackup" command be run with the array started or the array stopped, and how would that affect parity?
  3. should the "cryptsetup luksHeaderRestore" command be run on "/dev/sdX" (the raw drive) or on "/dev/mdX" (the mounted volume) on array data drives?
  4. should the "cryptsetup luksHeaderRestore" command be run with the array started or the array stopped, and how would that affect parity?

 

I thank you very much in advance.

 

As I said earlier, I would love to see @limetech implement backup/restore of LUKS headers directly into Unraid, as I feel I really should not play with theses scary headers.

 

With regards,

OP

 

 

Edited by Opawesome
  • Upvote 1
Link to comment
  • Opawesome changed the title to Backup and restore of LUKS headers on ARRAY data drives

Using sdX1 would be the same as using mdX, except it won't maintain parity, and if the LUKS headers are on the partition they should be included in parity, but I've never used encryption, so not sure what would be the best way, maybe you could ask the user to start the array in militance mode when a restore is needed and that way you could use mdX.

Link to comment
4 minutes ago, JorgeB said:

Using sdX1 would be the same as using mdX, except it won't maintain parity, and if the LUKS headers are on the partition they should be included in parity, but I've never used encryption, so not sure what would be the best way, maybe you could ask the user to start the array in militance mode when a restore is needed and that way you could use mdX.

Many Thanks. This is useful. I will have to check whether the array can be started (even in maintenance mode) if the LUKS header of one array data drive is missing, because I think I remember that an error is issues by Unraid about "missing encryption key".

Link to comment
17 hours ago, JorgeB said:

The array should still start and display that error, at least pretty sure it does.

I checked and the array does not start (even in maintenance mode) if one data drive doesn't have the proper LUKS key is its header:

Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (117): set md_num_stripes 1280
Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (118): set md_queue_limit 80
Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (119): set md_sync_limit 5
Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (120): set md_write_method
Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (121): start STOPPED
Jan 21 11:24:29 BEETHOVEN kernel: unraid: allocating 20870K for 1280 stripes (4 disks)
Jan 21 11:24:29 BEETHOVEN kernel: md1: running, size: 2097120 blocks
Jan 21 11:24:29 BEETHOVEN kernel: md2: running, size: 2097120 blocks
Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5231): udevadm settle
Jan 21 11:24:29 BEETHOVEN emhttpd: Opening encrypted volumes...
Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5234): /usr/sbin/cryptsetup luksOpen /dev/md1 md1 --allow-discards
Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5237): /usr/sbin/cryptsetup luksOpen /dev/md2 md2 --allow-discards
Jan 21 11:24:31 BEETHOVEN emhttpd: shcmd (5240): /usr/sbin/cryptsetup luksOpen /dev/sdc1 sdc1 --allow-discards
Jan 21 11:24:33 BEETHOVEN emhttpd: Missing encryption key
Jan 21 11:24:33 BEETHOVEN emhttpd: shcmd (5241): /usr/sbin/cryptsetup luksClose md2
Jan 21 11:24:33 BEETHOVEN emhttpd: shcmd (5242): /usr/sbin/cryptsetup luksClose sdc1
Jan 21 11:24:33 BEETHOVEN kernel: mdcmd (122): stop
Jan 21 11:24:33 BEETHOVEN kernel: md1: stopping
Jan 21 11:24:33 BEETHOVEN kernel: md2: stopping

 

So the operations should be performed on /dev/sdX1 I guess.

Edited by Opawesome
Link to comment
2 minutes ago, JorgeB said:

Did you enter the encryption key?

Yes.

But in any case, I think that the encryption key is in the LUKS header, and is not the passphrase, which is only a way to decipher the actual encryption key in the LUKS header.

Also, I think the encryption keys in the LUKS header are different on each drive.

Edited by Opawesome
Link to comment
21 minutes ago, Opawesome said:

I checked and the array does not start (even in maintenance mode) if one data drive doesn't have the proper LUKS key is its header:

That makes no sense since when you add a new drive it has no header until you format it, and obviously you need to start the array for that...

Or is it a problem only if the header is there but not the key?

Edited by Kilrah
Link to comment

I did further testing:

 

If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive (but maintaining the LUKS header) then the array won't start, even if I tick "maintenance mode".

 

If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive and then also "wipefs -a /dev/sdX1" to erase the LUKS remaining header (as instructed here), then the array will start, as on your system.

 

Link to comment
18 minutes ago, Opawesome said:

If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive (but maintaining the LUKS header) then the array won't start, even if I tick "maintenance mode".

I've just tried this and it worked here.

  • array started in normal mode
  • run backup script
  • cryptsetup erase /dev/md2
  • reboot so the passphrase isn't cached
  • enter passphrase, start array in normal mode - no problem, just obviously the disk is unmountable ("Unmountable: Volume not encrypted")
  • cryptsetup luksHeaderRestore /dev/md2 --header-backup-file /boot/luks/luks-headers-backup-20230121115834/0QEMU_QEMU_HARDDISK_drive-scsi4.bin 
  • stop array
  • start array in normal mode
  • disk mounts fine
  • parity check returns no error

 

Edited by Kilrah
Link to comment
3 minutes ago, Kilrah said:

I've just tried this and it worked here.

  • array started in normal mode
  • run backup script
  • cryptsetup erase /dev/md2
  • reboot so the passphrase isn't cached
  • enter passphrase, start array in normal mode - no problem, just obviously the disk is unmountable ("Unmountable: Volume not encrypted")
  • cryptsetup luksHeaderRestore /dev/md2 --header-backup-file /boot/luks/luks-headers-backup-20230121115834/0QEMU_QEMU_HARDDISK_drive-scsi4.bin 
  • stop array
  • start array in normal mode
  • disk mounts fine
  • parity check returns no error

On my former test I used "cryptsetup erase /dev/sdx1".

 

So I did like you:

  • array started in normal mode
  • run backup script
  • cryptsetup erase /dev/md1 (variation in my testing)
  • stop array
  • reboot (new step in my testing)
  • > the array fails to start, as with "cryptsetup erase /dev/sdx1"

Our results are clearly different..

 

FYI I use Unraid 6.9.2 on my testing VM.

Link to comment

2nd try with 

dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase

 

Same, no issue starting array and restore worked, parity maintained all the way.

 

7 minutes ago, Opawesome said:

FYI I use Unraid 6.9.2 on my testing VM.

Current 6.11.5 here.

 

Also I'm using a passphrase, not a keyfile.

Edited by Kilrah
Link to comment
2 hours ago, Kilrah said:

2nd try with 

dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase

 

Same, no issue starting array and restore worked, parity maintained all the way.

 

I tried on my system and had parity errors...:

 

root@BEETHOVEN:/boot/luks# cryptsetup erase /dev/md1

WARNING!
========
This operation will erase all keyslots on device /dev/md1.
Device will become unusable after this operation.

Are you sure? (Type 'yes' in capital letters): YES
root@BEETHOVEN:/boot/luks# wipefs -a /dev/md1
wipefs: error: /dev/md1: probing initialization failed: Device or resource busy
root@BEETHOVEN:/boot/luks# dd if=/dev/zero of=/dev/md1 bs=1 count=16777216
16777216+0 records in
16777216+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 17.0588 s, 983 kB/s
root@BEETHOVEN:/boot/luks# cryptsetup luksHeaderRestore /dev/md1 --header-backup-file "QEMU_HARDDISK_QM00011.bin"

WARNING!
========
Device /dev/md1 does not contain LUKS2 header. Replacing header can destroy data on that device.

Are you sure? (Type 'yes' in capital letters): YES
root@BEETHOVEN:/boot/luks# 

 

Last check completed on Sat 21 Jan 2023 03:14:40 PM CET (today)
Finding 63 errors Duration: 13 seconds. Average speed: 247.8 MB/sec

 

Edited by Opawesome
Link to comment
7 hours ago, Kilrah said:

2nd try with 

dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase

 

Same, no issue starting array and restore worked, parity maintained all the way.

 

Current 6.11.5 here.

 

Also I'm using a passphrase, not a keyfile.

After updating to Unraid 6.11.5, I finally get the same results as you:

I was able to start the array in maintenance mode after deleting only the LUKS keys.

I was able to wipe the LUKS header completely with "wipefs -a /dev/md1" (but I had to restart the array in maintenance mode).

I was able to restore the LUKS header on /dev/mdX (i.e. with the array started in maintenance mode)

I did not get any error after a performing a parity check after all that.

 

I opened a new thread here to get additional help regarding how to obtain the SERIAL_ID of the disk hosting a certain dev/mdX managed partition.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...