January 20, 20233 yr A few month ago I tried writing a script in order to backup and restore LUKS headers on encrypted drives. Some interesting issues were raised by @Kilrah here, highlighting in particular the fact that the first 32kB of every drive is "reserved" by Unraid on array data drives, which could cause issues. Has anyone input to provide on the following ? should the "cryptsetup luksHeaderBackup" command be run on "/dev/sdX" (the raw drive) or on "/dev/mdX" (the mounted volume) on array data drives? should the "cryptsetup luksHeaderBackup" command be run with the array started or the array stopped, and how would that affect parity? should the "cryptsetup luksHeaderRestore" command be run on "/dev/sdX" (the raw drive) or on "/dev/mdX" (the mounted volume) on array data drives? should the "cryptsetup luksHeaderRestore" command be run with the array started or the array stopped, and how would that affect parity? I thank you very much in advance. As I said earlier, I would love to see @limetech implement backup/restore of LUKS headers directly into Unraid, as I feel I really should not play with theses scary headers. With regards, OP Edited January 20, 20233 yr by Opawesome
January 20, 20233 yr Author After some testing, it seems that "/dev/sdX" should maybe be avoided and that "/dev/sdX1" (the partition) shall be preferred. It would be awesome if someone knowledgeable could confirm.
January 20, 20233 yr Community Expert Using sdX1 would be the same as using mdX, except it won't maintain parity, and if the LUKS headers are on the partition they should be included in parity, but I've never used encryption, so not sure what would be the best way, maybe you could ask the user to start the array in militance mode when a restore is needed and that way you could use mdX.
January 20, 20233 yr Author 4 minutes ago, JorgeB said: Using sdX1 would be the same as using mdX, except it won't maintain parity, and if the LUKS headers are on the partition they should be included in parity, but I've never used encryption, so not sure what would be the best way, maybe you could ask the user to start the array in militance mode when a restore is needed and that way you could use mdX. Many Thanks. This is useful. I will have to check whether the array can be started (even in maintenance mode) if the LUKS header of one array data drive is missing, because I think I remember that an error is issues by Unraid about "missing encryption key".
January 20, 20233 yr Community Expert 6 minutes ago, Opawesome said: because I think I remember that an error is issues by Unraid about "missing encryption key". The array should still start and display that error, at least pretty sure it does.
January 21, 20233 yr Author 17 hours ago, JorgeB said: The array should still start and display that error, at least pretty sure it does. I checked and the array does not start (even in maintenance mode) if one data drive doesn't have the proper LUKS key is its header: Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (117): set md_num_stripes 1280 Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (118): set md_queue_limit 80 Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (119): set md_sync_limit 5 Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (120): set md_write_method Jan 21 11:24:29 BEETHOVEN kernel: mdcmd (121): start STOPPED Jan 21 11:24:29 BEETHOVEN kernel: unraid: allocating 20870K for 1280 stripes (4 disks) Jan 21 11:24:29 BEETHOVEN kernel: md1: running, size: 2097120 blocks Jan 21 11:24:29 BEETHOVEN kernel: md2: running, size: 2097120 blocks Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5231): udevadm settle Jan 21 11:24:29 BEETHOVEN emhttpd: Opening encrypted volumes... Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5234): /usr/sbin/cryptsetup luksOpen /dev/md1 md1 --allow-discards Jan 21 11:24:29 BEETHOVEN emhttpd: shcmd (5237): /usr/sbin/cryptsetup luksOpen /dev/md2 md2 --allow-discards Jan 21 11:24:31 BEETHOVEN emhttpd: shcmd (5240): /usr/sbin/cryptsetup luksOpen /dev/sdc1 sdc1 --allow-discards Jan 21 11:24:33 BEETHOVEN emhttpd: Missing encryption key Jan 21 11:24:33 BEETHOVEN emhttpd: shcmd (5241): /usr/sbin/cryptsetup luksClose md2 Jan 21 11:24:33 BEETHOVEN emhttpd: shcmd (5242): /usr/sbin/cryptsetup luksClose sdc1 Jan 21 11:24:33 BEETHOVEN kernel: mdcmd (122): stop Jan 21 11:24:33 BEETHOVEN kernel: md1: stopping Jan 21 11:24:33 BEETHOVEN kernel: md2: stopping So the operations should be performed on /dev/sdX1 I guess. Edited January 21, 20233 yr by Opawesome
January 21, 20233 yr Community Expert 13 minutes ago, Opawesome said: Jan 21 11:24:33 BEETHOVEN emhttpd: Missing encryption key Did you enter the encryption key?
January 21, 20233 yr Author 2 minutes ago, JorgeB said: Did you enter the encryption key? Yes. But in any case, I think that the encryption key is in the LUKS header, and is not the passphrase, which is only a way to decipher the actual encryption key in the LUKS header. Also, I think the encryption keys in the LUKS header are different on each drive. Edited January 21, 20233 yr by Opawesome
January 21, 20233 yr Community Expert If I set the devices to encrypted, and they are not, so there's no LUKS heathers, I can still start the array after entering a key:
January 21, 20233 yr Community Expert 21 minutes ago, Opawesome said: I checked and the array does not start (even in maintenance mode) if one data drive doesn't have the proper LUKS key is its header: That makes no sense since when you add a new drive it has no header until you format it, and obviously you need to start the array for that... Or is it a problem only if the header is there but not the key? Edited January 21, 20233 yr by Kilrah
January 21, 20233 yr Author I did further testing: If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive (but maintaining the LUKS header) then the array won't start, even if I tick "maintenance mode". If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive and then also "wipefs -a /dev/sdX1" to erase the LUKS remaining header (as instructed here), then the array will start, as on your system.
January 21, 20233 yr Community Expert 18 minutes ago, Opawesome said: If I run "cryptsetup erase" on an encrypted drive to clear the LUKS encryption key on that drive (but maintaining the LUKS header) then the array won't start, even if I tick "maintenance mode". I've just tried this and it worked here. array started in normal mode run backup script cryptsetup erase /dev/md2 reboot so the passphrase isn't cached enter passphrase, start array in normal mode - no problem, just obviously the disk is unmountable ("Unmountable: Volume not encrypted") cryptsetup luksHeaderRestore /dev/md2 --header-backup-file /boot/luks/luks-headers-backup-20230121115834/0QEMU_QEMU_HARDDISK_drive-scsi4.bin stop array start array in normal mode disk mounts fine parity check returns no error Edited January 21, 20233 yr by Kilrah
January 21, 20233 yr Author 3 minutes ago, Kilrah said: I've just tried this and it worked here. array started in normal mode run backup script cryptsetup erase /dev/md2 reboot so the passphrase isn't cached enter passphrase, start array in normal mode - no problem, just obviously the disk is unmountable ("Unmountable: Volume not encrypted") cryptsetup luksHeaderRestore /dev/md2 --header-backup-file /boot/luks/luks-headers-backup-20230121115834/0QEMU_QEMU_HARDDISK_drive-scsi4.bin stop array start array in normal mode disk mounts fine parity check returns no error On my former test I used "cryptsetup erase /dev/sdx1". So I did like you: array started in normal mode run backup script cryptsetup erase /dev/md1 (variation in my testing) stop array reboot (new step in my testing) > the array fails to start, as with "cryptsetup erase /dev/sdx1" Our results are clearly different.. FYI I use Unraid 6.9.2 on my testing VM.
January 21, 20233 yr Community Expert 2nd try with dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase Same, no issue starting array and restore worked, parity maintained all the way. 7 minutes ago, Opawesome said: FYI I use Unraid 6.9.2 on my testing VM. Current 6.11.5 here. Also I'm using a passphrase, not a keyfile. Edited January 21, 20233 yr by Kilrah
January 21, 20233 yr Author 1 hour ago, Kilrah said: Also I'm using a passphrase, not a keyfile. Likewise.
January 21, 20233 yr Author 2 hours ago, Kilrah said: 2nd try with dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase Same, no issue starting array and restore worked, parity maintained all the way. I tried on my system and had parity errors...: root@BEETHOVEN:/boot/luks# cryptsetup erase /dev/md1 WARNING! ======== This operation will erase all keyslots on device /dev/md1. Device will become unusable after this operation. Are you sure? (Type 'yes' in capital letters): YES root@BEETHOVEN:/boot/luks# wipefs -a /dev/md1 wipefs: error: /dev/md1: probing initialization failed: Device or resource busy root@BEETHOVEN:/boot/luks# dd if=/dev/zero of=/dev/md1 bs=1 count=16777216 16777216+0 records in 16777216+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 17.0588 s, 983 kB/s root@BEETHOVEN:/boot/luks# cryptsetup luksHeaderRestore /dev/md1 --header-backup-file "QEMU_HARDDISK_QM00011.bin" WARNING! ======== Device /dev/md1 does not contain LUKS2 header. Replacing header can destroy data on that device. Are you sure? (Type 'yes' in capital letters): YES root@BEETHOVEN:/boot/luks# Last check completed on Sat 21 Jan 2023 03:14:40 PM CET (today) Finding 63 errors Duration: 13 seconds. Average speed: 247.8 MB/sec Edited January 21, 20233 yr by Opawesome
January 21, 20233 yr Author 7 hours ago, Kilrah said: 2nd try with dd if=/dev/zero of=/dev/md2 bs=1 count=16777216 to entirely wipe the 16MiB LUKS header after the cryptsetup erase Same, no issue starting array and restore worked, parity maintained all the way. Current 6.11.5 here. Also I'm using a passphrase, not a keyfile. After updating to Unraid 6.11.5, I finally get the same results as you: I was able to start the array in maintenance mode after deleting only the LUKS keys. I was able to wipe the LUKS header completely with "wipefs -a /dev/md1" (but I had to restart the array in maintenance mode). I was able to restore the LUKS header on /dev/mdX (i.e. with the array started in maintenance mode) I did not get any error after a performing a parity check after all that. I opened a new thread here to get additional help regarding how to obtain the SERIAL_ID of the disk hosting a certain dev/mdX managed partition.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.