Encryption and auto-start


Recommended Posts

  • 3 weeks later...

Thanks for this fetch/delete script solution to keeping the keyfile off the server. Just a quick note from a problem I just ran into... 

 

If you add a disk to your array, obviously the keyfile isn't there anymore for unraid to encrypt the new drive. The error given in the webUI in 6.5.3 is a little ambiguous, after clearing the disk you click Format, then it refreshes and indicates the drive is encrypted with a green lock, but that it's unmountable because of a missing encryption key. Made me afraid I had encrypted it with the wrong key and would be spending many more hours clearing it again.

 

But once you put a keyfile in /root you can click Format again and it will format correctly, but just a heads up to remember you have to do this manually when adding a drive if you go out of your way to keep the keyfile off the running machine.

Link to comment
  • 1 month later...
 
Thanks for the help
So I moved houses and me server was powered off. Once powered in it did not have internet access as I wanted to configure the network before putting my server online. I discovered that even with out internet access my array auto mounted
I was under the impression if there was no internet access to go out to the remote location ( using ftp) to get the key it would not start.

Is this true? Or did I miss something?

Sent from my BND-L34 using Tapatalk

Link to comment
  • 1 month later...
On 4/2/2018 at 9:01 AM, bonienl said:

A small variation if you want the key to be not locally present on the system when operational, the key is only needed during startup of the array.

 

In the go file the following is included before starting emhttp.


# auto unlock array
mkdir -p /usr/local/emhttp/webGui/event/starting
mkdir -p /usr/local/emhttp/webGui/event/started
mkdir -p /usr/local/emhttp/webGui/event/stopped
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting
cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped

# start webGUI
/usr/local/sbin/emhttp &

The above makes use of the built-in event system of unRAID. These events are created:

starting : this event is called before the array is started and is used to fetch the key from a remote source

started :  this event is called after the array is fully operational and is used to delete the key locally.

stopped : this event is called after the array is stopped and is used to fetch the key again from a remote source

 

The script "fetch_key"  can be any method to obtain the key remotely, e.g. using a mount method or a FTP (wget) method as explained in the video of @gridrunner

The script "delete_key" is a simple file to delete the key locally.

 

fetch_key


#!/bin/bash

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /unlock
  mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.99/index /unlock
  cp -f /unlock/somefile.png /root/keyfile
  umount /unlock
  rm -r /unlock
fi

delete_key


#!/bin/bash

rm -f /root/keyfile

You can start and stop the array as usual, and the key will be automatically fetched each time, provided that the remote service is up and running.

 

The files "fetch_key" and "delete_key" need to be stored on your flash device. I've created the folder /custom/bin to hold my custom scripts, but one is free to choose their own source folder, please update the lines in the go file accordingly.

I need some assistance getting this to work.  I add the two scripts to the flash.  the fetch_key doesn't load the key file from my windows 10 computer.

 

I checked the firewalls and that's not an issue.  What else could it be?

 

What log should I be referencing to see what happens during the boot?

Link to comment
  • 3 weeks later...
55 minutes ago, bonienl said:

Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors)

Didn't see any errors.  I modified the fetch key to the following

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /unlock
  mount -t cifs -o rw,nounix,iocharset=utf8,_netdev,file_mode=0777,dir_mode=0777,,vers=3.0,username=myuser,password=mypassword '//MEDIASERVER/index' '/mnt/disks/MEDIASERVER_index'
  cp -f /mnt/disks/MEDIASERVER_index/unlock.png /root/keyfile
#  umount /unlock
  rm -r /unlock
fi

restarted the server and got the following

Sep 23 17:09:00 Tower emhttpd: shcmd (32): echo 128 > /sys/block/sdb/queue/nr_requests
Sep 23 17:09:00 Tower emhttpd: shcmd (33): /usr/local/sbin/set_ncq sdd 1
Sep 23 17:09:00 Tower root: set_ncq: setting sdd queue_depth to 1
Sep 23 17:09:00 Tower emhttpd: shcmd (34): echo 128 > /sys/block/sdd/queue/nr_requests
Sep 23 17:09:00 Tower kernel: mdcmd (40): start STOPPED
Sep 23 17:09:00 Tower kernel: unraid: allocating 31100K for 1280 stripes (6 disks)
Sep 23 17:09:00 Tower kernel: md1: running, size: 3907018532 blocks
Sep 23 17:09:00 Tower kernel: md2: running, size: 3907018532 blocks
Sep 23 17:09:00 Tower kernel: md3: running, size: 2930266532 blocks
Sep 23 17:09:00 Tower kernel: md4: running, size: 1953514552 blocks
Sep 23 17:09:01 Tower emhttpd: shcmd (35): udevadm settle
Sep 23 17:09:01 Tower emhttpd: Opening encrypted volumes...
Sep 23 17:09:01 Tower emhttpd: shcmd (37): /usr/sbin/cryptsetup luksOpen /dev/md1 md1 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower kernel: device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: [email protected]
Sep 23 17:09:01 Tower emhttpd: shcmd (37): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (39): /usr/sbin/cryptsetup luksOpen /dev/md2 md2 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (39): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (41): /usr/sbin/cryptsetup luksOpen /dev/md3 md3 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (41): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (43): /usr/sbin/cryptsetup luksOpen /dev/md4 md4 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower avahi-daemon[11634]: Server startup complete. Host name is Tower.local. Local service cookie is 2421470760.
Sep 23 17:09:01 Tower emhttpd: shcmd (43): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (45): /usr/sbin/cryptsetup luksOpen /dev/sdf1 sdf1 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (45): exit status: 1
Sep 23 17:09:01 Tower emhttpd: Missing encryption key
Sep 23 17:09:01 Tower kernel: mdcmd (41): stop 
Sep 23 17:09:01 Tower kernel: md1: stopping
Sep 23 17:09:01 Tower kernel: md2: stopping
Sep 23 17:09:01 Tower kernel: md3: stopping
Sep 23 17:09:01 Tower kernel: md4: stopping
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/ssh.service) successfully established.
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/smb.service) successfully established.
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/sftp-ssh.service) successfully established.

Link to comment
  • 3 months later...
On 12/9/2017 at 9:13 AM, bonienl said:

unRAID 6.4 introduces encrypted volumes and allows individual disks to be protected by encryption.

image.png.6f7646e6c6fa9f4c5e814fa268bc82ba.png

The passphrase or keyfile which is used to unlock encryption is not permanently stored on the system itself, this means that upon a system start one has to re-enter the passphrase or re-select the keyfile. Until then the array can not start and the GUI will report a "missing key".

 

But what if we want to auto-start the array and not keep a local copy on the system itself? It kind of defeats the protection scheme to store a local copy, i.e. when your system get stolen, you don't want to include the keys to allow access.

 

The solution I have come up is to download the keyfile during startup from a different system in my local network which is in a different part of the house. In my case I am using an arbitrary .png file, but you could also store a passphrase and have this copied over. In my go file I added the following lines to download the keyfile from a remote SMB share before starting emhttp.


# auto unlock array
mkdir -p /unlock
mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.123/index /unlock
cp -f /unlock/#/some.png /root/keyfile
umount /unlock
rm -r /unlock

Of course the remote share must be accessible when the system starts and surely this approach can be cracked, but it is definitely safer than just keeping a local copy of the keyfile on the unRAID server itself.

 

Remember that once the array is started, the local keyfile may be deleted if you really don't want to keep the key while the system is operational.

 

Perhaps my idea is useful for others wanting to do something similar :)

 

THANK YOU!!!

 

I am adapting this to simply:

# auto unlock array
cp -f /boot/config/keyfile /root/keyfile

I copied the mounted keyfile under /root out to /boot/config/keyfile and placed this as first thing in go script.

Hope it works, rebooting now. My reasons for not having to use some other devices are simple, I still want encryption and automatic mounting without wrecking my docker containers on my encrypted and unassigned SSD and if I ever want, can just get a USB drive out and plug it into unraid as "Encryption key" or something if I need that level of safety.

 

UPDATE: It worked!

Edited by Trunkton
Link to comment
  • 2 weeks later...

So I've done a semi-equivalent of what you guys are doing here except I'm using scp to pull my key down from an AWS micro instance (might switch this to a Pi Zero on premise though - thoughts on drawbacks/benefits here?) that is always on.  I like the examples where I can delete the key after the disks are mounted which is why I stumbled onto this thread.

 

One idea I had was to somehow enhance the scp/ftp/whatever protocol process to somehow fire off a process on the remote machine that holds the key to send maybe an SMS text to my phone where I respond with a PIN code to allow the key transmission to take place.  Are there any inherent flaws with pursuing this?  One concern I have is there will obviously be a lag between when the ssh command between Unraid go script and the key and be transmitted back and then it's a race condition?  Is there a shell command on the Unraid side that I can call again to mount the drives once the file has been received?

Link to comment
  • 1 month later...

Hi everybody.

Just to be sure: if I encrypt only my data disks but not cache, will all docker containers (set to cache only) autostart with server boot or do I need to enter passphrase?

I'm asking this because I use an open VPN server, and if I'd remotely reboot the server I couldn't access web gui and thus couldnt start the array at all and think about your autostart strategies.

thanks so much

Link to comment
  • 4 months later...

I thought I'd share how you can enhanced the go file by reducing the six lines to a single command and it's not by using another script.  You can create a tar ball that contains the fetch_key and delete_key scripts. The go file calls the tar command. The tar ball files are extracted and event directories are created.

 

You MUST have a fully functioning auto-start that unlocks using the event directories. This works with FTP or SMB fetch_key scripts.

 

If you have changed the script names (fetch_key, delete_key) or changed the path where you store the scripts (/boot/custom/bin/), you will need to use your alternative names in the following procedure.

 

1) Create a tar ball call "events" from the existing files in the event directories.

 

At the terminal prompt enter the following:

tar -czf /boot/custom/bin/events -C /usr/local/ emhttp/webGui/event/starting/fetch_key emhttp/webGui/event/started/delete_key emhttp/webGui/event/stopped/fetch_key

2) Update the go file. Comment out the existing lines in order to test.

 

From:-

# auto unlock array
mkdir -p /usr/local/emhttp/webGui/event/starting
mkdir -p /usr/local/emhttp/webGui/event/started
mkdir -p /usr/local/emhttp/webGui/event/stopped
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting
cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped

To:

# auto unlock array
# mkdir -p /usr/local/emhttp/webGui/event/starting
# mkdir -p /usr/local/emhttp/webGui/event/started
# mkdir -p /usr/local/emhttp/webGui/event/stopped
# cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting
# cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started
# cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped
tar -xzf /boot/custom/bin/events -C /usr/local/

3) Once you're confident that everything works, rebooting IS necessary. You can clean up by deleting the event scripts (fetch_key, delete_key). Your files are now stored in the "event" tar ball.  And, updating the go file by removing the commented lines and any references to "unlock".

# auto start array
tar -xzf /boot/custom/bin/events -C /usr/local/

I hope some of you may find this interesting.

Edited by beckp
error in the created tar command line
  • Like 1
  • Thanks 1
Link to comment
  • 3 months later...
32 minutes ago, Nalith said:

This seem to be broken in the latest version (6.8.0-rc3), looks like the startup/shutdown scripts are not executing. Has anyone got this to work with the latest RC version?

Have you checked that they have the ‘execute’ bit set?     One of the changes in the 6.8 series is to limit the attributes of files on the flash drive.  This may mean you need to explicitly set the ‘x’ bit after copying the files off the flash drive.

Link to comment
On 10/22/2019 at 5:31 PM, itimpi said:

Have you checked that they have the ‘execute’ bit set?     One of the changes in the 6.8 series is to limit the attributes of files on the flash drive.  This may mean you need to explicitly set the ‘x’ bit after copying the files off the flash drive.

Thanks for the prompt response. I also thought that is the issue however for the life of me it seems the chmod changes are not being applied: 

 

image.png.5ce5f284c14b815b8354c1849c7dfbee.png

 

Is there something I am missing?

Link to comment
23 minutes ago, Nalith said:

Thanks for the prompt response. I also thought that is the issue however for the life of me it seems the chmod changes are not being applied: 

 

image.png.5ce5f284c14b815b8354c1849c7dfbee.png

 

Is there something I am missing?

 

The tightening of permissions on the boot drive for security reasons means it is no longer possible to have files stored on that drive with execute permissions set.   The files need to be copied off the drive to somewhere else before you can set execute permissions.

Link to comment
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.