Jump to content
bonienl

Encryption and auto-start

40 posts in this topic Last Reply

Recommended Posts

Am trying to modify the fetch_key to use sftp over ssh with a password .ppk file

 

anyone have a good example?

 

Thanks

Myk

 

Edited by MyKroFt

Share this post


Link to post

Try:

#!/bin/bash

if [[ ! -e /root/keyfile ]]; then
  scp -i /path/to/ssh-private-key remote_user@192.168.1.99:/path/to/keyfile /root/keyfile
fi

Just remember, the ssh-private-key should have permissions 600 as well as the directory its in, so you can't use the private key directly from the USB flash drive

Share this post


Link to post

Thanks for this fetch/delete script solution to keeping the keyfile off the server. Just a quick note from a problem I just ran into... 

 

If you add a disk to your array, obviously the keyfile isn't there anymore for unraid to encrypt the new drive. The error given in the webUI in 6.5.3 is a little ambiguous, after clearing the disk you click Format, then it refreshes and indicates the drive is encrypted with a green lock, but that it's unmountable because of a missing encryption key. Made me afraid I had encrypted it with the wrong key and would be spending many more hours clearing it again.

 

But once you put a keyfile in /root you can click Format again and it will format correctly, but just a heads up to remember you have to do this manually when adding a drive if you go out of your way to keep the keyfile off the running machine.

Share this post


Link to post
 
Thanks for the help
So I moved houses and me server was powered off. Once powered in it did not have internet access as I wanted to configure the network before putting my server online. I discovered that even with out internet access my array auto mounted
I was under the impression if there was no internet access to go out to the remote location ( using ftp) to get the key it would not start.

Is this true? Or did I miss something?

Sent from my BND-L34 using Tapatalk

Share this post


Link to post

Array will start when autostart is set to enabled regardless of internet access.

When the encryption key is missing it will not mount those devices which have encryption.

 

Share this post


Link to post

Ok that makes sense, thanks

Sent from my BND-L34 using Tapatalk

Share this post


Link to post
On 4/2/2018 at 9:01 AM, bonienl said:

A small variation if you want the key to be not locally present on the system when operational, the key is only needed during startup of the array.

 

In the go file the following is included before starting emhttp.


# auto unlock array
mkdir -p /usr/local/emhttp/webGui/event/starting
mkdir -p /usr/local/emhttp/webGui/event/started
mkdir -p /usr/local/emhttp/webGui/event/stopped
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting
cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped

# start webGUI
/usr/local/sbin/emhttp &

The above makes use of the built-in event system of unRAID. These events are created:

starting : this event is called before the array is started and is used to fetch the key from a remote source

started :  this event is called after the array is fully operational and is used to delete the key locally.

stopped : this event is called after the array is stopped and is used to fetch the key again from a remote source

 

The script "fetch_key"  can be any method to obtain the key remotely, e.g. using a mount method or a FTP (wget) method as explained in the video of @gridrunner

The script "delete_key" is a simple file to delete the key locally.

 

fetch_key


#!/bin/bash

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /unlock
  mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.99/index /unlock
  cp -f /unlock/somefile.png /root/keyfile
  umount /unlock
  rm -r /unlock
fi

delete_key


#!/bin/bash

rm -f /root/keyfile

You can start and stop the array as usual, and the key will be automatically fetched each time, provided that the remote service is up and running.

 

The files "fetch_key" and "delete_key" need to be stored on your flash device. I've created the folder /custom/bin to hold my custom scripts, but one is free to choose their own source folder, please update the lines in the go file accordingly.

I need some assistance getting this to work.  I add the two scripts to the flash.  the fetch_key doesn't load the key file from my windows 10 computer.

 

I checked the firewalls and that's not an issue.  What else could it be?

 

What log should I be referencing to see what happens during the boot?

Share this post


Link to post
On 9/6/2018 at 10:52 PM, sgt_spike said:

I need some assistance getting this to work.

Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors)

Share this post


Link to post
6 hours ago, sgt_spike said:

Can I use a SMB share to access the keyphase?

Yes, if you mean an SMB share on a different system.

The code example in the post above yours is exactly doing that.

Share this post


Link to post
4 minutes ago, bonienl said:

Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors)

Thanks for the guidance.  trying it now

 

Share this post


Link to post
55 minutes ago, bonienl said:

Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors)

Didn't see any errors.  I modified the fetch key to the following

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /unlock
  mount -t cifs -o rw,nounix,iocharset=utf8,_netdev,file_mode=0777,dir_mode=0777,,vers=3.0,username=myuser,password=mypassword '//MEDIASERVER/index' '/mnt/disks/MEDIASERVER_index'
  cp -f /mnt/disks/MEDIASERVER_index/unlock.png /root/keyfile
#  umount /unlock
  rm -r /unlock
fi

restarted the server and got the following

Sep 23 17:09:00 Tower emhttpd: shcmd (32): echo 128 > /sys/block/sdb/queue/nr_requests
Sep 23 17:09:00 Tower emhttpd: shcmd (33): /usr/local/sbin/set_ncq sdd 1
Sep 23 17:09:00 Tower root: set_ncq: setting sdd queue_depth to 1
Sep 23 17:09:00 Tower emhttpd: shcmd (34): echo 128 > /sys/block/sdd/queue/nr_requests
Sep 23 17:09:00 Tower kernel: mdcmd (40): start STOPPED
Sep 23 17:09:00 Tower kernel: unraid: allocating 31100K for 1280 stripes (6 disks)
Sep 23 17:09:00 Tower kernel: md1: running, size: 3907018532 blocks
Sep 23 17:09:00 Tower kernel: md2: running, size: 3907018532 blocks
Sep 23 17:09:00 Tower kernel: md3: running, size: 2930266532 blocks
Sep 23 17:09:00 Tower kernel: md4: running, size: 1953514552 blocks
Sep 23 17:09:01 Tower emhttpd: shcmd (35): udevadm settle
Sep 23 17:09:01 Tower emhttpd: Opening encrypted volumes...
Sep 23 17:09:01 Tower emhttpd: shcmd (37): /usr/sbin/cryptsetup luksOpen /dev/md1 md1 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower kernel: device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com
Sep 23 17:09:01 Tower emhttpd: shcmd (37): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (39): /usr/sbin/cryptsetup luksOpen /dev/md2 md2 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (39): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (41): /usr/sbin/cryptsetup luksOpen /dev/md3 md3 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (41): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (43): /usr/sbin/cryptsetup luksOpen /dev/md4 md4 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower avahi-daemon[11634]: Server startup complete. Host name is Tower.local. Local service cookie is 2421470760.
Sep 23 17:09:01 Tower emhttpd: shcmd (43): exit status: 1
Sep 23 17:09:01 Tower emhttpd: shcmd (45): /usr/sbin/cryptsetup luksOpen /dev/sdf1 sdf1 --key-file /root/keyfile
Sep 23 17:09:01 Tower root: Failed to open key file.
Sep 23 17:09:01 Tower emhttpd: shcmd (45): exit status: 1
Sep 23 17:09:01 Tower emhttpd: Missing encryption key
Sep 23 17:09:01 Tower kernel: mdcmd (41): stop 
Sep 23 17:09:01 Tower kernel: md1: stopping
Sep 23 17:09:01 Tower kernel: md2: stopping
Sep 23 17:09:01 Tower kernel: md3: stopping
Sep 23 17:09:01 Tower kernel: md4: stopping
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/ssh.service) successfully established.
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/smb.service) successfully established.
Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/sftp-ssh.service) successfully established.

Share this post


Link to post

You have a chicken and egg problem...

You want to mount /mnt/disks/... but this location is not existing yet because the key is missing.

I used the temporary folder /unlock which is created in RAM. This is possbile at any time, you need to do something similar.

Share this post


Link to post
Posted (edited)
On 12/9/2017 at 9:13 AM, bonienl said:

unRAID 6.4 introduces encrypted volumes and allows individual disks to be protected by encryption.

image.png.6f7646e6c6fa9f4c5e814fa268bc82ba.png

The passphrase or keyfile which is used to unlock encryption is not permanently stored on the system itself, this means that upon a system start one has to re-enter the passphrase or re-select the keyfile. Until then the array can not start and the GUI will report a "missing key".

 

But what if we want to auto-start the array and not keep a local copy on the system itself? It kind of defeats the protection scheme to store a local copy, i.e. when your system get stolen, you don't want to include the keys to allow access.

 

The solution I have come up is to download the keyfile during startup from a different system in my local network which is in a different part of the house. In my case I am using an arbitrary .png file, but you could also store a passphrase and have this copied over. In my go file I added the following lines to download the keyfile from a remote SMB share before starting emhttp.


# auto unlock array
mkdir -p /unlock
mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.123/index /unlock
cp -f /unlock/#/some.png /root/keyfile
umount /unlock
rm -r /unlock

Of course the remote share must be accessible when the system starts and surely this approach can be cracked, but it is definitely safer than just keeping a local copy of the keyfile on the unRAID server itself.

 

Remember that once the array is started, the local keyfile may be deleted if you really don't want to keep the key while the system is operational.

 

Perhaps my idea is useful for others wanting to do something similar :)

 

THANK YOU!!!

 

I am adapting this to simply:

# auto unlock array
cp -f /boot/config/keyfile /root/keyfile

I copied the mounted keyfile under /root out to /boot/config/keyfile and placed this as first thing in go script.

Hope it works, rebooting now. My reasons for not having to use some other devices are simple, I still want encryption and automatic mounting without wrecking my docker containers on my encrypted and unassigned SSD and if I ever want, can just get a USB drive out and plug it into unraid as "Encryption key" or something if I need that level of safety.

 

UPDATE: It worked!

Edited by Trunkton

Share this post


Link to post

So I've done a semi-equivalent of what you guys are doing here except I'm using scp to pull my key down from an AWS micro instance (might switch this to a Pi Zero on premise though - thoughts on drawbacks/benefits here?) that is always on.  I like the examples where I can delete the key after the disks are mounted which is why I stumbled onto this thread.

 

One idea I had was to somehow enhance the scp/ftp/whatever protocol process to somehow fire off a process on the remote machine that holds the key to send maybe an SMS text to my phone where I respond with a PIN code to allow the key transmission to take place.  Are there any inherent flaws with pursuing this?  One concern I have is there will obviously be a lag between when the ssh command between Unraid go script and the key and be transmitted back and then it's a race condition?  Is there a shell command on the Unraid side that I can call again to mount the drives once the file has been received?

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now