Encryption and auto-start

[Vaggeto]

On 5/27/2022 at 12:58 AM, Vaggeto said:

 

Has anyone had this process break once updating to 6.10 or other newer versions? I went from like 6.7 or 6.8 to 6.10.

 

It just doesn't work now but did consistently . I'm not seeing any message in the console, but I could just be missing it.

My auto-start fixed itself with 6.10.2 with no changes to the scripting.

[jojo345]

Hi folks,

 

I used for all the time the simple auto unlock script from the first post, placed in the go file.

Spoiler

Quote

# auto unlock array

mkdir -p /unlock

mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.123/index /unlock

cp -f /unlock/#/some.png /root/keyfile

umount /unlock

rm -r /unlock

And it worked allways without problems. I`m not 100% shure, but I think it happend after a plugin update like UD, UD+ or Preclear.

System was still on 6.9.2.

Now I updated to 6.10.3 in hope that it will be fixed, but its still not working.

 

If I try to mount in the console the process stocks in "kernel: CIFS: Attempting to mount \\.....\....." and the file does not get mounted.

 

Does anyone know how to fix this? Maybe some parameters have to be changed for the mount instructions?

[jojo345]

 

After a try with a clean install of Unraid 6.10.3 my auto-start still does not work anymore.

 

When I make a manual mointpoint on the main GUI, I do have acess to the share. But the instructions from the go-file wont work to mount it. --> "Attempting to mount..." and nothing happens

 

would be very happy about any help

 

 

---update---

Rightow I had another go to find the error, WOL to nas and array got stared!

No changes at all and now it works.

Edited July 29, 2022 by jojo345
update

[nick_s]

On 4/2/2018 at 6:01 AM, bonienl said:

A small variation if you want the key to be not locally present on the system when operational, the key is only needed during startup of the array.

 

In the go file the following is included before starting emhttp.

# auto unlock array
install -D /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting/fetch_key
install -D /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started/delete_key
install -D /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped/fetch_key

# start webGUI
/usr/local/sbin/emhttp &

The above makes use of the built-in event system of unRAID. These events are created:

starting : this event is called before the array is started and is used to fetch the key from a remote source

started :  this event is called after the array is fully operational and is used to delete the key locally.

stopped : this event is called after the array is stopped and is used to fetch the key again from a remote source

 

The script "fetch_key"  can be any method to obtain the key remotely, e.g. using a mount method or a FTP (wget) method as explained in the video of @gridrunner

The script "delete_key" is a simple file to delete the key locally.

 

fetch_key

#!/bin/bash

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /unlock
  mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.99/index /unlock
  cp -f /unlock/somefile.png /root/keyfile
  umount /unlock
  rm -r /unlock
fi

delete_key

#!/bin/bash

rm -f /root/keyfile

You can start and stop the array as usual, and the key will be automatically fetched each time, provided that the remote service is up and running.

 

The files "fetch_key" and "delete_key" need to be stored on your flash device. I've created the folder /custom/bin to hold my custom scripts, but one is free to choose their own source folder, please update the lines in the go file accordingly.

Hmm, why would you want the key to be fetched upon array-stop in addition to starting? Does starting mean at the start of the OS or at the start if the array? Is there documentation on these events? I haven't been able to find any.

[Salzgablah]

I'm able to run each step of the below 'fetch_key' script in terminal without issue,

#!/bin/bash

if [[ ! -e /root/keyfile ]]; then
  mkdir -p /keys
  mount -t cifs -o user='username',password='secret_key',iocharset=utf8 //'IP ADDRESS'/keys
 /keys
  cp -f /keys/AMD_array/keyfile /root/keyfile
  umount /keys
  rm -r /keys
fi

 

but when trying to run the script itself, i'm getting two errors.

root@Tower:~# bash /usr/local/emhttp/webGui/event/starting/fetch_key
/usr/local/emhttp/webGui/event/starting/fetch_key: line 2: $'\r': command not found
/usr/local/emhttp/webGui/event/starting/fetch_key: line 11: syntax error: unexpected end of file

 

My go file for reference.

#!/bin/bash

# auto unlock array by making use of events to fetch keyfile and delete it after decryption
mkdir -p /usr/local/emhttp/webGui/event/starting
mkdir -p /usr/local/emhttp/webGui/event/started
mkdir -p /usr/local/emhttp/webGui/event/stopped
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting
cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started
cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped
chmod a+x /usr/local/emhttp/webGui/event/starting/fetch_key
chmod a+x /usr/local/emhttp/webGui/event/started/delete_key
chmod a+x /usr/local/emhttp/webGui/event/stopped/fetch_key

 

Did something major change?

Edited October 13, 2022 by Salzgablah

[beckp]

Your script file contains DOS/Windows style line endings (\r\n), this is what confuses your shell. Try to save it with unix line endings (\n). 

You can execute

sed -i "s/\r//" /usr/local/emhttp/webGui/event/starting/fetch_key

 then check it again.

If that works, you'll need to convert the fetch_key and delete_key on the boot drive. You should try to simplify the go file by using the install command.

https://forums.unraid.net/topic/61973-encryption-and-auto-start/?do=findComment&comment=648148

[Salzgablah]

That worked. It removed the windows formatting. I also updated the go file to use the new install commands and that worked as well. What's the main reason for using the three install commands instead of the copy and changing permissions? Just reducing the amount of lines in go?

 

Thanks for the pointers and help.

[beckp]

Glad it worked for you.

 

Install makes a cleaner and simplified go file. Perfect for the noob. Just as you stated, it makes the directory, copies the script and sets attributes with one command. I took if a step further and reduced it to a one liner and a one file solution. Although, it's a little more complicated to setup. I shared it before the permissions needed to be set thus it was six lines at that time. Fortunately it also sets the attributes.

  

https://forums.unraid.net/topic/61973-encryption-and-auto-start/?do=findComment&comment=758563  

 

 

[FlexibleToast]

I'm looking to move from OMV to unRAID and I'm just trialing unRAID in a VM on my Proxmox cluster right now. I'm trying to setup encryption with an auto unlock. Instead of using smb, wget, or ftp, I'm using rsync so that it is ssh encrypted. My keyfile transfers just fine, but for some reason the array doesn't start. I have to log in to the web UI and click start. From that point it "just works" I don't have to enter a password. I have `Settings > Disk Settings > Enable auto start` set to yes. Is there something I'm missing?

[FlexibleToast]

14 hours ago, FlexibleToast said:

I'm looking to move from OMV to unRAID and I'm just trialing unRAID in a VM on my Proxmox cluster right now. I'm trying to setup encryption with an auto unlock. Instead of using smb, wget, or ftp, I'm using rsync so that it is ssh encrypted. My keyfile transfers just fine, but for some reason the array doesn't start. I have to log in to the web UI and click start. From that point it "just works" I don't have to enter a password. I have `Settings > Disk Settings > Enable auto start` set to yes. Is there something I'm missing?

It might have been because my array was still building the initial parity? I'm not sure but it works as expected now.

[nitrosont]

How about encrypting the array and the cache?
 

As I understood it, the array places its key under 

/root/keyfile

 

Is the location and name of a keyfile for a cache different? How do I need to proceed when I want to encrypt my cache as well?

Or can I only (do I need to) encrypt the cache with the same keyfile as the array?

[Salzgablah]

Both my array and cache drives are encrypted with the same key. So when pulling the keyfile, it unlocks and mounts all drives (array and cache). If you are using different encryption keys, I'm not sure how that would impact the process. I would recommend using the same key for all, unless you have a specific reason to use different key's...

[nitrosont]

Thank you for your reply!

I also set it up like that - now array and cache are using the same key.

 

One could say, the key(file) is system-wide.

[grana]

I want to add my take on this, thanking all the contributors of this thread but in particular @bonienl.

My passkey is a simple password, not an image, and I didn't feel comfortable in leaving it in plain text anywhere, even if just on my local network.

 

My idea is to encrypt the keyfile using an encryption key that UNRAID has readily available but that I don't have to memorize.

 

Also, I feel like mounting a remote path is overkill, I'd rather download the keyfile over scp. So, I put the keyfile on a USB key on my router; for some reason the router supports SSH, but not sftp; but supports authentication by public key.

 

So here's the script

#!/bin/bash -xv

if [[ ! -e /root/keyfile ]]; then
    # this is the equivalent of scp remote:source dest, using the host ssh key. The router does not support scp
	ssh -i /boot/config/ssh/ssh_host_rsa_key my.router.local "cat <path_to_key>/unraid_encrypted_keyfile.7z" > keyfile.7z
      
    # 7z (part of nerdtools) decompresses the file using the arg `-p` as password. The password for the decompression is the output of `cut -d' ' -f2 /boot/config/ssh/ssh_host_rsa_key.pub`, ie the host (unraid) public key.
	7z e -p`cut -d' ' -f2 /boot/config/ssh/ssh_host_rsa_key.pub` keyfile.7z
	rm keyfile.7z
	mv keyfile /root/keyfile
fi

 

I'm pretty happy with the solution. The only thing I don't like, but I guess it's minor, is that `7z` is really slow, and it takes maybe 1 second to decrypt the file (this is really surprising honestly). Given that I don't care about the compression but just the encryption, I'm wondering what better tools I should use: `gpg` is apparently not part of nerd tools any more?

 

[beckp]

Now that the Raspberry Pi Zero W is available and we’re not being scalped, I thought I’d share how to make your own key server. It is tailored to the Raspberry Pi Zero W for various reasons. A Raspberry Pi Zero 2 W could be used, it will slightly cost more and consume a little more power. Any other Raspberry Pi may or may not function properly. 

 

To save power it is configured to be headless, blue tooth and serial console disabled. The time synchronization daemon (ntpd) is not installed and time is synchronized only once a day and at boot.  The activity LED is limited to off, and two styles of flashing. 

 

I’ve attached a photo of my Diet Key Lime Pi.

 

What's in a name? Why Diet Key Lime Pi?

 

• Diet, as in DietPi, the lightweight Debian based operating system (O/S) the Diet Key Lime Pi uses.
• Key, for the key file the server needs to store.
• Lime, for Lime Technology Inc., the creator of Unraid.
• Pi, for the Raspberry Pi the server uses.
• Last, I like key lime pie.

 

Why the Raspberry Pi Zero W was chosen?

 

• The Raspberry Pi Zero W has built in WiFi. There is no physical connection to the unRAID server. It can be placed in an inconspicuous location within Wi-Fi range.
• The Raspberry Pi Zero W is the smallest form factor Raspberry Pi to have wireless connectivity. The PCB for the Raspberry Pi Zero series is approximately 2.6" x 1.2" (66mm x 30.5mm), almost credit card size.
• The Raspberry Pi Zero W is the least  expensive  Raspberry Pi to have wireless connectivity. The Raspberry Pi Zero W is approximately $15 USD.
• The Raspberry Pi Zero W is the lowest powered Raspberry Pi to have wireless connectivity.  The Raspberry Pi Zero W has only one core. My Diet Key Lime Pi consumes approximately 0.5 Watts idle. Something ideal for a system that will be operating 24/7.

 

Why was the DietPi image chosen?

 

• DietPi is an extremely lightweight Debian based operating system (O/S). It is 3X lighter than Raspberry Pi OS Lite.
• DietPi is highly optimized for minimal CPU and RAM resource usage, ensuring the Diet Key Lime Pi will always run at its maximum potential.
• DietPi boots faster than Raspberry Pi OS Lite. You want the key server up before the Unraid server needs the key file.
• DietPi can be optimized for the Diet Key Lime Pi system. Only the software the Diet Key Lime Pi needs is installed.
• DietPi allows for a complete automated installation.

 

Please find attached two files. KeyServerInstallationGuide.pdf, a tutorial to create your own Diet Key Lime Pi key server. MyKeyServer.zip, files to be used to configure the automated installation.

 

The most difficult portion of this process will be determining your static IP address. Google can be you friend here. 

KeyServerInstallationGuide.pdf MyKeyServer.zip

[cherrybullet]

Hello, would anyone know how to set this up with tailscale? I'm storing my keyfile in another offsite server connected through tailscale. Do plugins start before the go file? If not, how can I have the go file wait for it to start?

[DaFr0n]

On 12/21/2023 at 6:42 PM, beckp said:

Now that the Raspberry Pi Zero W is available and we’re not being scalped, I thought I’d share how to make your own key server. It is tailored to the Raspberry Pi Zero W for various reasons. A Raspberry Pi Zero 2 W could be used, it will slightly cost more and consume a little more power. Any other Raspberry Pi may or may not function properly. 

 

To save power it is configured to be headless, blue tooth and serial console disabled. The time synchronization daemon (ntpd) is not installed and time is synchronized only once a day and at boot.  The activity LED is limited to off, and two styles of flashing. 

 

I’ve attached a photo of my Diet Key Lime Pi.

 

What's in a name? Why Diet Key Lime Pi?

 

• Diet, as in DietPi, the lightweight Debian based operating system (O/S) the Diet Key Lime Pi uses.
• Key, for the key file the server needs to store.
• Lime, for Lime Technology Inc., the creator of Unraid.
• Pi, for the Raspberry Pi the server uses.
• Last, I like key lime pie.

 

Why the Raspberry Pi Zero W was chosen?

 

• The Raspberry Pi Zero W has built in WiFi. There is no physical connection to the unRAID server. It can be placed in an inconspicuous location within Wi-Fi range.
• The Raspberry Pi Zero W is the smallest form factor Raspberry Pi to have wireless connectivity. The PCB for the Raspberry Pi Zero series is approximately 2.6" x 1.2" (66mm x 30.5mm), almost credit card size.
• The Raspberry Pi Zero W is the least  expensive  Raspberry Pi to have wireless connectivity. The Raspberry Pi Zero W is approximately $15 USD.
• The Raspberry Pi Zero W is the lowest powered Raspberry Pi to have wireless connectivity.  The Raspberry Pi Zero W has only one core. My Diet Key Lime Pi consumes approximately 0.5 Watts idle. Something ideal for a system that will be operating 24/7.

 

Why was the DietPi image chosen?

 

• DietPi is an extremely lightweight Debian based operating system (O/S). It is 3X lighter than Raspberry Pi OS Lite.
• DietPi is highly optimized for minimal CPU and RAM resource usage, ensuring the Diet Key Lime Pi will always run at its maximum potential.
• DietPi boots faster than Raspberry Pi OS Lite. You want the key server up before the Unraid server needs the key file.
• DietPi can be optimized for the Diet Key Lime Pi system. Only the software the Diet Key Lime Pi needs is installed.
• DietPi allows for a complete automated installation.

 

Please find attached two files. KeyServerInstallationGuide.pdf, a tutorial to create your own Diet Key Lime Pi key server. MyKeyServer.zip, files to be used to configure the automated installation.

 

The most difficult portion of this process will be determining your static IP address. Google can be you friend here. 

KeyServerInstallationGuide.pdf 143.27 kB · 13 downloads MyKeyServer.zip 2.33 kB · 8 downloads

Does anyone know if this method still works, I'm struggling to get it up and running.

[beckp]

Were you able to copy your key file to the raspberry pi zero?

 

Most of the time users have errors in the go, fetch_key or delete_key files.

 

What editor did you use to create the fetch_key and delete_key scripts? The files can't contain DOS/Windows line endings.

 

Where did you save the delete_key and fetch_key files? Does it match the location defined in the go file?

 

Carefully follow bonienl's instructions.

 

https://forums.unraid.net/profile/2736-bonienl/

 

Can you manually run the fetch script? Is the /root/keyfile file present?

 

[neuer_unraider]

I have some questions:

• Is /root/keyfile stored in RAM only (i. e. not written to disk)?
• Does UNRAID constantly scan for this file in the background? i. e. I boot the server and only 8 minutes later I am placing the keyfile there. Will it automatically decrypt/start the array then?
• Is there a HTTP-API available that I can call with the encryption key to start the array?
• Does this also somehow work with a passphrase only? Aka me sending the passphrase on-the-fly and not having a keyfile at all?

Edited Saturday at 11:55 AM by neuer_unraider