MyKroFt Posted May 29, 2018 Share Posted May 29, 2018 (edited) Am trying to modify the fetch_key to use sftp over ssh with a password .ppk file anyone have a good example? Thanks Myk Edited May 29, 2018 by MyKroFt Quote Link to comment
ken-ji Posted May 30, 2018 Share Posted May 30, 2018 Try: #!/bin/bash if [[ ! -e /root/keyfile ]]; then scp -i /path/to/ssh-private-key [email protected]:/path/to/keyfile /root/keyfile fi Just remember, the ssh-private-key should have permissions 600 as well as the directory its in, so you can't use the private key directly from the USB flash drive Quote Link to comment
cthrumplu Posted June 15, 2018 Share Posted June 15, 2018 Thanks for this fetch/delete script solution to keeping the keyfile off the server. Just a quick note from a problem I just ran into... If you add a disk to your array, obviously the keyfile isn't there anymore for unraid to encrypt the new drive. The error given in the webUI in 6.5.3 is a little ambiguous, after clearing the disk you click Format, then it refreshes and indicates the drive is encrypted with a green lock, but that it's unmountable because of a missing encryption key. Made me afraid I had encrypted it with the wrong key and would be spending many more hours clearing it again. But once you put a keyfile in /root you can click Format again and it will format correctly, but just a heads up to remember you have to do this manually when adding a drive if you go out of your way to keep the keyfile off the running machine. Quote Link to comment
ijuarez Posted July 25, 2018 Share Posted July 25, 2018 Thanks for the helpSo I moved houses and me server was powered off. Once powered in it did not have internet access as I wanted to configure the network before putting my server online. I discovered that even with out internet access my array auto mounted I was under the impression if there was no internet access to go out to the remote location ( using ftp) to get the key it would not start. Is this true? Or did I miss something?Sent from my BND-L34 using Tapatalk Quote Link to comment
bonienl Posted July 25, 2018 Author Share Posted July 25, 2018 Array will start when autostart is set to enabled regardless of internet access. When the encryption key is missing it will not mount those devices which have encryption. Quote Link to comment
ijuarez Posted July 25, 2018 Share Posted July 25, 2018 Ok that makes sense, thanksSent from my BND-L34 using Tapatalk Quote Link to comment
sgt_spike Posted September 6, 2018 Share Posted September 6, 2018 On 4/2/2018 at 9:01 AM, bonienl said: A small variation if you want the key to be not locally present on the system when operational, the key is only needed during startup of the array. In the go file the following is included before starting emhttp. # auto unlock array mkdir -p /usr/local/emhttp/webGui/event/starting mkdir -p /usr/local/emhttp/webGui/event/started mkdir -p /usr/local/emhttp/webGui/event/stopped cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped # start webGUI /usr/local/sbin/emhttp & The above makes use of the built-in event system of unRAID. These events are created: starting : this event is called before the array is started and is used to fetch the key from a remote source started : this event is called after the array is fully operational and is used to delete the key locally. stopped : this event is called after the array is stopped and is used to fetch the key again from a remote source The script "fetch_key" can be any method to obtain the key remotely, e.g. using a mount method or a FTP (wget) method as explained in the video of @gridrunner The script "delete_key" is a simple file to delete the key locally. fetch_key #!/bin/bash if [[ ! -e /root/keyfile ]]; then mkdir -p /unlock mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.99/index /unlock cp -f /unlock/somefile.png /root/keyfile umount /unlock rm -r /unlock fi delete_key #!/bin/bash rm -f /root/keyfile You can start and stop the array as usual, and the key will be automatically fetched each time, provided that the remote service is up and running. The files "fetch_key" and "delete_key" need to be stored on your flash device. I've created the folder /custom/bin to hold my custom scripts, but one is free to choose their own source folder, please update the lines in the go file accordingly. I need some assistance getting this to work. I add the two scripts to the flash. the fetch_key doesn't load the key file from my windows 10 computer. I checked the firewalls and that's not an issue. What else could it be? What log should I be referencing to see what happens during the boot? Quote Link to comment
sgt_spike Posted September 23, 2018 Share Posted September 23, 2018 Can I use a SMB share to access the keyphase? Quote Link to comment
bonienl Posted September 23, 2018 Author Share Posted September 23, 2018 On 9/6/2018 at 10:52 PM, sgt_spike said: I need some assistance getting this to work. Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors) Quote Link to comment
bonienl Posted September 23, 2018 Author Share Posted September 23, 2018 6 hours ago, sgt_spike said: Can I use a SMB share to access the keyphase? Yes, if you mean an SMB share on a different system. The code example in the post above yours is exactly doing that. Quote Link to comment
sgt_spike Posted September 23, 2018 Share Posted September 23, 2018 4 minutes ago, bonienl said: Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors) Thanks for the guidance. trying it now Quote Link to comment
sgt_spike Posted September 23, 2018 Share Posted September 23, 2018 55 minutes ago, bonienl said: Open a terminal session and copy/paste each line (skip the 'if' condition) to see the replies (errors) Didn't see any errors. I modified the fetch key to the following if [[ ! -e /root/keyfile ]]; then mkdir -p /unlock mount -t cifs -o rw,nounix,iocharset=utf8,_netdev,file_mode=0777,dir_mode=0777,,vers=3.0,username=myuser,password=mypassword '//MEDIASERVER/index' '/mnt/disks/MEDIASERVER_index' cp -f /mnt/disks/MEDIASERVER_index/unlock.png /root/keyfile # umount /unlock rm -r /unlock fi restarted the server and got the following Sep 23 17:09:00 Tower emhttpd: shcmd (32): echo 128 > /sys/block/sdb/queue/nr_requestsSep 23 17:09:00 Tower emhttpd: shcmd (33): /usr/local/sbin/set_ncq sdd 1Sep 23 17:09:00 Tower root: set_ncq: setting sdd queue_depth to 1Sep 23 17:09:00 Tower emhttpd: shcmd (34): echo 128 > /sys/block/sdd/queue/nr_requestsSep 23 17:09:00 Tower kernel: mdcmd (40): start STOPPEDSep 23 17:09:00 Tower kernel: unraid: allocating 31100K for 1280 stripes (6 disks)Sep 23 17:09:00 Tower kernel: md1: running, size: 3907018532 blocksSep 23 17:09:00 Tower kernel: md2: running, size: 3907018532 blocksSep 23 17:09:00 Tower kernel: md3: running, size: 2930266532 blocksSep 23 17:09:00 Tower kernel: md4: running, size: 1953514552 blocksSep 23 17:09:01 Tower emhttpd: shcmd (35): udevadm settleSep 23 17:09:01 Tower emhttpd: Opening encrypted volumes...Sep 23 17:09:01 Tower emhttpd: shcmd (37): /usr/sbin/cryptsetup luksOpen /dev/md1 md1 --key-file /root/keyfileSep 23 17:09:01 Tower root: Failed to open key file.Sep 23 17:09:01 Tower kernel: device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: [email protected]Sep 23 17:09:01 Tower emhttpd: shcmd (37): exit status: 1Sep 23 17:09:01 Tower emhttpd: shcmd (39): /usr/sbin/cryptsetup luksOpen /dev/md2 md2 --key-file /root/keyfileSep 23 17:09:01 Tower root: Failed to open key file.Sep 23 17:09:01 Tower emhttpd: shcmd (39): exit status: 1Sep 23 17:09:01 Tower emhttpd: shcmd (41): /usr/sbin/cryptsetup luksOpen /dev/md3 md3 --key-file /root/keyfileSep 23 17:09:01 Tower root: Failed to open key file.Sep 23 17:09:01 Tower emhttpd: shcmd (41): exit status: 1Sep 23 17:09:01 Tower emhttpd: shcmd (43): /usr/sbin/cryptsetup luksOpen /dev/md4 md4 --key-file /root/keyfileSep 23 17:09:01 Tower root: Failed to open key file.Sep 23 17:09:01 Tower avahi-daemon[11634]: Server startup complete. Host name is Tower.local. Local service cookie is 2421470760.Sep 23 17:09:01 Tower emhttpd: shcmd (43): exit status: 1Sep 23 17:09:01 Tower emhttpd: shcmd (45): /usr/sbin/cryptsetup luksOpen /dev/sdf1 sdf1 --key-file /root/keyfileSep 23 17:09:01 Tower root: Failed to open key file.Sep 23 17:09:01 Tower emhttpd: shcmd (45): exit status: 1Sep 23 17:09:01 Tower emhttpd: Missing encryption keySep 23 17:09:01 Tower kernel: mdcmd (41): stop Sep 23 17:09:01 Tower kernel: md1: stoppingSep 23 17:09:01 Tower kernel: md2: stoppingSep 23 17:09:01 Tower kernel: md3: stoppingSep 23 17:09:01 Tower kernel: md4: stoppingSep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/ssh.service) successfully established.Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/smb.service) successfully established.Sep 23 17:09:02 Tower avahi-daemon[11634]: Service "Tower" (/services/sftp-ssh.service) successfully established. Quote Link to comment
bonienl Posted September 24, 2018 Author Share Posted September 24, 2018 You have a chicken and egg problem... You want to mount /mnt/disks/... but this location is not existing yet because the key is missing. I used the temporary folder /unlock which is created in RAM. This is possbile at any time, you need to do something similar. Quote Link to comment
Trunkton Posted January 5, 2019 Share Posted January 5, 2019 (edited) On 12/9/2017 at 9:13 AM, bonienl said: unRAID 6.4 introduces encrypted volumes and allows individual disks to be protected by encryption. The passphrase or keyfile which is used to unlock encryption is not permanently stored on the system itself, this means that upon a system start one has to re-enter the passphrase or re-select the keyfile. Until then the array can not start and the GUI will report a "missing key". But what if we want to auto-start the array and not keep a local copy on the system itself? It kind of defeats the protection scheme to store a local copy, i.e. when your system get stolen, you don't want to include the keys to allow access. The solution I have come up is to download the keyfile during startup from a different system in my local network which is in a different part of the house. In my case I am using an arbitrary .png file, but you could also store a passphrase and have this copied over. In my go file I added the following lines to download the keyfile from a remote SMB share before starting emhttp. # auto unlock array mkdir -p /unlock mount -t cifs -o user=name,password=password,iocharset=utf8 //192.168.1.123/index /unlock cp -f /unlock/#/some.png /root/keyfile umount /unlock rm -r /unlock Of course the remote share must be accessible when the system starts and surely this approach can be cracked, but it is definitely safer than just keeping a local copy of the keyfile on the unRAID server itself. Remember that once the array is started, the local keyfile may be deleted if you really don't want to keep the key while the system is operational. Perhaps my idea is useful for others wanting to do something similar THANK YOU!!! I am adapting this to simply: # auto unlock array cp -f /boot/config/keyfile /root/keyfile I copied the mounted keyfile under /root out to /boot/config/keyfile and placed this as first thing in go script. Hope it works, rebooting now. My reasons for not having to use some other devices are simple, I still want encryption and automatic mounting without wrecking my docker containers on my encrypted and unassigned SSD and if I ever want, can just get a USB drive out and plug it into unraid as "Encryption key" or something if I need that level of safety. UPDATE: It worked! Edited January 5, 2019 by Trunkton Quote Link to comment
Mike Howles Posted January 18, 2019 Share Posted January 18, 2019 So I've done a semi-equivalent of what you guys are doing here except I'm using scp to pull my key down from an AWS micro instance (might switch this to a Pi Zero on premise though - thoughts on drawbacks/benefits here?) that is always on. I like the examples where I can delete the key after the disks are mounted which is why I stumbled onto this thread. One idea I had was to somehow enhance the scp/ftp/whatever protocol process to somehow fire off a process on the remote machine that holds the key to send maybe an SMS text to my phone where I respond with a PIN code to allow the key transmission to take place. Are there any inherent flaws with pursuing this? One concern I have is there will obviously be a lag between when the ssh command between Unraid go script and the key and be transmitted back and then it's a race condition? Is there a shell command on the Unraid side that I can call again to mount the drives once the file has been received? Quote Link to comment
melagodo Posted February 26, 2019 Share Posted February 26, 2019 Hi everybody. Just to be sure: if I encrypt only my data disks but not cache, will all docker containers (set to cache only) autostart with server boot or do I need to enter passphrase? I'm asking this because I use an open VPN server, and if I'd remotely reboot the server I couldn't access web gui and thus couldnt start the array at all and think about your autostart strategies. thanks so much Quote Link to comment
bonienl Posted February 26, 2019 Author Share Posted February 26, 2019 When encryption is used then a passphrase needs to be entered to start the array. Both data disks and cache disks are part of the array. Quote Link to comment
Squid Posted February 26, 2019 Share Posted February 26, 2019 The open vm plugin will let you access the server when its stoppedSent via telekinesis Quote Link to comment
JonathanM Posted February 27, 2019 Share Posted February 27, 2019 4 hours ago, melagodo said: I use an open VPN server, and if I'd remotely reboot the server I couldn't access web gui It would probably be a good idea to set up a backup contingency, if your router has a VPN server set that up as a backup, or maybe another machine on the same lan with teamviewer. Quote Link to comment
beckp Posted July 13, 2019 Share Posted July 13, 2019 (edited) I thought I'd share how you can enhanced the go file by reducing the six lines to a single command and it's not by using another script. You can create a tar ball that contains the fetch_key and delete_key scripts. The go file calls the tar command. The tar ball files are extracted and event directories are created. You MUST have a fully functioning auto-start that unlocks using the event directories. This works with FTP or SMB fetch_key scripts. If you have changed the script names (fetch_key, delete_key) or changed the path where you store the scripts (/boot/custom/bin/), you will need to use your alternative names in the following procedure. 1) Create a tar ball call "events" from the existing files in the event directories. At the terminal prompt enter the following: tar -czf /boot/custom/bin/events -C /usr/local/ emhttp/webGui/event/starting/fetch_key emhttp/webGui/event/started/delete_key emhttp/webGui/event/stopped/fetch_key 2) Update the go file. Comment out the existing lines in order to test. From:- # auto unlock array mkdir -p /usr/local/emhttp/webGui/event/starting mkdir -p /usr/local/emhttp/webGui/event/started mkdir -p /usr/local/emhttp/webGui/event/stopped cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped To: # auto unlock array # mkdir -p /usr/local/emhttp/webGui/event/starting # mkdir -p /usr/local/emhttp/webGui/event/started # mkdir -p /usr/local/emhttp/webGui/event/stopped # cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting # cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started # cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped tar -xzf /boot/custom/bin/events -C /usr/local/ 3) Once you're confident that everything works, rebooting IS necessary. You can clean up by deleting the event scripts (fetch_key, delete_key). Your files are now stored in the "event" tar ball. And, updating the go file by removing the commented lines and any references to "unlock". # auto start array tar -xzf /boot/custom/bin/events -C /usr/local/ I hope some of you may find this interesting. Edited November 26, 2020 by beckp error in the created tar command line 1 1 Quote Link to comment
Nalith Posted October 22, 2019 Share Posted October 22, 2019 This seem to be broken in the latest version (6.8.0-rc3), looks like the startup/shutdown scripts are not executing. Has anyone got this to work with the latest RC version? Quote Link to comment
itimpi Posted October 22, 2019 Share Posted October 22, 2019 32 minutes ago, Nalith said: This seem to be broken in the latest version (6.8.0-rc3), looks like the startup/shutdown scripts are not executing. Has anyone got this to work with the latest RC version? Have you checked that they have the ‘execute’ bit set? One of the changes in the 6.8 series is to limit the attributes of files on the flash drive. This may mean you need to explicitly set the ‘x’ bit after copying the files off the flash drive. Quote Link to comment
Nalith Posted October 24, 2019 Share Posted October 24, 2019 On 10/22/2019 at 5:31 PM, itimpi said: Have you checked that they have the ‘execute’ bit set? One of the changes in the 6.8 series is to limit the attributes of files on the flash drive. This may mean you need to explicitly set the ‘x’ bit after copying the files off the flash drive. Thanks for the prompt response. I also thought that is the issue however for the life of me it seems the chmod changes are not being applied: Is there something I am missing? Quote Link to comment
itimpi Posted October 24, 2019 Share Posted October 24, 2019 23 minutes ago, Nalith said: Thanks for the prompt response. I also thought that is the issue however for the life of me it seems the chmod changes are not being applied: Is there something I am missing? The tightening of permissions on the boot drive for security reasons means it is no longer possible to have files stored on that drive with execute permissions set. The files need to be copied off the drive to somewhere else before you can set execute permissions. Quote Link to comment
JohnS Posted November 3, 2019 Share Posted November 3, 2019 All the methods above are using servers on the lan or remotely, which I can see the use case for, but could Unraid also use a similar method as Bitlockered Microsoft Windows, using an inserted USB flash drive which has the keyfile on it. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.