December 3, 200916 yr Hi guys, I got encFS working on unRAID 4.5-beta12. It was pretty straightforward. 1) Download the following packages into /boot/packages : http://repository.slacky.eu/slackware-12.2/system/encfs/1.5/encfs-1.5-i486-2kc.tgz http://repository.slacky.eu/slackware-12.2/system/rlog/1.4/rlog-1.4-i486-1kc.tgz ftp://slackware.opennix.com/pub/slackbuilds/slackware-12.2/development/boost-1.39.0-i686-1_mm.tgz 2) And then install them: installpkg /boot/packages/boost-1.39.0-i686-1_mm.tgz installpkg /boot/packages/rlog-1.4-i486-1kc.tgz installpkg /boot/packages/encfs-1.5-i486-2kc.tgz 3) That's it... You are now in business! Let's go over a quick tutorial: - Create a couple of directories mkdir /mnt/disk2/encrypted mkdir /mnt/disk2/decrypted Now it is time to initialize the encrypted space encfs /mnt/disk2/encrypted /mnt/disk2/decrypted Creating new encrypted volume. Please choose from one of the following options: enter "x" for expert configuration mode, enter "p" for pre-configured paranoia mode, anything else, or an empty line will select standard mode. ?> You can go for the default settings for testing however I would recommend the settings found below to avoid putting too much load on the processor and also to permit hardlinks (if you need them). So for now just press enter. Standard configuration selected. Configuration finished. The filesystem to be created has the following properties: Filesystem cipher: "ssl/aes", version 2:2:1 Filename encoding: "nameio/block", version 3:0:1 Key Size: 192 bits Block Size: 1024 bytes Each file contains 8 byte header with unique IV data. Filenames encoded using IV chaining mode. File holes passed through to ciphertext. Now you will need to enter a password for your filesystem. You will need to remember this password, as there is absolutely no recovery mechanism. However, the password can be changed later using encfsctl. New Encfs Password: Verify Encfs Password: Time to test it now. cd /mnt/disk2/decrypted cp /var/log/dmesg . ls -al total 28 drwxr-xr-x 2 root root 120 Dec 3 18:17 ./ drwx------ 7 root root 168 Dec 3 18:16 ../ -rw-r--r-- 1 root root 26550 Dec 3 18:17 dmesg Let's move to the encrypted version. cd /mnt/disk2/encrypted ls -al total 32 drwxr-xr-x 2 root root 120 Dec 3 18:17 ./ drwx------ 7 root root 168 Dec 3 18:16 ../ -rw-r--r-- 1 root root 1072 Dec 3 18:16 .encfs6.xml -rw-r--r-- 1 root root 26558 Dec 3 18:17 2xptcJA0VdAgGEg4KBU1PGcP The .xml file contains the encfs configuration including an encrypted copy of the key material. The other file is the encrypted version of dmesg. Now let's unmount the secure area: fusermount -u /mnt/disk2/decrypted Let's see what it looks like: cd /mnt/disk2/decrypted ls -al total 0 drwxr-xr-x 2 root root 48 Dec 3 18:16 ./ drwx------ 7 root root 168 Dec 3 18:16 ../ Yep...it is empty because not mounted. We can check the encrypted directory now. cd /mnt/disk2/encrypted ls -al total 32 drwxr-xr-x 2 root root 120 Dec 3 18:17 ./ drwx------ 7 root root 168 Dec 3 18:16 ../ -rw-r--r-- 1 root root 1072 Dec 3 18:16 .encfs6.xml -rw-r--r-- 1 root root 26558 Dec 3 18:17 2xptcJA0VdAgGEg4KBU1PGcP There is a switch for encFS called '--public'. It is required if you want other users to access the mount. It is required if you mount your encrypted directory as user x and want user x or y (or root) to acesss it. Although I didn't need it to access my encrypted share mounted by root over SMB. Regarding encFS settings, there is a pre-defined mode called paranoia mode. This mode slightly increase security but also will add significant load to your CPU and furthermore will prevent you from using hardlinks. For my encfs mount I use the following custom mode that can be set during encFS setup by pressing 'x'. Enter 'x' for customer encfs settings then use the following settings: * Enter ‘y’ to create the encrypted directory * Enter ‘y’ to create the unencrypted directory * Enter ‘x’ to choose expert Mode * Enter ‘1? to use the AES : 16 byte block cipher * Enter ‘256? for the key size * Enter ‘1024? for the block size * Enter ‘1? for block filename encoding * Enter ‘y’ for filename initialization vector chaining * Enter ‘y’ for per-file initialization vectors * Enter 'n' for filename to IV header chaining * Enter ‘n’ for block authentication code headers * Enter ‘y’ for file-hole pass-through The last thing I need to check is the automatic mounting of the encFS directory upon reboot. I'll keep you posted. Alphazo [EDIT] For higher throughput you might want to change the above custom settings for Blowfish and 4096-block size. Read more here: http://lime-technology.com/forum/index.php?topic=4804.msg46566#msg46566
December 3, 200916 yr Thanks for the write up alphazo. I will play around with this, but I will need to do plenty of testing before I switch my encryption methods. What CPU do you have?
December 3, 200916 yr Author My unRAID server is based on a modest Intel D945GCLF2 with 2GB of RAM. I haven't loaded it yet with content so I don't even know how it will work with unRAID RAID technology. Encryption will probably impact throughput. Even if not used on unRAID side it can still be worth using it on the client side using a regular user share. As an example there are fun things to do with encFS and online storage services such as DropBox for example. Anyway, we should definitely keep this topic alive and share our experience. Alphazo
December 3, 200916 yr Very nice writeup. Once you have an encrypted file system folder on a disk, how do you re-establish/re-mount the "decrypted" space after you have un-mounted it so you can get access to your files the next time you want to get to them? Joe L.
December 4, 200916 yr Very nice writeup. Once you have an encrypted file system folder on a disk, how do you re-establish/re-mount the "decrypted" space after you have un-mounted it so you can get access to your files the next time you want to get to them? Joe L. Basically run the same encfs /encrypted/folder/ /decrypted/mountpoint/ command you used to create it in the first place. It will recognise there is already an encrypted directory there and ask you for the passphrase to mount. From the other thread : I'm having a curious issue where creating a new encfs filesystem from scratch works with no problem and you can write to it (although doing this via a user share kills performance - due to going through two layers of fuse?) However if you unmount the filesystem (fusermount -u) and then remount, any writes you send to it will stall the system disk i/o to anything until the encfs process is killed. Anyone else tried? update : this seems to be quite random and is made worse by trying to write a large file. Smaller files seem to go through sometimes instantly, sometimes with a delay of seconds to minutes. Very odd.
December 4, 200916 yr Author You can even add the following block to your 'go' script so you can be nicely prompted for your encFS password on boot for automatic mount of the encrypted directory. installpkg /boot/packages/boost-1.39.0-i686-1_mm.tgz installpkg /boot/packages/rlog-1.4-i486-1kc.tgz installpkg /boot/packages/encfs-1.5-i486-2kc.tgz encfs /mnt/disk2/encrypted /mnt/disk2/decrypted Now I agree with boof, it needs to be tested (both from reliability and performance standpoint) before going to production. Alphazo.
December 4, 200916 yr Now I agree with boof, it needs to be tested (both from reliability and performance standpoint) before going to production. Alphazo. Do you see the same issue? I noticed you've used a different version of boost than me - I got mine from the same slackware repo as the other packages and is versioned as : boost-1.39.0-i686-1as.tgz Maybe this could cause an issue? Is there a reason you went elsewhere for your boost library and ended up with a different version?
December 4, 200916 yr Author I took the boost package mentioned above because I didn't notice it was also available in the same repository of the two others. I did what you suggested and after unmounting the encFS dir and remounting it I could get the same (slow) transfer speed on both 100MB and 700MB files. Here are the results of my non-scientific tests running on a 100Mbit network (only) and without any cache drive. Under my ArchLinux box, I didn't mount a 'user share' but rather the disk2 via mount.cifs //192.168.0.11/disk2 /home/alphazo/testmount Under disk2 there are: - testdir (test dir without encryption) - decrypted (directory encrypted via encFS) I then copied a 700MB and 100MB file to and from the mounted directories with the following results: - Send 700MB to unRAID (testdir) : 1'24" - Receive 700MB from unRAID (testdir) : 1'14" - Send 95MB to unRAID (testdir) : 0'12" - Receive 95MB from unRAID (testdir) : 0'12" - Send 700MB to unRAID (decrypted) : 2'25" - Receive 700MB from unRAID (decrypted) : 1'37" - Send 95MB to unRAID (decrypted) : 0'20" - Receive 95MB from unRAID (decrypted) : 0'12" Even without encryption it seems pretty, especially for fetching data. Is this a normal behavior when running on a 100Mb/s network? Alphazo
December 4, 200916 yr Using a custom encryption setup - with the lightest algorithm and smallest keysize I could get 'usual' speeds to the array going direct to the disk shares. So around 20-40 megabytes per second. (This is using the latest betas with the speed tweaks). Doing the same via a user share crippled performance to around 4-5 megabytes per second. When it worked (see above) I have a relatively slow celeron so perhaps I was just being too demanding. I'll try again over the weekend. your transfer rates will be dependant unraid side on if you go via the user share(s) or not and at encfs side by what permutation of crypto options you choose when creating the folder. And your CPU... I've read some things about encfs having backwards compatibility issues when upgrading encfs and / or fuse which concerns me but nothing conclusive yet. I wouldn't want to encrypt a terbayte of data in unraid and be locked out if limetech upgraded unraid with a new Fuse version. It's a nice solution, I just hope I can narrow down or eliminate the hangs I saw. Although if you didn't see the same issue it implies something to do with my configuration. I was doing all i/o via samba and using the 4.5 beta release(s).
December 4, 200916 yr Bit more testing - done remotely via ssh. It seems to behave itself perfectly well via the command line. I can make 5000 (empty) files in a reasonable space of time, read them back remove them etc with no hiccups or locking. I can also copy a large (33gig blu ray image) file with no issues (other than speed if going via user shares. Again no real overhead if going direct to disk share) I did notice at the start of the copy there is a delay, possibly whilst encfs is setting things up with regards to the file protection it does. This may just be being exaggerated by samba over the gigabit link between client and unraid. Either way it points to samba being upset for some reason. When I'm back on the same network I'll run encfs in debug and see if it throws anything up during samba writes.
December 4, 200916 yr http://code.google.com/p/encfs/issues/detail?id=25 *sounds* very similar. I wonder if I'll see the same. p.s sorry to keep spamming your thread alphazo. I'm very interested in using this / making sure it works and I hadn't even heard of it yesterday until you posted your thread on it!
December 4, 200916 yr Author http://code.google.com/p/encfs/issues/detail?id=25 *sounds* very similar. I wonder if I'll see the same. p.s sorry to keep spamming your thread alphazo. I'm very interested in using this / making sure it works and I hadn't even heard of it yesterday until you posted your thread on it! http://code.google.com/p/encfs/issues/detail?id=25 *sounds* very similar. I wonder if I'll see the same. No problem. I'm glad that other unRAID users are interested by this idea. And if there are bugs or limitations they need to be known. Right now, as my initial tests revealed some pretty bad transfer rate even without encFS (it is a brand new installation), I'm going to investigate that first. As a side note, have you tried mounting the encrypted dir with the --public option? Cheers Alphazo
December 4, 200916 yr As a side note, have you tried mounting the encrypted dir with the --public option? Cheers Alphazo No - I didn't think I needed it as everything under unraid goes via root. My client (via samba) could see all files etc in the encfs mounted directory with no problems. It was just the writing. However, the clients samba user will be being mapped to root at somepoint so it's definitely worth a try to see if --public changes behavior!
December 4, 200916 yr Author Couldn't shutdown my server as encFS was still in use. I manually needed to 'fusermount -u /mountpoint'. What is the most elegant way to perform this action on shutdown (powerdown script?) and considering that some files my still be in use (force option?) ? Thanks Alphazo
December 4, 200916 yr Couldn't shutdown my server as encFS was still in use. I manually needed to 'fusermount -u /mountpoint'. What is the most elegant way to perform this action on shutdown (powerdown script?) and considering that some files my still be in use (force option?) ? Thanks Alphazo At this time, there is no elegant method. The powerdown script will kill processes holding files open, but I do not think it will un-mount file-systems from mount points. You could, of course, extend it. Be very careful though, in recent versions of unRAIDTom has included a "powerdown" script that resides in /usr/local/sbin. It will execute before the one you are thinking of in /sbin/powerdown as it is first in the execution $PATH. Tom's version simply uses "wget" to press the "Power Down" button on the web-interface (even though it is not visible) and from one report, apparently powers down the server without stopping the array cleanly first. (I've not verified this) This will force a parity check when next powered up. Eventually, we will have a pre-array-stop event from emhttp... Look for it some-time after the 5.0 beta series is in development. In the interim, you'll need to stop encFS and un-mount the FUSE file-system as you've already learned before the array will finish the disk un-mounting process and stop. Joe L.
December 4, 200916 yr I'm interested in this project too. Boof, The backward compatibility issue with fuse could be a deal breaker. I will play around with encFS this weekend and post some results. What is the purpose if the "File holes option"? In the google link (which is a year old), they point to that option as the culprit.
December 4, 200916 yr I'm interested in this project too. Boof, The backward compatibility issue with fuse could be a deal breaker. I will play around with encFS this weekend and post some results. What is the purpose if the "File holes option"? In the google link (which is a year old), they point to that option as the culprit. My understanding is a recompile of encfs against the new fuse version will fix it. So, in theory, as long as you keep the version of encfs in lockstep with the slakcware version of unraid from the official slackware repos you might be fine? No idea what file holes does It is a new feature (enable / disable) as of 1.5 which wasn't in 1.4 though. My concern is that its marked as un reproducable and hasn't been further addressed! The last release was a year ago though and the svn is in heavy development recently it would seem. I couldn't get the svn to compile on my ubuntu desktop at work - but I didn't try very hard I've yet to test it with encfs in verbose mode though, I might be seeing a completely different problem - but the description on google sounds too close a fit... I guess if / when we iron out these issues we then face the question of how best to do it. One of the most appealing things for this (against my current method of truecrypt containers on the array) is that it is flexible and we could leverage the user shares to split the data over the drives - thus avoiding the non expandable container issue of truecrypt and only using space you actually need to store the data. But is this a good idea? how would it handle this in terms of writing to the encrypted directory on the backend? We would possibly, split level dependant, end up in the situation where we have the encfs config and secrets file on one drive with the actual encrypted files scattered across others. In the worst case and we lost enough disks to take the encfs config out - is the surviving encrypted data useless? Can we back up the encfs secrets file and restore in the event of disaster would that be enough? If some of the encrypted data disks fail but the secrets file remains can we still validly mount it? Questions questions It would be a perfect solution and might,dare I say, be quite easy for limetech to integrate into the main control panel as an easy encryption solution (?)
December 4, 200916 yr Reproduced my problem with encfs in verbose / debug mode. Sadly I get identical results to the google code report : 23:22:39 (BlockFileIO.cpp:348) padding block 103915 23:22:39 (BlockFileIO.cpp:348) padding block 103916 23:22:39 (BlockFileIO.cpp:348) padding block 103917 23:22:39 (BlockFileIO.cpp:348) padding block 103918 23:22:39 (BlockFileIO.cpp:348) padding block 103919 23:22:39 (BlockFileIO.cpp:348) padding block 103920 23:22:39 (BlockFileIO.cpp:348) padding block 103921 23:22:39 (BlockFileIO.cpp:348) padding block 103922 23:22:39 (BlockFileIO.cpp:348) padding block 103923 23:22:39 (BlockFileIO.cpp:348) padding block 103924 23:22:39 (BlockFileIO.cpp:348) padding block 103925 23:22:39 (BlockFileIO.cpp:348) padding block 103926 23:22:39 (BlockFileIO.cpp:348) padding block 103927 23:22:39 (BlockFileIO.cpp:348) padding block 103928 Killed root@192:/mnt/user/Private# repeat x lots before and after I guess the next step would be : - for someone else to reproduce this on their setup...anyone? Instructions previously on this thread and in the google code report http://code.google.com/p/encfs/issues/detail?id=25. The author has tagged it as unreproducible so it *may* be a quirk in an individual setup. - To compile the latest SVN build and install. - Contact the author and tell him his unreproducible problem has been reproduced Verification from someone else here would be very useful and may convince me of my sanity!
December 6, 200916 yr Author Here are my results. I also posted more numbers here http://lime-technology.com/forum/index.php?topic=4821.0 I noticed some bad slow down depending of the hard drive controller. Alphazo --------------------------------- | MacBook / MAC OS | --------------------------------- | Disk mode | User share | -----------------------------------------------------| | To disk2 | 2:40 | 23MB/s | 2:31 | 25MB/s | |----------------------------------------------------| | To disk2 (encFS) | 6:25 | 9MB/s | 6:44 | 9MB/s | |----------------------------------------------------| | From disk2 | 1:50 | 34MB/s | 2:03 | 25MB/s | |----------------------------------------------------| | From disk2 (encFS) | 4:23 | 14MB/s | 4:15 | 14MB/s | |----------------------------------------------------| | To disk1 | 2:08 | 29MB/s | 2:15 | 30MB/s | |----------------------------------------------------| | To disk1 (encFS)* | 6:56 | 9MB/s | 7:53 | 8MB/s | |----------------------------------------------------| | From disk1 | 1:48 | 34MB/s | 1:48 | 34MB/s | |----------------------------------------------------| | From disk1 (encFS)*| 4:45 | 13MB/s | 5:19 | 12MB/s | |----------------------------------------------------| * - MacBook -> disk1 (encFS) : dropped to 4MB/s toward the end - disk1 (encFS) -> MacBook: dropped to 8MB/s toward the end [EDIT] For performance boost, please read the thread on new settings and cryptographic algorithm selection : http://lime-technology.com/forum/index.php?topic=4804.msg46566#msg46566
December 6, 200916 yr Probably closely matched to the level of encryption you used with encfs and your cpu. Doing the lightest level of encryption on my dual core celeron gave a very minimal hit on the throughput.
December 7, 200916 yr Boof, Do you have the same degradation speed using truecrypt? I don't mount the container on the unraid server - I mount it on the client machine via samba. So, usually, its a quad core chewing through the encryption rather than the celeron in the unraid server. It's not as noticeable but there is a degredation based on the container encryption level. Edited to add : I guess your point is how does encfs compare with truecrypt. I'd say that as encfs is userland and goes via fuse to talk to the kernel and fs it will always be slower than truecrypt (which uses it's own kernel module I believe). I'm not of the impression fuse is renowned for it's performance. Truecrypt also does clever i/o optimisations (pipelining etc) to further improve speed.
December 11, 200916 yr Verification from someone else here would be very useful and may convince me of my sanity! bump.. anyone else tried and seen or not seen the issue I'm able to reproduce?
December 11, 200916 yr I hope to get around to this soon, but it may not happen this weekend. Stay sane boof
December 11, 200916 yr I hope to get around to this soon, but it may not happen this weekend. Stay sane boof far too late for that
Archived
This topic is now archived and is closed to further replies.