DZMM Posted July 21, 2016 Share Posted July 21, 2016 Has anyone managed to mount Amazon Cloud Drive as an encrypted drive? I've found various interesting posts on the internet like this one https://ryanclouser.com/2015/12/01/Amazon-Cloud-Drive-EncFS-and-acd-cli/ and I've attempted to do this on windows a few times (e.g using Netdrive to mount amazon and a windows port of EncFS to mirror it), but the performance was terrible. There's an all-in-one solution in development https://stablebit.com/ , but their amazon implementation is throttled at the moment as they are not getting on well with Amazon. I've found a few dockers like this one https://hub.docker.com/r/joshhogle/ecloudfs/ - I just wanted to know if anyone has any experience in this area? It's one of the first things I want to try when I get my server built (just got 2 E5-2670s so far - other parts on the way/being selected) as for $60 for unlimited storage with Amazon I could do my backup jobs rather than using Crashplan, as well as having 'unlimited' file storage, use it for Plex Cloud Sync etc etc Quote Link to comment
EvilSpice Posted July 24, 2016 Share Posted July 24, 2016 also interested in this topic, following. Quote Link to comment
DZMM Posted July 24, 2016 Author Share Posted July 24, 2016 also interested in this topic, following. Welcome - it's been a bit lonely, but hopefully we'll get some advice soon! Quote Link to comment
FritzsCorner Posted August 7, 2016 Share Posted August 7, 2016 I am very interested in this as well. Quote Link to comment
2devnull Posted August 18, 2016 Share Posted August 18, 2016 Seems like Josh Hogle took his stuff down completely from everywhere (perhaps too many people were bothering him or AMZ caught up to him). Been trying to get Marcus https://github.com/msh100/acd-docker to update his for this. Quote Link to comment
RXWatcher Posted August 20, 2016 Share Posted August 20, 2016 Shortly rclone will support better encryption than encfs, mounting with support for gdrive, acd and many other cloud providers. it will be a one stop shop and make life a lot easier and more secure than the acd_cli+encfs. I can totally see using rclone in a docker. Quote Link to comment
BlackDwarf Posted August 27, 2016 Share Posted August 27, 2016 Keeping an eye on this now ACD has been launched in the UK. Would love to move away from Crashplan. Quote Link to comment
Pducharme Posted August 31, 2016 Share Posted August 31, 2016 Interested in this as well. I dropped Crashplan since it was not just working for me. Have about 25TB of data already, Crashplan stopped for me at 10TB, saying everything completed. I saw someone on Amazon Cloud with 96TB saying it's awesome AND fast ! having the rclone using encryption (seems to be implemented already?) is just even more better! Quote Link to comment
bobbintb Posted September 27, 2016 Share Posted September 27, 2016 I've been looking at this for a while, tried some options like the OP but nothing practical. I stated looking at it again with the recent announcement of Plex Cloud. I was thinking about trying the acd-cli+encfs route but it sounds like rclone might be better. Anything tangible yet? Edit: Answered my own question. I will be testing this. https://lime-technology.com/forum/index.php?topic=52033.0 Quote Link to comment
Wob76 Posted October 4, 2016 Share Posted October 4, 2016 Hi, I have been playing around with ACD options for the last couple of weeks, I am mostly looking at the mount options, as I want to be able to mount a drive and have a merged view of local and cloud storage. rclone for me was too slow, just browsing folders is not quick at all, acd_cli maintains a local db of the folder tree so browses much faster. rclone mount also can only do sequential reads (at least at the moment) and this was an issue for me. At the moment I have acd_cli and encfs working, I am still testing but so far the performance has been good, you can copy to\from the cloud mount, also most people recommend against it to ensure file integrity, so if I want to upload big chunks I unmount the acd folder and do the upload using acd_cli upload. I am still trying to work out how to do a merged mount of the local\cloud data. I will basically have the same data in the cloud and local, but I will then remove\archive some local content, so will reference the cloud if it doesn't exist locally. I found a guide for using UnionFS-Fuse, but since it is a similar process to the local user folder and how it references cache and local discs I am trying to work out if I can do it without installing unionfs-fuse. I made a post in another thread for anyone interested (http://lime-technology.com/forum/index.php?topic=45338.msg502980#msg502980) Cheers, Wob Quote Link to comment
2devnull Posted October 9, 2016 Share Posted October 9, 2016 well since Marcus hasn't yet got around to updating his stuff, I went ahead and got this all working following the amc.ovh tutorials. I pass the union mount to docker volume and therefore able to see the files in docker while having to perform the rest of the operations on the host. Not ideal but gets the job done. I wrote some of the commands as zsh functions to keep it simple. EDIT: Marcus did https://hub.docker.com/r/msh100/acd-docker/ start to do this update using a new single-container. However, I am not sure how this works as I need to pass something (the union mount I presume) to my other containers for them to write and delete items. Quote Link to comment
Wob76 Posted October 9, 2016 Share Posted October 9, 2016 Hi 2devnull, Can I ask how you went about installing unionfs? I couldn't find a nice slackware install for it and didn't want to go installing make just for that task. I ended up using the overlayfs built into the kernel, it works fine for reading, but it doesn't seem to like fuse mounts as RW so my merged mount is just Read Only. I scripted my install with a basic start up script, but it doesn't seem to like mapping the acd_cli during boot. I have already copied the oauth in place, but still get For the one-time authentication a browser (tab) will be opened at https://tensile-runway-92512.appspot.com/. Please accept the request and save the plaintext response data into a file called "oauth_data" in the directory "/.cache/acd_cli". Running the script after boot it all works fine, I have tried adding some delays, but no luck yet. I am running acd_cli with -fg (foreground) using screen, as I was seeing the mount being a little buggy, timing out etc, as this was a noted fix on the issues on github. Here is a copy of my startup script for anyone interested. #!/usr/bin/bash LOGFILE=/boot/acd_cli/logs/cloudstore-$(date "+%Y%m%d").log echo CloudSotre log $(date) $'\r'$'\r' >> $LOGFILE 2>&1 echo "Starting Cloud Mounts" $'\r'>> $LOGFILE 2>&1 #Copy oauth file to system mkdir -p /root/.cache/acd_cli/ cp /boot/acd_cli/config/oauth_data /root/.cache/acd_cli/oauth_data && #Install dependancies upgradepkg --install-new /boot/acd_cli/install/boost-1.59.0-x86_64-1.txz >> $LOGFILE 2>&1 upgradepkg --install-new /boot/acd_cli/install/rlog-1.4-x86_64-1pw.txz >> $LOGFILE 2>&1 upgradepkg --install-new /boot/acd_cli/install/slocate-3.1-x86_64-4.txz >> $LOGFILE 2>&1 #Install encfs upgradepkg --install-new /boot/acd_cli/install/encfs-1.8.1-x86_64-1gv.txz >> $LOGFILE 2>&1 #Install acd_cli pip3 install --upgrade git+https://github.com/yadayada/acd_cli.git >> $LOGFILE 2>&1 #Sleep for 10s and then run a acd_cli sync sleep 10s && acdcli sync >> $LOGFILE 2>&1 #Mount Amazon Cloud Drive (using screen) echo Mounting Amazon Cloud Drive >> $LOGFILE 2>&1 screen -S acdcli -d -m /usr/bin/acd_cli -nl mount -fg -ao --uid 99 --gid 100 --modules="subdir,subdir=/Plex" /mnt/cache/cloud/.acd >> $LOGFILE 2>&1 #Mount Decrypted view of ACD echo Mounting ENCFS points >> $LOGFILE 2>&1 echo <password> | ENCFS6_CONFIG='/boot/acd_cli/config/encfs.xml' encfs -S -o ro -o allow_other -o uid=99 -o gid=100 /mnt/cache/cloud/.acd/ /mnt/cache/cloud/acd/ >> $LOGFILE 2>&1 #Mount Encrypted view of Local Media (Use for uploading Data to ACD) echo <password>| ENCFS6_CONFIG='/boot/acd_cli/config/encfs.xml' encfs -S --reverse -o ro -o allow_other -o uid=99 -o gid=100 /mnt/user/Media/ /mnt/cache/cloud/.local/ >> $LOGFILE 2>&1 #Overlay Mount with Local Data taking preference. (Read Only) echo Mounting Overlay point >> $LOGFILE 2>&1 mount -t overlay -o lowerdir=/mnt/user/Media/:/mnt/user/cloud/acd/ overlay /mnt/user/cloud/media/ >> $LOGFILE 2>&1 #Restart the plex docker (so it can see data in the mount point) docker restart plex Cheers, Wob Quote Link to comment
MrCrispy Posted October 9, 2016 Share Posted October 9, 2016 I was interested in this till a few days ago. Signed up for ACD as well. Then came the Comcast data caps >, so this isn't a feasible option anymore. Quote Link to comment
2devnull Posted October 10, 2016 Share Posted October 10, 2016 @Wob76 - I use RancherOS as the base to run my dockers which provides a selection of console (I'm using the Ubuntu one). Installing the unionfs therefore is just the same as installing on Ubuntu (install the unionfs-fuse package) Quote Link to comment
Wob76 Posted October 10, 2016 Share Posted October 10, 2016 Thanks for the response, are you running RacherOS on another box to your unRAID, or in a VM? Regards, Wob @Wob76 - I use RancherOS as the base to run my dockers which provides a selection of console (I'm using the Ubuntu one). Installing the unionfs therefore is just the same as installing on Ubuntu (install the unionfs-fuse package) Quote Link to comment
2devnull Posted October 10, 2016 Share Posted October 10, 2016 Thanks for the response, are you running RacherOS on another box to your unRAID, or in a VM? Another. BTW - I think I may venture into using this method with the 'reverse' option and rclone as it appears it does not require encrypting until sending to ACD. https://alpha-labs.net/2016/09/amazon-clouddrive-encfs-rsync/ Quote Link to comment
Wob76 Posted October 10, 2016 Share Posted October 10, 2016 I am using the --reverse option (the second encfs mount in my script) that mount is purely there for uploading to ACD, and yeah, then you are only hit with the encryption overhead during the upload process. I end up with the following mount points from my script encfs on /mnt/cache/cloud/acd type fuse.encfs (rw,nosuid,nodev,allow_other,default_permissions) - The decrypted view of ACD encfs on /mnt/cache/cloud/.local type fuse.encfs (ro,nosuid,nodev,allow_other,default_permissions) - The encrypted view of my local media folder *the reverse mount* overlay on /mnt/user/cloud/media type overlay (ro,lowerdir=/mnt/user/Media/:/mnt/user/cloud/acd/) - The overlay mount with local media being the top layer and acd below that. ACDFuse on /mnt/cache/cloud/.acd type fuse.ACDFuse (rw,nosuid,nodev,allow_other) - The encrypted ACD mount point I solved the issue with my script not working at start up. I enabled verbose on the acd_cli sync command and it was looking in /.cache/acd_cli for the oauth, so no home variable was being parsed. I could move the file to that location, but it I want to be able to call acd_cli from the command line for uploading etc. So I just added the following to the start of my script. HOME=/root If anyone has been able to install unionfs-fused on unRAID I would love to hear about it. Wob Quote Link to comment
2devnull Posted October 10, 2016 Share Posted October 10, 2016 Glad you got to the bottom of that problem. I presume the unionfs-fuse mount will only need to combine the media directory (RW) with the decrypted view of the acd directory (RO). Quote Link to comment
Wob76 Posted October 10, 2016 Share Posted October 10, 2016 Glad you got to the bottom of that problem. I presume the unionfs-fuse mount will only need to combine the media directory (RW) with the decrypted view of the acd directory (RO). Yep, that would do the trick. The solution I have is fine for the most part, but if I want to track the collections (including ACD only files) with the likes of CP and SR then I need them to point to a mount that is RW. At the moment they will only see local content, as they will point just to local data as they need RW access for renaming. It is not a big issue for SR as it will just tag files that are moved to ACD as archived, but CP would no longer see track that are moved to ACD. At the moment it is mostly theoretical, I only have a small amount of data in ACD for testing, it appears to stream fine. I am also hesitant to have ACD the only copy of something, so if I do want to remove something local I will be looking at a backup of some kind before I remove my local copy. At the moment, local space is not an issue. My plan is to maintain local data for a least a few years, then have older stuff pulled from ACD on the rare occasion that someone views it. Quote Link to comment
publicENEMY Posted October 19, 2016 Share Posted October 19, 2016 Glad you got to the bottom of that problem. I presume the unionfs-fuse mount will only need to combine the media directory (RW) with the decrypted view of the acd directory (RO). Yep, that would do the trick. The solution I have is fine for the most part, but if I want to track the collections (including ACD only files) with the likes of CP and SR then I need them to point to a mount that is RW. At the moment they will only see local content, as they will point just to local data as they need RW access for renaming. It is not a big issue for SR as it will just tag files that are moved to ACD as archived, but CP would no longer see track that are moved to ACD. At the moment it is mostly theoretical, I only have a small amount of data in ACD for testing, it appears to stream fine. I am also hesitant to have ACD the only copy of something, so if I do want to remove something local I will be looking at a backup of some kind before I remove my local copy. At the moment, local space is not an issue. My plan is to maintain local data for a least a few years, then have older stuff pulled from ACD on the rare occasion that someone views it. Are there updates to your scripts? I would like to test this but if there are updated scripts, I would like to test. Thanks. Quote Link to comment
publicENEMY Posted October 21, 2016 Share Posted October 21, 2016 I have created a github repo based on the script. https://github.com/syaifulnizamyahya/acd_unraid/blob/master/unraidacd.sh By the way, I have successfully run the script without errors and what next? Which folder do I put my files to be excrypted? how do I upload excrypted files to amazon? THanks. Quote Link to comment
nick5429 Posted October 21, 2016 Share Posted October 21, 2016 If anyone has been able to install unionfs-fused on unRAID I would love to hear about it. Here's a unionfs package for slackware 14.2 -- https://pkgs.org/slackware-14.2/slackel-x86_64/unionfs-fuse-0.26-x86_64-1dj.txz.html Quote Link to comment
squirrellydw Posted October 23, 2016 Share Posted October 23, 2016 I found this if it helps anyone. Hoping for some kind of docker to make everything easier if possible. https://enztv.wordpress.com/2016/10/19/using-amazon-cloud-drive-with-plex-media-server-and-encrypting-it/ Quote Link to comment
publicENEMY Posted October 26, 2016 Share Posted October 26, 2016 I manually run this command in screen. acd_cli -nl mount -fg -ao --uid 99 --gid 100 --modules="subdir,subdir=/encfs" /mnt/user/Amazon/.acd Nothing happens. What is the expected response? Is it usual to take a long time? How long does it usually take? Edit: Okay, I checked the directory. The command works. Quote Link to comment
bobbintb Posted October 26, 2016 Share Posted October 26, 2016 I found this if it helps anyone. Hoping for some kind of docker to make everything easier if possible. https://enztv.wordpress.com/2016/10/19/using-amazon-cloud-drive-with-plex-media-server-and-encrypting-it/ That looks interesting but I wonder what the performance is like, especially considering how many programs it uses. It seems unnecessarily complex. I've seem tutorials use just rclone and others use a combination on acd_cli and encfs. What benefit is there from combining them all and throwing unionfs on top of it? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.