DZMM

3/1/20 UPDATE: TO MIGRATE FROM UNIONFS TO MERGERFS READ THIS POST. New users continue reading 13/3/20 Update: For a clean version of the 'How To' please use the github site https://github.com/BinsonBuzz/unraid_rclone_mount 17/11/21 Update: Poll to see how much people are storing I've added a Paypal.me upon request if anyone wants to buy me a beer. There’s been a number of scattered discussions around the forum on how to use rclone to mount cloud media and play it locally via Plex, Emby etc. After discussions with @Kaizac @slimshizn and a few others, we thought it’d be useful to start a thread where we can all share and improve our setups. Why do this? Well, if set-up correctly Plex can play cloud files regardless of size e.g. I play 4K media with no issues, with start times of under 5 seconds i.e. comparable to spinning up a local disk. With access to unlimited cloud space available for the cost of a domain name and around $510/pm, then this becomes a very interesting proposition as it reduces local storage requirements, noise etc etc. At the moment I have about 80% of my library in the cloud and I struggle to tell if a file is local or in the cloud when playback starts. To kick the thread off, I’ll share my current setup using gdrive. I’ll try and keep this initial thread updated. Update: I've moved my scripts to github to make it easier to keep them updated https://github.com/BinsonBuzz/unraid_rclone_mount Changelog 6/11/18 – Initial setup (updated to include rclone rc refresh) 7/11/18 - updated mount script to fix rc issues 10/11/18 - added creation of extra user directories ( /mnt/user/appdata/other/rclone & /mnt/user/rclone_upload/google_vfs) to mount script. Also fixed typo for filepath 11/11/18 - latest scripts added to https://github.com/BinsonBuzz/unraid_rclone_mount for easier editing 3/1/20 - switched from unionfs to mergerfs 4/2/20 - updated the scripts to make easier to use and control. Thanks to @senpaibox for the inspiration My Setup Plugins needed Rclone – installs rclone and allows the creation of remotes and mounts. New scripts require V1.5.1+ User Scripts – controls how mounts get created How It Works Rclone is used to access files on your google drive and to mount them in a folder on your server e.g. mount a gdrive remote called gdrive_vfs: at /mnt/user/mount_rlone/gdrive_vfs Mergerfs is used to merge files from your rclone mount (/mnt/user/mount_rlone/gdrive_vfs) with local files that exist on your server and haven't been uploaded yet (e.g. /mnt/user/local/gdrive_vfs) in a new mount /mnt/user/mount_unionfs/gdrive_vfs This mergerfs mount allows files to be played by dockers such as Plex, or added to by dockers like radarr etc without the dockers even being aware that some files are local and some are remote. It just doesn't matter The use of a rclone vfs remote allows fast playback, with files streaming within seconds New files added to the mergerfs share are actually written to the local share, where they will stay until the upload script processes them An upload script is used to upload files in the background from the local folder to the remote. This activity is masked by mergerfs i.e. to plex, radarr etc files haven't 'moved' Getting Started Install the rclone plugin and via command line run rclone config and create 2 remotes: gdrive: - a drive remote that connects to your gdrive account. Recommend creating your own client_id gdrive_media_vfs: - a crypt remote that is mounted locally and decrypts the encrypted files uploaded to gdrive: It is advisable to create your own client_id to avoid API bans. Mount Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and paste in the rclone_mount script Edit the config lines at the start of the script to choose your remote name, paths etc Choose a suitable cron job. I run this script on a 10 min */10 * * * * schedule so that it automatically remounts if there’s a problem. The script: Checks if an instance is already running, remounts (if cron job set) automatically if mount drops Mounts your rclone gdrive remote Installs mergerfs and creates a mergerfs mount Starts dockers that need the mergerfs mount e.g. plex, radarr Upload Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and paste in the rclone_mount script Edit the config lines at the start of the script to choose your remote name, paths etc - USE THE SAME PATHS Choose a suitable cron job e.g hourly Features: Checks if rclone is installed correctly sets bwlimits There is a cap on uploads by google of 750GB/day. I have added bandwidth scheduling to the script so you can e.g. set an overnight job to upload the daily quota at 30MB/s, have it trickle up over the day at a constant 10MB/s, or set variable speeds over the day The script now stops once the 750GB/day limit is hit (rclone 1.5.1+ required) so there is more flexibility over upload strategies I've also added --min age 10mins to stop any premature uploads and exclusions to stop partial files etc getting uploaded. Cleanup Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and set to run at array start (recommended) or array stop In the next post I'll explain my rclone mount command in a bit more detail, to hopefully get the discussion going!

@smdion the link is broken - can you share the script again please.

I'm glad I found this thread as I have two drives throwing up this error. Is there a fix yet?

Help please! I've read https://github.com/Novik/ruTorrent/wiki/Config#authentication but I still don't see how I add a username and password to this docker. I've just moved from Deluge (slow downloads and keeps freezing) and qbittorrent (won't upload) Thanks

How far away are the rooms located? HDMI over ethernet is a potential solution, as well as long HDMI/USB cables

Pity - thanks for checking

docker stop transmission docker start transmission

Is mergerfs possible please?

Anyone ever installed mergerfs on unRAID? I need it to merge a local folder with a rclone mount.

It's ok - I've managed to get around using mergerfs this evening - sorry for any hassle caused with any research.

@dmacias is it possible to add mergerfs?

Hi I'm trying this docker for the first time and I'm getting an error on first run: Help please root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='binhex-krusader' --net='br0' --ip='172.30.12.3' --privileged=true -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'TCP_PORT_6080'='6080' -e 'TEMP_FOLDER'='/config/krusader/tmp' -e 'WEBPAGE_TITLE'='Highlander' -e 'VNC_PASSWORD'='Removed' -e 'UMASK'='000' -e 'PUID'='99' -e 'PGID'='100' -v '/mnt/user':'/media':'rw' -v '/mnt/user/':'/user':'rw' -v '/mnt/disks/':'/disks':'rw' -v '/boot/':'/flash':'rw' -v '/mnt/cache/appdata/dockers/binhex-krusader':'/config':'rw' 'binhex/arch-krusader'

Quick poll - what do you guys have for your Bandwidth settings?I haven't touched the default half-open connections or the max connections attempts per second on my 200/200 line, and I wonder if I should. I use two private trackers IPT and Torrentday

I don't know if you do requests, but is it possible to add mergerfs please as an addition to unionfs?

Bump. This would be major result for me - I just need a bit of help please

Ahh - thanks

Hi I'm getting tonnes of wrong csrf_token errors in my logs - fixable or not worth worrying about? Thanks Oct 10 23:11:15 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:16 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:17 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:18 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:19 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:21 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:22 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:23 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:24 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:25 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:26 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:27 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:28 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:29 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:30 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:31 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:32 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:33 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:34 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:35 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:36 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:37 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:38 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:39 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:41 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token highlander-diagnostics-20181011-1559.zip

I'm posting here as I wasn't sure which board this fits in. I currently use unionfs to combine local files /rclone_upload/google_vfs with gdrive files I mount using rclone /mount_rclone/google_vfs : unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs I'd like to switch to MergerFS as it supports hardlinking, whereas Unionfs doesn't. I've installed unionfs via Nerd Pack - can someone give me pointers on how to install Mergerfs please?

In the docker settings change the name e.g plex-2, the appdata path e.g. /mnt/user/appdata/plex-2 and ports or ip address

It was uploading files in the background so it was 'working'. However, stopped it as I need to be able to add several sub-folders. I worked out how to get the client working on my VM (changed mac) so using this method.

I just tried again and I managed to login which is one step further than I get on my VM, but then I just got a blank black screen in the VNC Viewer

oh sorry, you want to be able to view the S3 files. Found this on google - https://aws.amazon.com/es/customerapps/s3fm-a-free-online-file-manager-for-amazon-s3/?nc1=f_ls

rclone mount <remote_name>:<s3_bucket_name> <mount_path> for <mount_path> use something like /mnt/user/picture_archive. There are lots of other options for mount, but that'll do the job https://rclone.org/commands/rclone_mount/

I loved those books as a kid - the Belgariad was one of the first fantasy series I read.

I just tried installing but got an error oot@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='Google-MusicManager' --net='host' --cpuset-cpus='5,6,7,19,20,21' -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'PORT'='5920' -e 'PASSWORD'='Removed' -v '/mnt/user/media/other_media/music/':'/music':'ro' -v '/mnt/cache/appdata/dockers/Google-MusicManager':'/config':'rw' false 'rix1337/docker-google-musicmanager' false Unable to find image 'false:latest' locally /usr/bin/docker: Error response from daemon: pull access denied for false, repository does not exist or may require 'docker login'. See '/usr/bin/docker run --help'. The command failed.