DZMM

Can't help you re the upload script as you've gone way off-piste with your arguments - try the rclone forum

I'm not sure how to create mergerfs mounts with more than 2 drives, but I think this is what you want: mergerfs /mnt/user/Archiv:/mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true where the first folder /mnt/user/Archiv will be the only RW folder. Test it by creating a mergerfs mount and turn off radarr, sonarr etc anything that might write to the mount and create a few text files and see what happens

Thanks - I'm researching how to browse a docker to find the correct location

I've read, but I'd rather use the user_scripts plugin so I can remember I did this. Would this modify the docker if I run hourly? #!/bin/bash docker exec -it plex /bin/bash rm -f PlexRelay exit exit

I also think mergerfs is a bit faster than unionfs - not sure yet

re -it Not sure. It worked for me when I was doing via command line and in scripts, but I haven't rebooted since. Let's see how others get on.

lol the vfs-cache-mode is the main point of this guide to get playback startup times similar to spinning up a HDD!

I don't think there's a limit - I've got 30+ remotes setup I think. Mounting may be different - I have 4 rclone mounts

If done carefully no impact. It will probably solve your transfer problems and other file handling issues unionfs created. It's one command - it's just long. Check out github where it's clearer. If you want to switch just the unionfs mount quickly then swap: unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs for: # Build mergerfs binary and delete old binary as precaution rm /bin/mergerfs # Create Docker docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build # move to bin to use for commands mv /mnt/user/appdata/other/mergerfs/mergerfs /bin # Create mergerfs mount mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true But, I'd also change the way your uploads work (new script, plus changing your download client mappings) so that you can benefit from the better file handling between your download clients and the new mergerfs mount.

read my latest post - if you're careful, there will be no impact for a lot of gain

I have updated my scripts on https://github.com/BinsonBuzz/unraid_rclone_mount and the first post in this thread (partially) to include mergerfs support. I will keep github updated, so if you want the latest versions then watch that thread as well. I've also improved the annotation so if you are already setup with unionfs it should be easy to change: How to Migrate from unionfs: In the new mount and unmount scripts find and replace mount_mergerfs with mount_unionfs and all should still work. You need to keep your unionfs mount location, otherwise Plex, radarr, sonarr etc will need to rescan (very painful if lots of files). New users can start off with mount_mergerfs Change your mappings for nzbget, transmission etc to point to /mnt/user/mount_unionfs/downloads - IMPORTANT if you want the best benefits from mergerfs - see below Either finish any existing uploads from rclone_upload before updating, or move pending downloads from /mnt/user/rclone_upload to the new /mnt/user/local folder, or create a version of the new upload script to upload from /mnt/user/local Benefits over mergerfs: No need for cleanup script: mergerfs moves, copies,renames, deletes etc files that are on the mount actually on the mount, not how unionfs (i) would download and then re-upload a file or (ii) mask that the file hadn't actually been renamed or deleted on the mount = MUCH BETTER FILE HANDLING AND LESS TRANSFER, PLUS ONE LESS SCRIPT Hardlink support: Unionfs didn't support hardlinks so any torrents had to be copied to rclone_upload and then uploaded. Mergerfs supports hardlinks so no wasted transfer. I've added an upload exclusion to /mnt/user/local/downloads so that download files (intermediate, pending sonarr/raddarr import, or seeds) are not uploaded. For hardlinks to work, transmission/torrent clients HAVE to be mapped to the same disk 'storage' so files need to be in /mnt/user/mount_unionfs or for newer users /mnt/user/mount_mergerfs...hope this makes sense = MUCH BETTER FILE HANDLING AND LESS TRANSFER AND FILES NOT STORED TWICE Other Transfer benefits: moving files within mergerfs is better supported and appear seamless e.g. scenarios before where a new nzbget file had to be copied by unionfs to the upload folder no longer occurs - the file is now moved instantly i.e. no disk write; moving files between folders even if non-cloud to cloud and vice versa, are much much faster = MUCH BETTER FILE HANDLING AND LESS TRANSFER I'd advise everyone to do the migration as your drives will spin a lot less and the end product is a lot smoother. UPDATE: if you just want a quick migration without all the benefits, then swap: unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs for: # Build mergerfs binary and delete old binary as precaution rm /bin/mergerfs # Create Docker docker run -v /mnt/user/appdata/other/mergerfs:/build --rm trapexit/mergerfs-static-build # move to bin to use for commands mkdir -p /mnt/user/appdata/other/mergerfs/ mv /mnt/user/appdata/other/mergerfs/mergerfs /bin # Create mergerfs mount mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true But, I'd advise updating your download docker mappings to get all the file handling benefits.

Thanks - I'll have a play over the weekend

I'm struggling to block some undesired users because of PlexRelay kicking in if I block the IP address. Is there a way to disable Plex relay in the docker? These posts have some methods, but I'm tying to find a more elegant/permanent solution using this docker. Thanks

Just discovered a major benefit! With unionfs if you wanted to move files that existed in the rclone mount, it would first download them to the relevant folder in rclone_upload, then upload them and finally then delete the original file in the mount - a lot of transfer. The only way to do quickly was to work directly on the rclone mount. mergerfs just moves the file within the mount without downloading and re-uploading! This should solve some of the transfer problems people have been having, as the move acts like a normal move within a drive without the lockups that you'd get sometimes with unionfs. Much better handling of the merged folders 🙂

not over-complicating, just showing how dumb I can be! I got it working and playing with now - thanks!

Can a few people help me test mergerfs please. Benefits: 1. no need for unionfs/mount cleanup script as mergerfs doesn't create temp files and new writes are always to the first folder - much cleaner 2. supports hardlinking for torrents. To test, replace in the mount script: unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/tdrive_vfs=RW:/mnt/user/mount_rclone/tdrive_vfs=RO /mnt/user/mount_unionfs/google_vfs with: # removing old binary as a precaution rm /bin/mergerfs docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build mv /mnt/user/appdata/other/mergerfs/mergerfs /bin mergerfs /mnt/user/rclone_upload/tdrive_vfs:/mnt/user/mount_rclone/tdrive_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true It's working for me, but will play with for a few days

supports hardlinking and no need for my unionfs cleanup script.

Looks like mergerfs support is available!

Thanks for getting this created - you'll make a lot of users happy when they see this. I'm not sure how to use though. Once I've created the docker, can I create mergerfs mounts/drives? I don't understand the bit about moving to /bin?

Glad you got it working. I find --delete-empty-src-dirs a bit confusing as well, so I don't use it personally and I do this instead without delete-empty-src-dirs: rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_fast1_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 4 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 9M --tpslimit 8 --min-age 10m --fast-list and then I remove any empty folders myself and then recreate the main media folders when the upload completes, to make it easier for to add any manual uploads: find /mnt/user/rclone_upload/tdrive_vfs -empty -type d -delete mkdir -p /mnt/user/rclone_upload/tdrive_vfs/{documentaries/kids,documentaries/adults,movies_adults_gd,movies_kids_gd,movies_uhd_gd/adults,movies_uhd_gd/kids,tv_adults_gd,tv_kids_gd,tv_uhd/adults,tv_uhd/kids,unfiled/documentaries,unfiled/movies,unfiled/movies_uhd}

Post your upload script. Remember there's sometimes a few mins lag between files being uploaded i.e. deleted from the local drive and showing up in the mount via the unionfs folder

you must have deleted the mountcheck file. To upload content, add media to the mount_unionfs folders (this is where you dockers should be mapped to) or rclone_upload directly, but NEVER unless you know what you are doing directly to mount_rclone.

did you follow step 2 in the first post when you did the initial setup and created the mountcheck file?

Post your rclone config and your mount script please. Also is your rclone mount folder empty before mounting?