DZMM

Thanks - I'll have a play over the weekend

I'm struggling to block some undesired users because of PlexRelay kicking in if I block the IP address. Is there a way to disable Plex relay in the docker? These posts have some methods, but I'm tying to find a more elegant/permanent solution using this docker. Thanks

Just discovered a major benefit! With unionfs if you wanted to move files that existed in the rclone mount, it would first download them to the relevant folder in rclone_upload, then upload them and finally then delete the original file in the mount - a lot of transfer. The only way to do quickly was to work directly on the rclone mount. mergerfs just moves the file within the mount without downloading and re-uploading! This should solve some of the transfer problems people have been having, as the move acts like a normal move within a drive without the lockups that you'd get sometimes with unionfs. Much better handling of the merged folders 🙂

not over-complicating, just showing how dumb I can be! I got it working and playing with now - thanks!

Can a few people help me test mergerfs please. Benefits: 1. no need for unionfs/mount cleanup script as mergerfs doesn't create temp files and new writes are always to the first folder - much cleaner 2. supports hardlinking for torrents. To test, replace in the mount script: unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/tdrive_vfs=RW:/mnt/user/mount_rclone/tdrive_vfs=RO /mnt/user/mount_unionfs/google_vfs with: # removing old binary as a precaution rm /bin/mergerfs docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build mv /mnt/user/appdata/other/mergerfs/mergerfs /bin mergerfs /mnt/user/rclone_upload/tdrive_vfs:/mnt/user/mount_rclone/tdrive_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true It's working for me, but will play with for a few days

supports hardlinking and no need for my unionfs cleanup script.

Looks like mergerfs support is available!

Thanks for getting this created - you'll make a lot of users happy when they see this. I'm not sure how to use though. Once I've created the docker, can I create mergerfs mounts/drives? I don't understand the bit about moving to /bin?

Glad you got it working. I find --delete-empty-src-dirs a bit confusing as well, so I don't use it personally and I do this instead without delete-empty-src-dirs: rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_fast1_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 4 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 9M --tpslimit 8 --min-age 10m --fast-list and then I remove any empty folders myself and then recreate the main media folders when the upload completes, to make it easier for to add any manual uploads: find /mnt/user/rclone_upload/tdrive_vfs -empty -type d -delete mkdir -p /mnt/user/rclone_upload/tdrive_vfs/{documentaries/kids,documentaries/adults,movies_adults_gd,movies_kids_gd,movies_uhd_gd/adults,movies_uhd_gd/kids,tv_adults_gd,tv_kids_gd,tv_uhd/adults,tv_uhd/kids,unfiled/documentaries,unfiled/movies,unfiled/movies_uhd}

Post your upload script. Remember there's sometimes a few mins lag between files being uploaded i.e. deleted from the local drive and showing up in the mount via the unionfs folder

you must have deleted the mountcheck file. To upload content, add media to the mount_unionfs folders (this is where you dockers should be mapped to) or rclone_upload directly, but NEVER unless you know what you are doing directly to mount_rclone.

did you follow step 2 in the first post when you did the initial setup and created the mountcheck file?

Post your rclone config and your mount script please. Also is your rclone mount folder empty before mounting?

I just overwrite the existing xml (if necessary) and vdisk files

yes, you can create more than one mount. Just create more rclone remotes

hmm it didn't for me, but maybe I was too quick - dunno. Thanks again for making this into a plugin.

That's why I did the hard work with the scripts (and I can't write scripts - all my scripts are cobbled together from others, google, trial and error!) and annotated them and wrote a guide so anyone who can install unRAID can copy them and edit them quickly, or at least know how to do safely until they've done it correctly. Plus this thread is added help for those who can't manage that! Nobody has to write a shell script to get this setup - just install a few plugins, copy and paste, and have a text editor handy if they want to go off-piste. I think if you don't know how to startup a shell session as a first time unraid user, then it's something you'll have to learn within a few weeks as I can't see how you could avoid it for long.

Will this eventually get added to CA so it will update automatically?

I don't think anything about unraid is for beginners, so I disagree and say any unRAID user should be able to complete these guides and follow space invader's video's (particularly as they can see and copy what he does), @kagzz is right and maybe a video guide would be helpful just to see the steps in action - I'm not in the mood to do a youtube video, but I might record my screen one day over xmas if I get bored doing a setup from a scratch.

thanks @testdasi

I'm about to try going from rc7 to 6.8 - rc8 failed for me, but I didn't know why until now. I currently have: <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='6' threads='1'/> <feature policy='require' name='topoext'/> </cpu> will this work for my threadripper and maintain AES-NI support? Thanks <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Opteron_G5</model> <vendor>AMD</vendor> <topology sockets='1' cores='6' threads='1'/> <feature policy='require' name='vme'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='arat'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='bmi1'/> <feature policy='require' name='mmxext'/> <feature policy='require' name='fxsr_opt'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='cr8legacy'/> <feature policy='require' name='osvw'/> <feature policy='disable' name='rdtscp'/> <feature policy='disable' name='svm'/> </cpu> This is all way beyond my VM level so it's a blind cut & paste job for me.

No. I would just stick with the teamdrive as it gives you more options going forwards. Probably the symlink - I don't know how unionfs handles these. I'd just move the files to rclone_upload or just add the current folder location to the unionfs mount - you can include more folders in the union. I'm not putting it on github as it goes beyond basic usage, but here it is. I put a bwlimit of 70M as I don't want to max out my 1G line. There's probably a clever way to shorten the script and automatically insert the counter, but I'm not a coder so I can't do that: #!/bin/bash ####### Check if script already running ########## if [[ -f "/mnt/user/appdata/other/rclone/rclone_upload" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting as script already running." exit else touch /mnt/user/appdata/other/rclone/rclone_upload fi ####### End Check if script already running ########## ####### check if rclone installed ########## if [[ -f "/mnt/user/mount_rclone/tdrive_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: rclone installed successfully - proceeding with upload." else echo "$(date "+%d.%m.%Y %T") INFO: rclone not installed - will try again later." rm /mnt/user/appdata/other/rclone/rclone_upload exit fi ####### end check if rclone installed ########## ##Run 1 if [[ -f "/mnt/user/appdata/other/rclone/counter_one" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_one" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user1_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_one echo "$(date "+%d.%m.%Y %T") creating counter_two" touch /mnt/user/appdata/other/rclone/counter_two rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 1" fi ##Run 2 if [[ -f "/mnt/user/appdata/other/rclone/counter_two" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_two" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user2_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_two echo "$(date "+%d.%m.%Y %T") creating counter_three" touch /mnt/user/appdata/other/rclone/counter_three rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 2" fi ##Run 3 if [[ -f "/mnt/user/appdata/other/rclone/counter_three" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_three" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user3_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_three echo "$(date "+%d.%m.%Y %T") creating counter_four" touch /mnt/user/appdata/other/rclone/counter_four rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 3" fi ##Run 4 if [[ -f "/mnt/user/appdata/other/rclone/counter_four" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_four" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user4_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_four echo "$(date "+%d.%m.%Y %T") creating counter_five" touch /mnt/user/appdata/other/rclone/counter_five rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 4" fi ##Run 5 if [[ -f "/mnt/user/appdata/other/rclone/counter_five" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_five" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user5_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_five echo "$(date "+%d.%m.%Y %T") creating counter_six" touch /mnt/user/appdata/other/rclone/counter_six rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 5" fi ##Run 6 if [[ -f "/mnt/user/appdata/other/rclone/counter_six" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_six" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user6_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_six echo "$(date "+%d.%m.%Y %T") creating counter_seven" touch /mnt/user/appdata/other/rclone/counter_seven rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 6" fi ##Run 7 if [[ -f "/mnt/user/appdata/other/rclone/counter_seven" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_seven" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user7_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_seven echo "$(date "+%d.%m.%Y %T") creating counter_eight" touch /mnt/user/appdata/other/rclone/counter_eight rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 7" fi ##Run 8 if [[ -f "/mnt/user/appdata/other/rclone/counter_eight" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_eight" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user8_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_eight echo "$(date "+%d.%m.%Y %T") creating counter_nine" touch /mnt/user/appdata/other/rclone/counter_nine rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 8" fi ##Run 9 if [[ -f "/mnt/user/appdata/other/rclone/counter_nine" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_nine" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user9_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_nine echo "$(date "+%d.%m.%Y %T") creating counter_ten" touch /mnt/user/appdata/other/rclone/counter_ten rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 9" fi ##Run 10 if [[ -f "/mnt/user/appdata/other/rclone/counter_ten" ]]; then echo "$(date "+%d.%m.%Y %T") found counter_ten" rclone move /mnt/user/rclone_upload/tdrive_vfs tdrive_user10_vfs: --user-agent="unRAID" -vv --buffer-size 512M --drive-chunk-size 512M --checkers 8 --transfers 6 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --bwlimit 70M --tpslimit 8 --min-age 10m --user-list rm /mnt/user/appdata/other/rclone/counter_ten echo "$(date "+%d.%m.%Y %T") creating counter_one" touch /mnt/user/appdata/other/rclone/counter_one rm /mnt/user/appdata/other/rclone/rclone_upload exit else echo "$(date "+%d.%m.%Y %T") skipping run 10" fi else # added in case counters missing and script makes it this far echo "$(date "+%d.%m.%Y %T") creating counter_one as missed counters" touch /mnt/user/appdata/other/rclone/counter_one fi rm /mnt/user/appdata/other/rclone/rclone_upload exit

You've got it all right. 1. Either - content added to mount_unionfs goes into rclone_upload to be transferred to gdrive. Best practice is probably to do m_u, in case you're overwriting an existing file and to make sure apps stop/don't work if the mount goes down 2. TeamDrives are useful if (I) you have good upload. Ive added multiple users to mine as each user gets 750GB/day, and my upload script rotates the account in use, so I can remove the bwlimit as well as increase daily upload limit to 750GB x # of users (II) if you want to share a teamdrive e.g. for backup purposes or a different way to share other than via Plex 3. Yes all docker/app paths should use m_u paths

Make sure all traces of the folder in mount_unionfs and probably rclone_upload are deleted before you mount