Guide: How To Use Rclone To Mount Cloud Drives And Play Files

[yendi]

Will try this probably this weekend!

Thanks @DZMM for sharing this with the community !

[L0rdRaiden]

Proabably a noob question but I have a little problem I can't resolve.

I have followed this video

And everything is fine until I try to see the files with Krusader, I can see the contents with "mc" in command line but not with Kusader

Why is this? how can I fix it? in Krusader /mnt/ is mounted as /Media.

 

 

 

EDIT: IN CASE SOMEONE ELSE HAS THIS PROBLEM HERE IS THE SOLUTION

 

Edited January 2, 2020 by L0rdRaiden

[NewDisplayName]

On 1/2/2020 at 12:41 AM, DZMM said:

Can a few people help me test mergerfs please.  Benefits:

 

1. no need for unionfs/mount cleanup script as mergerfs doesn't create temp files and new writes are always to the first folder - much cleaner

2. supports hardlinking for torrents.

 

To test, replace in the mount script:

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/tdrive_vfs=RW:/mnt/user/mount_rclone/tdrive_vfs=RO /mnt/user/mount_unionfs/google_vfs

with:

# removing old binary as a precaution
rm /bin/mergerfs

docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

mergerfs /mnt/user/rclone_upload/tdrive_vfs:/mnt/user/mount_rclone/tdrive_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

It's working for me, but will play with for a few days

didnt noticed there was a follow up...

 

ill try it the next days, like always, THANK YOU.

 

Question is, what happens with the old files?

[Spladge]

Will switch over soon - too much going on here at the moment. 
Another benefit, much like the server side copying between remotes is the renaming happens server side.

 

Is there a character limit on the rclone.cofig file size with the plugin, it doesn't seem to let me go past four remotes. Have yet to look into this so apologies if it is in the readme already.

[NewDisplayName]

What i forget to ask... what happens to old files? Like i had the problem it wont transfer anything because "size differ", will that fix it?

 

Iam a bit confused bc it looks different then my unionfs entry...

 

is that correct? (i dont know about the file paths)

 

# removing old binary as a precaution

rm /bin/mergerfs

docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

mergerfs /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

 

 

Thats how it looked before:
 

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

 

 

Another question... is "-o rw,..." one command or 2 commands?

Edited January 3, 2020 by nuhll

[sauso]

On 1/2/2020 at 9:37 PM, DZMM said:

Just discovered a major benefit!  With unionfs if you wanted to move files that existed in the rclone mount, it would first download them to the relevant folder in rclone_upload, then upload them and finally then delete the original file in the mount - a lot of transfer.  The only way to do quickly was to work directly on the rclone mount.

 

mergerfs just moves the file within the mount without downloading and re-uploading!  This should solve some of the transfer problems people have been having, as the move acts like a normal move within a drive without the lockups that you'd get sometimes with unionfs.

 

Much better handling of the merged folders 🙂

Oh man this will be awesome.  I had a bunch of files in my drive but wanted to change the modified date.   it downloaded it locally before it change the file.  will give this a shot tomorrow.

[DZMM]

I have updated my scripts on https://github.com/BinsonBuzz/unraid_rclone_mount and the first post in this thread (partially) to include mergerfs support.  I will keep github updated, so if you want the latest versions then watch that thread as well.   I've also improved the annotation so if you are already setup with unionfs it should be easy to change:

 

How to Migrate from unionfs:

1. In the new mount and unmount scripts find and replace mount_mergerfs with mount_unionfs and all should still work.  You need to keep your unionfs mount location, otherwise Plex, radarr, sonarr etc will need to rescan (very painful if lots of files).  New users can start off with mount_mergerfs
2. Change your mappings for nzbget, transmission etc to point to /mnt/user/mount_unionfs/downloads - IMPORTANT if you want the best benefits from mergerfs - see below
3. Either finish any existing uploads from rclone_upload before updating, or move pending downloads from /mnt/user/rclone_upload to the new /mnt/user/local folder, or create a version of the new upload script to upload from /mnt/user/local 

 

Benefits over mergerfs:

1. No need for cleanup script: mergerfs moves, copies,renames, deletes etc files that are on the mount actually on the mount, not how unionfs (i) would download and then re-upload a file or (ii) mask that the file hadn't actually been renamed or deleted on the mount = MUCH BETTER FILE HANDLING AND LESS TRANSFER, PLUS ONE LESS SCRIPT
2. Hardlink support: Unionfs didn't support hardlinks so any torrents had to be copied to rclone_upload and then uploaded.  Mergerfs supports hardlinks so no wasted transfer.  I've added an upload exclusion to /mnt/user/local/downloads so that download files (intermediate, pending sonarr/raddarr import, or seeds) are not uploaded.  For hardlinks to work, transmission/torrent clients HAVE to be mapped to the same disk 'storage' so files need to be in /mnt/user/mount_unionfs or for newer users /mnt/user/mount_mergerfs...hope this makes sense = MUCH BETTER FILE HANDLING AND LESS TRANSFER AND FILES NOT STORED TWICE
3. Other Transfer benefits: moving files within mergerfs is better supported and appear seamless e.g. scenarios before where a new nzbget file had to be copied by unionfs to the upload folder no longer occurs - the file is now moved instantly i.e. no disk write; moving files between folders even if non-cloud to cloud and vice versa, are much much faster  = MUCH BETTER FILE HANDLING AND LESS TRANSFER

 

I'd advise everyone to do the migration as your drives will spin a lot less and the end product is a lot smoother.

 

UPDATE: if you just want a quick migration without all the benefits, then swap:

 

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

for:

# Build mergerfs binary and delete old binary as precaution
rm /bin/mergerfs

# Create Docker
docker run -v /mnt/user/appdata/other/mergerfs:/build --rm trapexit/mergerfs-static-build

# move to bin to use for commands
mkdir -p /mnt/user/appdata/other/mergerfs/
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

# Create mergerfs mount

mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

But, I'd advise updating your download docker mappings to get all the file handling benefits.

Edited January 4, 2020 by DZMM
tidy up

[DZMM]

10 hours ago, nuhll said:

didnt noticed there was a follow up...

 

ill try it the next days, like always, THANK YOU.

 

Question is, what happens with the old files?

read my latest post - if you're careful, there will be no impact for a lot of gain

[DZMM]

4 hours ago, nuhll said:

What i forget to ask... what happens to old files? Like i had the problem it wont transfer anything because "size differ", will that fix it?

 

If done carefully no impact.  It will probably solve your transfer problems and other file handling issues unionfs created.

 

4 hours ago, nuhll said:

Iam a bit confused bc it looks different then my unionfs entry...

 

is that correct? (i dont know about the file paths)

 

 

 

 

Thats how it looked before:
 

 

 

 

 

Another question... is "-o rw,..." one command or 2 commands?

It's one command - it's just long.  Check out github where it's clearer.

 

If you want to switch just the unionfs mount quickly then swap:

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

for:

# Build mergerfs binary and delete old binary as precaution
rm /bin/mergerfs

# Create Docker
docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build

# move to bin to use for commands
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

# Create mergerfs mount

mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

But, I'd also change the way your uploads work (new script, plus changing your download client mappings) so that you can benefit from the better file handling between your download clients and the new mergerfs mount.

[DZMM]

Just now, DZMM said:

If you want to switch just the unionfs mount quickly then swap:

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

for:

# Build mergerfs binary and delete old binary as precaution
rm /bin/mergerfs

# Create Docker
docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build

# move to bin to use for commands
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

# Create mergerfs mount

mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

But, I'd also change the way your uploads work (new script, plus changing your download client mappings) so that you can benefit from the better file handling between your download clients and the new mergerfs mount.

 

[DZMM]

8 hours ago, Spladge said:

Will switch over soon - too much going on here at the moment. 
Another benefit, much like the server side copying between remotes is the renaming happens server side.

 

Is there a character limit on the rclone.cofig file size with the plugin, it doesn't seem to let me go past four remotes. Have yet to look into this so apologies if it is in the readme already.

I don't think there's a limit - I've got 30+ remotes setup I think.  Mounting may be different - I have 4 rclone mounts

[sauso]

Hey @DZMM i seem to be having problems with the docker run in the script.  Works perfectly if i run it manually.  Getting a TTY error.

 

Removing the -it seems to make it work.  Not sure if you are seeing the same issues.

[sauso]

wow.  I seemed to also be missing --vfs-cache-mode writes from my mount.  I have just seen a drastic improvement of playback with that one change.

 

Also after i fixed the docker not running in the script everything is working perfectly.

 

So happy this finally got done!

[DZMM]

4 minutes ago, sauso said:

wow.  I seemed to also be missing --vfs-cache-mode writes from my mount.  I have just seen a drastic improvement of playback with that one change.

 

Also after i fixed the docker not running in the script everything is working perfectly.

 

So happy this finally got done!

lol the vfs-cache-mode is the main point of this guide to get playback startup times similar to spinning up a HDD!

[DZMM]

14 minutes ago, sauso said:

Hey @DZMM i seem to be having problems with the docker run in the script.  Works perfectly if i run it manually.  Getting a TTY error.

 

Removing the -it seems to make it work.  Not sure if you are seeing the same issues.

re -it Not sure.  It worked for me when I was doing via command line and in scripts, but I haven't rebooted since.

 

Let's see how others get on.

[sauso]

1 minute ago, DZMM said:

lol the vfs-cache-mode is the main point of this guide to get playback startup times similar to spinning up a HDD!

yeah no idea how i missed it!  I was getting decent times already but now I would even say it is faster than my spin up time!

[DZMM]

29 minutes ago, sauso said:

yeah no idea how i missed it!  I was getting decent times already but now I would even say it is faster than my spin up time!

I also think mergerfs is a bit faster than unionfs - not sure yet

[NewDisplayName]

9 hours ago, DZMM said:

If done carefully no impact.  It will probably solve your transfer problems and other file handling issues unionfs created.

 

It's one command - it's just long.  Check out github where it's clearer.

 

If you want to switch just the unionfs mount quickly then swap:

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

for:

# Build mergerfs binary and delete old binary as precaution
rm /bin/mergerfs

# Create Docker
docker run -v /mnt/user/appdata/other/mergerfs:/build --rm -it trapexit/mergerfs-static-build

# move to bin to use for commands
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

# Create mergerfs mount

mergerfs /mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

But, I'd also change the way your uploads work (new script, plus changing your download client mappings) so that you can benefit from the better file handling between your download clients and the new mergerfs mount.

Thanks. Ill try that and restart and report back.

 

edit:

In the mean time... im again a little bit confused.  My "extra" directory is missing in your line. (/mnt/user/Archiv - so this wouldnt work for me?) Thats my old command:

 

Quote

 

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RO:/mnt/user/Archiv=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

 

 

I dont use rclone upload folder anyway. I run this:

Quote

rclone move --rc --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Filme gdrive_media_vfs:Filme -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Serien gdrive_media_vfs:Serien -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Musik gdrive_media_vfs:Musik -vv --drive-chunk-size 8M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/*

 

Edited January 3, 2020 by nuhll

[DZMM]

3 hours ago, nuhll said:

Thanks. Ill try that and restart and report back.

 

edit:

In the mean time... im again a little bit confused.  My "extra" directory is missing in your line. (/mnt/user/Archiv - so this wouldnt work for me?) 

I'm not sure how to create mergerfs mounts with more than 2 drives, but I think this is what you want:

 

mergerfs /mnt/user/Archiv:/mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

where the first folder /mnt/user/Archiv will be the only RW folder.   Test it by creating a mergerfs mount and turn off radarr, sonarr etc anything that might write to the mount and create a few text files and see what happens

 

 

Edited January 3, 2020 by DZMM

[Spladge]

Not sure if it has been mentioned here but you will also want server_side_across_configs = true in your rclone.config.
 

[tv_drive3]
team_drive = 0123ABC123ABC
service_account_file = /mnt/user/opt/sa/4.json
server_side_across_configs = true
type = drive
scope = drive

 

[NewDisplayName]

1 hour ago, DZMM said:

I'm not sure how to create mergerfs mounts with more than 2 drives, but I think this is what you want:

 

mergerfs /mnt/user/Archiv:/mnt/user/rclone_upload/google_vfs:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true

where the first folder /mnt/user/Archiv will be the only RW folder.   Test it by creating a mergerfs mount and turn off radarr, sonarr etc anything that might write to the mount and create a few text files and see what happens

 

 

Ok, ive changed it and restartet, seems to work (?).

 

Atleast all dockers seems to have startet. (tehy shouldnt start if mount is not working)

 

 

I could write and read data to and from \mount_unionfs\google_vfs\ does that mean its working?

 

rename is instant (but i never notcied it was, if im real)

 

Only one problem left (i posted this some pages ago)

 

The upload script doesnt do what it should  in the documentation for rclone move it says it will overwrite data, but im not sure if its correct.

 

thousends of 

 

2020/01/04 02:01:02 DEBUG : Name1 (2008)/Name1 2008.mkv: Sizes differ (src 24396751949 vs dst 1720934864)
2020/01/04 02:01:02 DEBUG : Name3 (2017)/Name3 2017.mkv: Sizes differ (src 13663756845 vs dst 4746737011)
2020/01/04 02:01:02 DEBUG : Name4 (2012)/Name4 2012.mkv: Sizes differ (src 23590880172 vs dst 7592177683)

2020/01/04 02:01:02 NOTICE: Name5 (1975)/Name4 1975.mkv: Duplicate object found in destination - ignoring
2020/01/04 02:01:02 DEBUG : Name5 (1975)/Name4 1975.mkv: Sizes differ (src 15120966311 vs dst 4675106354)

 

This are only notices(?), does that mean it will get at some point overwritten?

 

rclone move --rc --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Filme gdrive_media_vfs:Filme -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Serien gdrive_media_vfs:Serien -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Musik gdrive_media_vfs:Musik -vv --drive-chunk-size 8M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/*
 

 

Edited January 4, 2020 by nuhll

[DZMM]

13 minutes ago, nuhll said:

Ok, ive changed it and restartet, seems to work (?).

 

Atleast all dockers seems to have startet. (tehy shouldnt start if mount is not working)

 

 

I could write and read data to and from \mount_unionfs\google_vfs\ does that mean its working?

 

rename is instant (but i never notcied it was, if im real)

 

Only one problem left (i posted this some pages ago)

 

The upload script doesnt do what it should  in the documentation for rclone move it says it will overwrite data, but im not sure if its correct.

 

thousends of 

 

2020/01/04 02:01:02 DEBUG : Name1 (2008)/Name1 2008.mkv: Sizes differ (src 24396751949 vs dst 1720934864)
2020/01/04 02:01:02 DEBUG : Name3 (2017)/Name3 2017.mkv: Sizes differ (src 13663756845 vs dst 4746737011)
2020/01/04 02:01:02 DEBUG : Name4 (2012)/Name4 2012.mkv: Sizes differ (src 23590880172 vs dst 7592177683)

2020/01/04 02:01:02 NOTICE: Name5 (1975)/Name4 1975.mkv: Duplicate object found in destination - ignoring
2020/01/04 02:01:02 DEBUG : Name5 (1975)/Name4 1975.mkv: Sizes differ (src 15120966311 vs dst 4675106354)

 

This are only notices(?), does that mean it will get at some point overwritten?

 

rclone move --rc --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Filme gdrive_media_vfs:Filme -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Serien gdrive_media_vfs:Serien -vv --drive-chunk-size 128M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/* && rclone move --rc-addr=192.168.86.103:5573 --rc-user=test --rc-pass=test /mnt/user/Archiv/Musik gdrive_media_vfs:Musik -vv --drive-chunk-size 8M --checkers 4 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --tpslimit 3 --min-age 1y && rm /mnt/user/appdata/other/speed/*
 

 

Can't help you re the upload script as you've gone way off-piste with your arguments - try the rclone forum

[Stupifier]

13 hours ago, DZMM said:

Benefits over mergerfs:

1. No need for cleanup script: mergerfs moves, copies,renames, deletes etc files that are on the mount actually on the mount, not how unionfs (i) would download and then re-upload a file or (ii) mask that the file hadn't actually been renamed or deleted on the mount = MUCH BETTER FILE HANDLING AND LESS TRANSFER, PLUS ONE LESS SCRIPT
2. Hardlink support: Unionfs didn't support hardlinks so any torrents had to be copied to rclone_upload and then uploaded.  Mergerfs supports hardlinks so no wasted transfer.  I've added an upload exclusion to /mnt/user/local/downloads so that download files (intermediate, pending sonarr/raddarr import, or seeds) are not uploaded.  For hardlinks to work, transmission/torrent clients HAVE to be mapped to the same disk 'storage' so files need to be in /mnt/user/mount_unionfs or for newer users /mnt/user/mount_mergerfs...hope this makes sense = MUCH BETTER FILE HANDLING AND LESS TRANSFER AND FILES NOT STORED TWICE
3. Other Transfer benefits: moving files within mergerfs is better supported and appear seamless e.g. scenarios before where a new nzbget file had to be copied by unionfs to the upload folder no longer occurs - the file is now moved instantly i.e. no disk write; moving files between folders even if non-cloud to cloud and vice versa, are much much faster  = MUCH BETTER FILE HANDLING AND LESS TRANSFER

In addition....mergerfs actually has a straight-up read/write performance gain vs unionfs. Just thought I'd throw that in there. I'm really not sure of any downside really

[NewDisplayName]

3 minutes ago, DZMM said:

Can't help you re the upload script as you've gone way off-piste with your arguments - try the rclone forum

Its literally what youve posted, just chained together, so it starts with Filme, then goes to Series and so on...

 

&& rm /mnt/user/appdata/other/speed/* is just there to make my "limit upload bw script" working... 

 

 

In addition....mergerfs actually has a straight-up read/write performance gain vs unionfs. Just thought I'd throw that in there. I'm really not sure of any downside really

 

yeah ive noticed it does more mb/s then my connection should be capable of

 

Edited January 4, 2020 by nuhll

[DZMM]

10 hours ago, Spladge said:

Not sure if it has been mentioned here but you will also want server_side_across_configs = true in your rclone.config.
 

[tv_drive3]
team_drive = 0123ABC123ABC
service_account_file = /mnt/user/opt/sa/4.json
server_side_across_configs = true
type = drive
scope = drive

 

ooohhh, never seen that one before!  I really need to read-up on the advanced config options one day.

 

https://rclone.org/drive/#drive-server-side-across-configs

 

I wish this was available last year when I was trying to do some major moving.

 

Added.  Thanks