DZMM

@drogg if you are certain there's not an upgrade instance running, go into /mnt/user/appdata/other/rclone and your upload remote folder and manually delete the upload running checker file, or run the cleanup script. Basically the upload script didn't exit properly at some point.

yes. Use my scripts to: in your rclone config make sure server_side_across_configs is set to true mount script to mount your gdrive folder create your service accounts. I'd add about 100 if you're doing server side copies as they blow through the 750GB in no time as you discovered - 15 to 16 are needed per Gbps transferring full pelt per day in the upload script set your team drive as the destination and the gdrive from #2 as the source

@Stinkpickle I updated #3 as you should always use the mergerfs mount in dockers

no

Oh I see. I would do the following: LocalFilesShare2="/mnt/user/Media" in the mount script so that files from /mnt/user/Media just appear in /mnt/user/sgdrive_media/sgdrive_media. Make sure plex has empty trash automatically off In plex add the new /mnt/user/sgdrive_media_rclone/sgdrive_media /mnt/user/sgdrive_media or /mnt/user/sgdrive_media_mergerfs folders and remove the old /mnt/user/Media Also update sonarr, radarr locations etc Do a full plex scan - it will update all the file locations and keep all your existing metadata Once scan complete, check all looks ok, then manually empty trash Move files from /mnt/user/Media to /mnt/user/sgdrive_media_local Remove /mnt/user/Media from LocalFilesShare2 as shouldn't need it now use /mnt/user/sgdrive_media_local going forwards Edit: Personally I also would use this as opportunity to change MergerfsMountShare="/mnt/user/sgdrive_media" to MergerfsMountShare="/mnt/user/sgdrive_media_mergerfs" to make things easier to follow Edit2: Fixed #3

Unless I'm missing something, just do LocalFilesShare="/mnt/user/Media/" and then those files will get moved to google

mountcheck file either doesn't exist in the source folder to be copied to gdrive, or the target folder doesn't exist

I'd move everything to the team drive - as @testdasi said as soon as you start understanding what you can do with this setup, you'll want a teamdrive. To move: - stop all mounts, dockers etc - create a team drive and note the ID - it'll be in the url - add your google account as a user of the teamdrive if needed - within google drive move all the files from the gdrive folder to the teamdrive folder - ensure all the paths are the same i.e. gdrive/crypt/sub-folder -->tdrive/crypt/sub-folder. This should be pretty quick for 1TB - change your rclone config via the plugin settings page (easiest) to: [tdrive] type = drive scope = drive service_account_file = /mnt/user/appdata/other/rclone/service_accounts/sa_tdrive.json team_drive = Team DRIVE ID server_side_across_configs = true if you're setting up service_accounts at the same time. I'd advise doing this if you've got a decent connection and/or will be uploading more than 750GB/day now or in the future. If not, then change to: [tdrive] type = drive client_id = xxxxxxxxx.apps.googleusercontent.com client_secret = xxxxxxxxxxxxxxxx scope = drive root_folder_id = TEAM DRIVE ID token = {"access_token":"xxxxxxxxxxxxxxxxxx"} server_side_across_configs = true Edit: if you're using gdrive_media_vfs for your decrypted remote remember to change: remote = gdrive:crypt to remote = tdrive:crypt Then re-mount and if all looks ok, start your dockers.

Nope

teamdrive NOT team https://support.google.com/a/users/answer/9310351#!/

Thanks - I'll give this a try

Something's wrong as folders created either in /local or /mount_mergerfs should behave like normal folders i.e. radarr/sonarr etc adding/upgrading/removing when they want. Some apps like krusader and Plex need to be started after the mount, but that's the only problem I'm aware of. What are your docker mappings and rclone mount options?

You're welcome. What are you going to do with your empty HDDs? I sold mine, even my parity drive as I don't need it now. 12 simultaneous streams is good going - I think I've only hit 10 once over Christmas.

So, is everyone else's server getting hammered with everyone staying at home?

Sometimes when I convert books, my docker image fills up very fast and doesn't go down. I think this means the conversion occurs within the docker. Is there a way to fix this please as sometimes I have to delete my whole image. Also, how do I stop the Trash-99 folder being created? Thanks

Actually any schedule is fine - that's why the checker file is there to stop additional upload jobs starting if there's an instance already running. If you don't make user script changes while an instance is running, the logs should still keep updating to show you what's happening.

The unmount script doesn't have any fusermount commands, as the new script structure makes this difficult (mount locations are variable). The script is intended to be a cleanup script to be run at array start. Do you need it to run at array stop? If so, just add your own fusermount commands to the script.

It works. I have an asymmetric 360/180 connection (moved before xmas and lost my 1G symmetric connection 😞 ), so I tend to have more than 750GB pending upload - also because I use bwlimits to make sure I've got some spare upload left, even though I use traffic shaping on my pfsense VM. It stops once any transfers that started before 750GB hits have finished, and then resumes for the next run with a new SA.

Thanks - got some jobs coming now

I've managed to get the WebUI working, but it keeps saying 'Could not get an assignment'. Any ideas what's wrong? I've opened all ports on pfsense 08:48:01:************************* Folding@home Client ************************* 08:48:01: Website: https://foldingathome.org/ 08:48:01: Copyright: (c) 2009-2018 foldingathome.org 08:48:01: Author: Joseph Coffland <[email protected]> 08:48:01: Args: --config /config/config.xml 08:48:01: Config: /config/config.xml 08:48:01:******************************** Build ******************************** 08:48:01: Version: 7.5.1 08:48:01: Date: May 11 2018 08:48:01: Time: 19:59:04 08:48:01: Repository: Git 08:48:01: Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0 08:48:01: Branch: master 08:48:01: Compiler: GNU 6.3.0 20170516 08:48:01: Options: -std=gnu++98 -O3 -funroll-loops 08:48:01: Platform: linux2 4.14.0-3-amd64 08:48:01: Bits: 64 08:48:01: Mode: Release 08:48:01:******************************* System ******************************** 08:48:01: CPU: AMD Ryzen Threadripper 2950X 16-Core Processor 08:48:01: CPU ID: AuthenticAMD Family 23 Model 8 Stepping 2 08:48:01: CPUs: 32 08:48:01: Memory: 125.85GiB 08:48:01:Free Memory: 987.81MiB 08:48:01: Threads: POSIX_THREADS 08:48:01: OS Version: 4.19 08:48:01:Has Battery: false 08:48:01: On Battery: false 08:48:01: UTC Offset: 0 08:48:01: PID: 27 08:48:01: CWD: /config 08:48:01: OS: Linux 4.19.98-Unraid x86_64 08:48:01: OS Arch: AMD64 08:48:01: GPUs: 0 08:48:01: CUDA: Not detected: cuInit() returned 100 08:48:01: OpenCL: Not detected: clGetPlatformIDs() returned -1001 08:48:01:*********************************************************************** 08:48:01:<config> 08:48:01: <!-- Client Control --> 08:48:01: <fold-anon v='true'/> : 08:48:01: <!-- Folding Slot Configuration --> 08:48:01: <gpu v='false'/> : 08:48:01: <!-- HTTP Server --> 08:48:01: <allow v='192.168.30.0/24'/> : 08:48:01: <!-- Remote Command Server --> 08:48:01: <password v='********'/> : 08:48:01: <!-- Slot Control --> 08:48:01: <power v='FULL'/> : 08:48:01: <!-- User Information --> 08:48:01: <team v='227802'/> 08:48:01: <user v='DZMM'/> : 08:48:01: <!-- Web Server --> 08:48:01: <web-allow v='192.168.30.1/24'/> : 08:48:01: <!-- Folding Slots --> 08:48:01: <slot id='0' type='CPU'/> 08:48:01:</config> 08:48:01:Trying to access database... 08:48:01:Successfully acquired database lock 08:48:01:Enabled folding slot 00: READY cpu:32 08:48:01:WU00:FS00:Connecting to 65.254.110.245:8080 [93m08:48:02:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration[0m 08:48:02:WU00:FS00:Connecting to 18.218.241.186:80 [93m08:48:03:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration[0m [91m08:48:03:ERROR:WU00:FS00:Exception: Could not get an assignment[0m 08:48:04:WU00:FS00:Connecting to 65.254.110.245:8080 [93m08:48:04:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration[0m 08:48:04:WU00:FS00:Connecting to 18.218.241.186:80 [93m08:48:05:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration[0m [91m08:48:05:ERROR:WU00:FS00:Exception: Could not get an assignment[0m 08:49:04:WU00:FS00:Connecting to 65.254.110.245:8080 08:49:04:WU00:FS00:Connecting to 65.254.110.245:8080 [93m08:49:04:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration[0m 08:49:04:WU00:FS00:Connecting to 18.218.241.186:80 [93m08:49:05:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration[0m [91m08:49:05:ERROR:WU00:FS00:Exception: Could not get an assignment[0m

It rotates the SA the next time the script is run

It rotates the SA the next time the script is run

For the dockers who are starting with the script

It's been a while but I didn't do anything clever - I just followed these instructions https://github.com/xyou365/AutoRclone/blob/master/Readme.md. somehow I ended up with 500 not 100 though

Not sure what that is - did you follow the guide for creating SAs? Easiest way is one script per mount.