DZMM

No backup strategy - it's been working so well for me for over a year now, that if I wasn't posting in this thread, I would honestly have forgotten all about it and wouldn't even remember how I set it up! One thing that I considered with one person was to backup our tdrives with each other, so as long as we both didn't get booted at the same time, we'd have access to our content - we might get around to sorting this one day. I guess I'm assuming that if Google ever pulled the plug I'd have a few days to download what I want and to decide if I want to build a new server with 350TB+ of storage to hold everything....

Yes, everything is streamed - nothing is preserved/cached

must be something wrong with your internet connection as my gdrive connection never drops. the mount script if run on a cron job auto-remounts the rclone and unionfs mounts, so maybe increase the frequency

I'm ok I'm way under the 400k limit - maybe use a backup utility to create a zip then upload?

Looks right. Re the space, that's why I always use dashs or underlines in names to make my life easier

good I'm not sure why you'd want to do this. If you want to test first, just manually copy a tv show or movie or two to see what happens. If you really want to test the full 40GB, then in the upload script just change rclone move to rclone sync

it wouldn't be needed if you're not using unionfs

Do you have a /mnt/user/mount_rclone/google_vfs folder? If not create one. I think this will solve your problem. I've added: mkdir -p /mnt/user/mount_rclone/google_vfs to the mount script. I think there was a reason why it's not there, but I can't remember why so adding until someone tells me it causes a problem.

It could be plex - I would definitely turn off thumbnail creation if you have that on in your library settings. Maybe also turn off 'extensive media analysis' in scheduled tasks - others suggest this but I've had no problems and actually think it's a bad idea. If you look in status/alerts in plex you'll see what plex is doing - it's probably analysing files or creating thumbnails

No. If you can, reboot your server and run the script in the background using the user scripts plugin

It looks like you've created /mnt/user/mount_rclone/google_vfs so all should be good. Are you running the script in the background?

Are you using the user scripts plugin to run the mount script?

post your mount script please

Thanks. My trdrive_backup_vfs includes my plex appdata backup, but I use the CA backup plugin so it's just one big tar file

ahh didn't realise folders counted. My backup tdrive is further away from the limit than I thought: root@Highlander:/mnt/user/public# rclone size tdrive_vfs: --fast-list Total objects: 72443 Total size: 322.704 TBytes (354816683643653 Bytes) root@Highlander:/mnt/user/public# rclone size tdrive_backup_vfs: --fast-list Total objects: 80245 Total size: 8.307 TBytes (9133993530852 Bytes) Edit: are you sure folders count as objects? My movie structure is /movies/movie_name so each of my 16k movies has an associated folder, so my objects should be over 80k not 72k

What performance problems have you seen beyond 200k files? I've just checked and I'm at around 70k items for my plex media library tdrive, but I have another tdrive for backups which probably is over 200k files due to all the little system files 06.07.2019 00:50:41 PLEX LIBRARY STATS Media items in Libraries Library = Movies Items = 16147 Library = TV - Adults Items = 34549 Library = TV - Kids Items = 19054

This is for streaming files not backing up. But, yes you could use it for moving your videos to the cloud - I'm not sure about moving photos as I haven't tried that yet.

fixed on github. Sorry about that - not sure how that went missing

I run my mount script on a 5 min cron job */5 * * * * so even if I'm unlucky and the unmount hasn't run yet, it gets fixed on the next run 5 mins later

Run the unmount script at array start rather than shutdown - that's what I do

I started this way and eventually as I got more comfortable I went all in for plex media, and I've also lots of other files to the point where my server is just photos and personal docs. I'm gone from around 44TB in my array to only 16TB, where only around 4TB is permanent. Re seeking, to be my mind it's got better since I started using rclone but it might be because I've got used to it now that I've gone all in. Have you tried experimenting with --buffer-size ?? I think a bigger buffer might help with forward seeking (not backward of course)

This setup supports that - i.e. plex, sonarr, radarr etc all act as normal. Sonarr 'looks' at the mount_unionfs folder so it functions normally i.e. grab->download->move to media folder - all the rclone script behaviour is invisible to sonarr. The upload script 15m delay ensures that Sonarr has finished any post-processing (renaming etc) before the file is uploaded.

Thanks

Hi I'm getting this error in my logs but it still seems to be working: nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html) nginx: [error] lua_load_resty_core failed to load the resty.core module from https://github.com/openresty/lua-resty-core; ensure you are using an OpenResty release from https://openresty.org/en/download.html (rc: 2, reason: module 'resty.core' not found: no field package.preload['resty.core'] no file './resty/core.lua' no file '/usr/share/luajit-2.1.0-beta3/resty/core.lua' no file '/usr/local/share/lua/5.1/resty/core.lua' no file '/usr/local/share/lua/5.1/resty/core/init.lua' no file '/usr/share/lua/5.1/resty/core.lua' no file '/usr/share/lua/5.1/resty/core/init.lua' no file '/usr/share/lua/common/resty/core.lua' no file '/usr/share/lua/common/resty/core/init.lua' no file './resty/core.so' no file '/usr/local/lib/lua/5.1/resty/core.so' no file '/usr/lib/lua/5.1/resty/core.so' no file '/usr/local/lib/lua/5.1/loadall.so' no file './resty.so' no file '/usr/local/lib/lua/5.1/resty.so' no file '/usr/lib/lua/5.1/resty.so' no file '/usr/local/lib/lua/5.1/loadall.so')

yes - if you copy a file that already exists in mount_rclone I think unionfs overwrites it.