francrouge Posted July 2, 2019 Share Posted July 2, 2019 6 minutes ago, Kaizac said: You could use mount_rclone as your RW folder and it will download directly to your Gdrive. However this will slowed by your upload speed. And it will probably also cause problems while direct writing to the mount. Rclone copy/move/etc. is intended to solve this issue by doing file checks. ok thx Quote Link to comment
NewDisplayName Posted July 2, 2019 Share Posted July 2, 2019 The real question is: what you want to do? Do you want to have some files local, some (maybe old?) files online? Then this is the right solution. You want all online? Then this is the right solution. You want all offline? Then why would you need this? Quote Link to comment
francrouge Posted July 2, 2019 Share Posted July 2, 2019 The real question is: what you want to do? Do you want to have some files local, some (maybe old?) files online? Then this is the right solution. You want all online? Then this is the right solution. You want all offline? Then why would you need this? My main goal is to be able to rss files localy and upload them automaticaly after the post processingThxEnvoyé de mon Pixel 2 XL en utilisant Tapatalk Quote Link to comment
NewDisplayName Posted July 2, 2019 Share Posted July 2, 2019 "rss files locally"??? Quote Link to comment
francrouge Posted July 2, 2019 Share Posted July 2, 2019 "rss files locally"???Download torrent rss feed then process then upload to my gdriveEnvoyé de mon Pixel 2 XL en utilisant Tapatalk Quote Link to comment
NewDisplayName Posted July 2, 2019 Share Posted July 2, 2019 This program has nothing to do with downloading and processing it. But ofc it can upload files to gdrive, you can choose all, or just after some time (i would wait atleast half a year so it doenst upload x better versions of your linux isos when a new release comes out) 1 Quote Link to comment
francrouge Posted July 2, 2019 Share Posted July 2, 2019 8 minutes ago, nuhll said: This program has nothing to do with downloading and processing it. But ofc it can upload files to gdrive, you can choose all, or just after some time (i would wait atleast half a year so it doenst upload x better versions of your linux isos when a new release comes out) Perfect thx for the info Quote Link to comment
DZMM Posted July 2, 2019 Author Share Posted July 2, 2019 2 hours ago, francrouge said: Download torrent rss feed then process then upload to my gdrive Envoyé de mon Pixel 2 XL en utilisant Tapatalk This setup supports that - i.e. plex, sonarr, radarr etc all act as normal. Sonarr 'looks' at the mount_unionfs folder so it functions normally i.e. grab->download->move to media folder - all the rclone script behaviour is invisible to sonarr. The upload script 15m delay ensures that Sonarr has finished any post-processing (renaming etc) before the file is uploaded. 1 Quote Link to comment
francrouge Posted July 3, 2019 Share Posted July 3, 2019 15 hours ago, DZMM said: This setup supports that - i.e. plex, sonarr, radarr etc all act as normal. Sonarr 'looks' at the mount_unionfs folder so it functions normally i.e. grab->download->move to media folder - all the rclone script behaviour is invisible to sonarr. The upload script 15m delay ensures that Sonarr has finished any post-processing (renaming etc) before the file is uploaded. thx a lot Quote Link to comment
sauso Posted July 3, 2019 Share Posted July 3, 2019 In mine i set the upload folder to be my current Media Share. I then changed Plex, Sonarr, Radarr etc to point to the union fs mount. Currently i'm just manually shunting video's up to the cloud. But once i'm happy with my process i'm going to set my script to copy anything older than 1 year to the cloud. Then as i start to run out of storage i will increase the time. The way i see it i will eventually replace my current HDDs with more flash local playback of new in demand shows and then copy up to the cloud after. Process is working flawlessly for me. I can't even tell the difference between spinning up a drive and playing from the cloud. The only thing that is a bit slower on cloud is seeking. unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/Media=RW:/mnt/user/mount_rclone/google_vfs/Media=RO /mnt/user/mount_unionfs/google_vfs/Media 1 Quote Link to comment
DZMM Posted July 3, 2019 Author Share Posted July 3, 2019 1 hour ago, sauso said: In mine i set the upload folder to be my current Media Share. I then changed Plex, Sonarr, Radarr etc to point to the union fs mount. Currently i'm just manually shunting video's up to the cloud. But once i'm happy with my process i'm going to set my script to copy anything older than 1 year to the cloud. Then as i start to run out of storage i will increase the time. The way i see it i will eventually replace my current HDDs with more flash local playback of new in demand shows and then copy up to the cloud after. Process is working flawlessly for me. I can't even tell the difference between spinning up a drive and playing from the cloud. The only thing that is a bit slower on cloud is seeking. unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/Media=RW:/mnt/user/mount_rclone/google_vfs/Media=RO /mnt/user/mount_unionfs/google_vfs/Media I started this way and eventually as I got more comfortable I went all in for plex media, and I've also lots of other files to the point where my server is just photos and personal docs. I'm gone from around 44TB in my array to only 16TB, where only around 4TB is permanent. Re seeking, to be my mind it's got better since I started using rclone but it might be because I've got used to it now that I've gone all in. Have you tried experimenting with --buffer-size ?? I think a bigger buffer might help with forward seeking (not backward of course) 1 Quote Link to comment
francrouge Posted July 3, 2019 Share Posted July 3, 2019 I started this way and eventually as I got more comfortable I went all in for plex media, and I've also lots of other files to the point where my server is just photos and personal docs. I'm gone from around 44TB in my array to only 16TB, where only around 4TB is permanent. Re seeking, to be my mind it's got better since I started using rclone but it might be because I've got used to it now that I've gone all in. Have you tried experimenting with --buffer-size ?? I think a bigger buffer might help with forward seeking (not backward of course)For me its working great the only thing right now is that it takes a couple of second to load a movie from the gdriveEnvoyé de mon Pixel 2 XL en utilisant Tapatalk Quote Link to comment
sauso Posted July 4, 2019 Share Posted July 4, 2019 18 hours ago, DZMM said: I started this way and eventually as I got more comfortable I went all in for plex media, and I've also lots of other files to the point where my server is just photos and personal docs. I'm gone from around 44TB in my array to only 16TB, where only around 4TB is permanent. Re seeking, to be my mind it's got better since I started using rclone but it might be because I've got used to it now that I've gone all in. Have you tried experimenting with --buffer-size ?? I think a bigger buffer might help with forward seeking (not backward of course) Will give it a shot. Haven't really experimented with it yet. Got sidelined moving all my dockers to their own vlan. Quote Link to comment
NewDisplayName Posted July 8, 2019 Share Posted July 8, 2019 Its important that you have your stuff in seperate folders (!) like /linuximages/linuximage1 /linuximages/linuximage2 not alle in /linuximages/ Quote Link to comment
sauso Posted July 11, 2019 Share Posted July 11, 2019 So had a power outage last night and my server restarted. (will be investing in a UPS soon!) When my server came back up the mount script failed as the rclone_running file was still present. If the server is gracefully shut down the unmount script removes this file. Has anyone else found a way to combat this? Quote Link to comment
Spladge Posted July 11, 2019 Share Posted July 11, 2019 You probably just need to manually delete the "running" file then restart it. I have a lot of power outages. Quote Link to comment
sauso Posted July 11, 2019 Share Posted July 11, 2019 might touch a new file called unclean_shutdown and work that into the script to delete the running file. if it was an unclean shutdown Quote Link to comment
Kaizac Posted July 11, 2019 Share Posted July 11, 2019 2 hours ago, sauso said: might touch a new file called unclean_shutdown and work that into the script to delete the running file. if it was an unclean shutdown Just create 2 scripts. One for your rclone mount first start and your rclone continous mount. In your first start script you put at the beginning to delete the check files which should be removed to run the script properly. Quote Link to comment
DZMM Posted July 11, 2019 Author Share Posted July 11, 2019 5 hours ago, sauso said: So had a power outage last night and my server restarted. (will be investing in a UPS soon!) When my server came back up the mount script failed as the rclone_running file was still present. If the server is gracefully shut down the unmount script removes this file. Has anyone else found a way to combat this? Run the unmount script at array start rather than shutdown - that's what I do 1 Quote Link to comment
sauso Posted July 11, 2019 Share Posted July 11, 2019 9 minutes ago, DZMM said: Run the unmount script at array start rather than shutdown - that's what I do Bingo, That makes total sense! Quote Link to comment
Kaizac Posted July 11, 2019 Share Posted July 11, 2019 13 minutes ago, DZMM said: Run the unmount script at array start rather than shutdown - that's what I do How do you make sure the unmount script is run before the mount script? Quote Link to comment
DZMM Posted July 11, 2019 Author Share Posted July 11, 2019 44 minutes ago, Kaizac said: How do you make sure the unmount script is run before the mount script? I run my mount script on a 5 min cron job */5 * * * * so even if I'm unlucky and the unmount hasn't run yet, it gets fixed on the next run 5 mins later Quote Link to comment
sauso Posted July 11, 2019 Share Posted July 11, 2019 I feel like i'm going loopy. I can't see anywhere in the unmount script that removes /mnt/user/appdata/other/rclone/rclone_mount_running Am i going mad or is there another place that this is removed? Quote Link to comment
sdamaged Posted July 11, 2019 Share Posted July 11, 2019 Guys, is this a working strategy to back up pictures and videos to my Google Drive? Quote Link to comment
DZMM Posted July 11, 2019 Author Share Posted July 11, 2019 (edited) On 7/11/2019 at 11:02 AM, sauso said: I feel like i'm going loopy. I can't see anywhere in the unmount script that removes /mnt/user/appdata/other/rclone/rclone_mount_running Am i going mad or is there another place that this is removed? fixed on github. Sorry about that - not sure how that went missing Edited July 13, 2019 by DZMM 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.