NewDisplayName Posted June 12, 2019 Share Posted June 12, 2019 (edited) 20 minutes ago, DZMM said: This thread has got a lot more action than I or @Kaizac or @slimshizn probably ever hoped for. There's been a lot of action getting people up and running, so I'm wondering how some of the people in the background are getting on? How intensively are other people using rclone? Have you moved all your media? How are you finding the playback experience? Personally, I don't have any plex content on my unRAID server anymore, except for my photos and I've now got over 300TB of Plex content stored on gdrive, as well as another big crypt with my backups (personal files, VM images etc). I don't even notice the impact of streaming anymore and when I do have any skips, I actually think they are because of local wi-fi issues rather than rclone. Perfectly 100%. Im only on a 50-100Mbits LTE connection, and its perfect. I dont mind to wait 7 sec for a video to start, if disks are spun down, it also takes some time... I store everything on gdrive (or in the move to it). But ill wait before i move EVERYTHING till i have my 1gbits connection... I dont see any problems, and i think it can only get better. Best scenario would be a all in one solution (docker and or plugin) I would pay for it. Like a UI with all settings, statistics, configs, fail saves, auto start and stop, warnings and so on.... Besides unraid its the best technical thing i found out about in years, i even startet removing drives (as my drives start to fail) which lowers my power consumption (and cost) Edited June 12, 2019 by nuhll 1 Quote Link to comment
DZMM Posted June 12, 2019 Author Share Posted June 12, 2019 6 minutes ago, nuhll said: Perfectly 100%. Im only on a 50-100Mbits LTE connection, and its perfect. I dont mind to wait 7 sec for a video to start, if disks are spun down, it also takes some time... I store everything on gdrive (or in the move to it). But ill wait before i move EVERYTHING till i have my 1gbits connection... I dont see any problems, and i think it can only get better. Best scenario would be a all in one solution (docker and or plugin) I would pay for it. Like a UI with all settings, statistics, configs, fail saves, auto start and stop, warnings and so on.... Besides unraid its the best technical thing i found out about in years, i even startet removing drives (as my drives start to fail) which lowers my power consumption (and cost) Only problem I used to have was syncing to local devices, but this has been working perfectly for the last month or so. I'm losing my 1Gbps connection in a few months, so I'll be investing the savings on my line rental on a dedicated server. I've also had to rethink how I use my local server now it's not a big media store anymore and I also didn't replace one drive when it failed and I'm considering selling a couple on eBay as they are redundant now. Quote Link to comment
NewDisplayName Posted June 12, 2019 Share Posted June 12, 2019 (edited) 18 minutes ago, DZMM said: Only problem I used to have was syncing to local devices, but this has been working perfectly for the last month or so. I'm losing my 1Gbps connection in a few months, so I'll be investing the savings on my line rental on a dedicated server. I've also had to rethink how I use my local server now it's not a big media store anymore and I also didn't replace one drive when it failed and I'm considering selling a couple on eBay as they are redundant now. Dont use mobile devices much, but streaming did work in holiday normally. Germany -> turkey Edited June 12, 2019 by nuhll Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 34 minutes ago, DZMM said: restart krusader (this issue keeps coming up) as the file rclone_mount_running is in /mnt/user/appdata/other/rclone I found it , what do you want me to do? this is in root unraid user appdata other rclone Quote Link to comment
DZMM Posted June 12, 2019 Author Share Posted June 12, 2019 9 minutes ago, neow said: I found it , what do you want me to do? this is in root unraid user appdata other rclone delete rclone_mount_running and re run the mount script 1 Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 1 minute ago, DZMM said: delete rclone_mount_running and re run the mount script ok i have this now Script location: /tmp/user.scripts/tmpScripts/rclone_mount_plugin/script Note that closing this window will abort the execution of this script 12.06.2019 23:33:03 INFO: mounting rclone vfs. 12.06.2019 23:33:08 INFO: Check rclone gdrive vfs mount success. 12.06.2019 23:33:08 INFO: Check successful, unionfs mounted. 12.06.2019 23:33:08 INFO: dockers already started seems better? isn't it? but upload script is not working i think Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script: line 4: $'\357\273\277': command not found /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script: line 26: $'\357\273\277': command not found 12.06.2019 23:33:41 INFO: rclone not installed - will try again later. Quote Link to comment
NewDisplayName Posted June 12, 2019 Share Posted June 12, 2019 (edited) the same issue as always, there are hidden things inside the config you copy and pasted, just remove it (rewrite it by hand) (u see the lines) Its a bug in this forum software, i think. Edited June 12, 2019 by nuhll Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 1 minute ago, nuhll said: the same issue as always, there are hidden things inside the config you copy and pasted, just remove it (rewrite it by hand) (u see the lines) Its a bug in this forum software, i think. I have not the understanding of the problem to act. I can follow instructions that’s all unfortunately Quote Link to comment
NewDisplayName Posted June 12, 2019 Share Posted June 12, 2019 hidden characters in what you have copied. /tmp/user.scripts/tmpScripts/rclone_upload_plugin line 4 line 26 rewrite these lines by hand to remove the invisible parts. 1 Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 5 minutes ago, nuhll said: hidden characters in what you have copied. /tmp/user.scripts/tmpScripts/rclone_upload_plugin line 4 line 26 rewrite these lines by hand to remove the invisible parts. ok i just did it, now i have this Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script 12.06.2019 23:47:29 INFO: rclone not installed - will try again later. Quote Link to comment
DZMM Posted June 12, 2019 Author Share Posted June 12, 2019 8 minutes ago, neow said: ok i just did it, now i have this Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script 12.06.2019 23:47:29 INFO: rclone not installed - will try again later. your mount isn't running - it needs to be for the upload script to be run Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 1 minute ago, DZMM said: your mount isn't running - it needs to be for the upload script to be run Ok i understand. What should i do? Quote Link to comment
DZMM Posted June 12, 2019 Author Share Posted June 12, 2019 2 minutes ago, neow said: Ok i understand. What should i do? run the mount script in the background 1 Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 (edited) 5 minutes ago, DZMM said: run the mount script in the background 5 minutes ago, DZMM said: run the mount script in the background ok like this? 1- i run on background mount 2- i run script upload 3- i get this : Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script 13.06.2019 00:05:22 INFO: rclone not installed - will try again later. 4- here is my upload script : #!/bin/bash ####### Check if script already running ########## if [[ -f "/mnt/user/appdata/other/rclone/rclone_upload" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting as script already running." exit else touch /mnt/user/appdata/other/rclone/rclone_upload fi ####### End Check if script already running ########## ####### check if rclone installed ########## if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: rclone installed successfully - proceeding with upload." else echo "$(date "+%d.%m.%Y %T") INFO: rclone not installed - will try again later." rm /mnt/user/appdata/other/rclone/rclone_upload exit fi ####### end check if rclone installed ########## # move files rclone move /mnt/user/rclone_upload/google_vfs/ gdrive_media_vfs: -vv --drive-chunk-size 512M --checkers 3 --fast-list --transfers 2 --exclude .unionfs/** --exclude *fuse_hidden* --exclude *_HIDDEN --exclude .recycle** --exclude *.backup~* --exclude *.partial~* --delete-empty-src-dirs --fast-list --bwlimit 9500k --tpslimit 3 --min-age 30m # remove dummy file rm /mnt/user/appdata/other/rclone/rclone_upload exit thanks for your patience i will be lost without you guys edit: it's maybe because my folder is not at the good place like i said 1 page ago in script this is the location it checked /mnt/user/rclone_upload/google_vfs/ gdrive_media_vfs Edited June 12, 2019 by neow Quote Link to comment
neow Posted June 12, 2019 Share Posted June 12, 2019 (edited) as you can see here this is a messup i don't know why i have one .appdata (i just deleted) one appdata one rclone_upload one mount_rclone at root unraid user one mount_rclone at root unraid user appdata other rclone i have the same kind of problem with script cleanup unionfs the problem is the script check root mnt user appdata and i have nothing there no? Edited June 12, 2019 by neow Quote Link to comment
NewDisplayName Posted June 12, 2019 Share Posted June 12, 2019 (edited) Do0 you have rclone installed? Just follow the instructions step by step. If you have questions ask specific questions. If you follow the guide, its working 100% (besides that hidden chars) If that is working and u understnad how it works, then you might adjust it to your needs. Edited June 12, 2019 by nuhll Quote Link to comment
DZMM Posted June 13, 2019 Author Share Posted June 13, 2019 8 hours ago, neow said: as you can see here this is a messup i don't know why i have one .appdata (i just deleted) one appdata one rclone_upload one mount_rclone at root unraid user one mount_rclone at root unraid user appdata other rclone i have the same kind of problem with script cleanup unionfs the problem is the script check root mnt user appdata and i have nothing there no? The info message i added is a bit misleading as the 'error' could be that the mount failed, not that rclone isn't installed. You successfully mounted previously so I think all your attempts haven't cleared properly or something - try manually removing the files in user/appears/other/rclone and rebooting your server and see what happens Quote Link to comment
neow Posted June 13, 2019 Share Posted June 13, 2019 7 hours ago, nuhll said: Do0 you have rclone installed? Just follow the instructions step by step. If you have questions ask specific questions. If you follow the guide, its working 100% (besides that hidden chars) If that is working and u understnad how it works, then you might adjust it to your needs. hello i followed step by step be sure of that Quote Link to comment
neow Posted June 13, 2019 Share Posted June 13, 2019 (edited) 1 hour ago, DZMM said: The info message i added is a bit misleading as the 'error' could be that the mount failed, not that rclone isn't installed. You successfully mounted previously so I think all your attempts haven't cleared properly or something - try manually removing the files in user/appears/other/rclone and rebooting your server and see what happens hello i reboot this morning an before i cleared the files you told me, please fin enclosed the result: Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script 13.06.2019 09:36:44 INFO: rclone installed successfully - proceeding with upload. 2019/06/13 09:36:44 DEBUG : --min-age 30m0s to 2019-06-13 09:06:44.438200373 +0200 CEST m=-1799.994050761 2019/06/13 09:36:44 DEBUG : rclone: Version "v1.47.0-123-g939b19c3-beta" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone-beta/.rclone.conf" "move" "/mnt/user0/rclone_upload/google_vfs/" "gdrive_media_vfs:" "-vv" "--drive-chunk-size" "512M" "--checkers" "3" "--fast-list" "--transfers" "2" "--exclude" ".unionfs/**" "--exclude" "*fuse_hidden*" "--exclude" "*_HIDDEN" "--exclude" ".recycle**" "--exclude" "*.backup~*" "--exclude" "*.partial~*" "--delete-empty-src-dirs" "--fast-list" "--bwlimit" "9500k" "--tpslimit" "3" "--min-age" "30m"] 2019/06/13 09:36:44 DEBUG : Using config file from "/boot/config/plugins/rclone-beta/.rclone.conf" 2019/06/13 09:36:44 INFO : Starting bandwidth limiter at 9.277MBytes/s 2019/06/13 09:36:44 INFO : Starting HTTP transaction limiter: max 3 transactions/s with burst 1 2019/06/13 09:36:45 DEBUG : tl1q5apobimhco7a7eht612i0c: Skipping undecryptable file name: Bad PKCS#7 padding - too long 2019/06/13 09:36:45 INFO : Encrypted drive 'gdrive_media_vfs:': Waiting for checks to finish 2019/06/13 09:36:45 INFO : Encrypted drive 'gdrive_media_vfs:': Waiting for transfers to finish 2019/06/13 09:36:45 INFO : Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA - Errors: 0 Checks: 0 / 0, - Transferred: 0 / 0, - Elapsed time: 1.3s 2019/06/13 09:36:45 DEBUG : 6 go routines active 2019/06/13 09:36:45 DEBUG : rclone: Version "v1.47.0-123-g939b19c3-beta" finishing with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone-beta/.rclone.conf" "move" "/mnt/user0/rclone_upload/google_vfs/" "gdrive_media_vfs:" "-vv" "--drive-chunk-size" "512M" "--checkers" "3" "--fast-list" "--transfers" "2" "--exclude" ".unionfs/**" "--exclude" "*fuse_hidden*" "--exclude" "*_HIDDEN" "--exclude" ".recycle**" "--exclude" "*.backup~*" "--exclude" "*.partial~*" "--delete-empty-src-dirs" "--fast-list" "--bwlimit" "9500k" "--tpslimit" "3" "--min-age" "30m"] i think is better isn't it? i tried to run cleanup plugin he is the result: Script location: /tmp/user.scripts/tmpScripts/rclone_custom_plugin/script Note that closing this window will abort the execution of this script 13.06.2019 09:39:28 INFO: starting unionfs cleanup. find: '/mnt/user/mount_unionfs/google_vfs/.unionfs': No such file or directory find: '/mnt/user/mount_unionfs/google_vfs/.unionfs': No such file or directory Edited June 13, 2019 by neow Quote Link to comment
Kaizac Posted June 13, 2019 Share Posted June 13, 2019 10 hours ago, DZMM said: This thread has got a lot more action than I or @Kaizac or @slimshizn probably ever hoped for. There's been a lot of action getting people up and running, so I'm wondering how some of the people in the background are getting on? How intensively are other people using rclone? Have you moved all your media? How are you finding the playback experience? Personally, I don't have any plex content on my unRAID server anymore, except for my photos and I've now got over 300TB of Plex content stored on gdrive, as well as another big crypt with my backups (personal files, VM images etc). I don't even notice the impact of streaming anymore and when I do have any skips, I actually think they are because of local wi-fi issues rather than rclone. I also have all my media stored on my Gdrive, with nothing local. Only thing I keep local are the small files, like subtitles and .nfo's because of the Gdrive/Tdrive file limit. I also keep a backup on Gdrive of my local files. Recently had a problem that I couldn't play files and it turned out my API was temporarily banned. I suspect that both Emby and Plex were analyzing files too much to get subtitles. So I switched to another API and it worked again. Something that has been an ongoing issue is memory creep. I've had multiple times that the server gave an out of memory error. I think it's because the scripts running simultaneously take up too much RAM (upload/upload backup/moving small files/cleanup). I will experiment a bit more with lowering the buffer sizes to lower the RAM usage. But with 28GB RAM I didn't expect to run into problems to be honest. Quote Link to comment
francrouge Posted June 13, 2019 Share Posted June 13, 2019 What plex docker are you using guys I'm using the offical one and it seem to bug a lot since 1 week.Envoyé de mon Pixel 2 XL en utilisant Tapatalk Quote Link to comment
testdasi Posted June 13, 2019 Share Posted June 13, 2019 On 11/6/2018 at 12:44 PM, DZMM said: With access to unlimited cloud space available for the cost of a domain name and around $510/pm How did you get "unlimited cloud space"? I thought it's only 2TB. Quote Link to comment
NewDisplayName Posted June 13, 2019 Share Posted June 13, 2019 44 minutes ago, testdasi said: How did you get "unlimited cloud space"? I thought it's only 2TB. Bc thats what it is. They say you get unlimited if you pay 5 (?) or so ppl, but they dont really do it that way, you also get unlimited with one person. Unlimited diskspace, unlimited bandwidth. I pay around 10€. Quote Link to comment
NewDisplayName Posted June 13, 2019 Share Posted June 13, 2019 1 hour ago, francrouge said: What plex docker are you using guys I'm using the offical one and it seem to bug a lot since 1 week. Envoyé de mon Pixel 2 XL en utilisant Tapatalk using official one. Quote Link to comment
NewDisplayName Posted June 13, 2019 Share Posted June 13, 2019 1 hour ago, neow said: hello i reboot this morning an before i cleared the files you told me, please fin enclosed the result: Script location: /tmp/user.scripts/tmpScripts/rclone_upload_plugin/script Note that closing this window will abort the execution of this script 13.06.2019 09:36:44 INFO: rclone installed successfully - proceeding with upload. 2019/06/13 09:36:44 DEBUG : --min-age 30m0s to 2019-06-13 09:06:44.438200373 +0200 CEST m=-1799.994050761 2019/06/13 09:36:44 DEBUG : rclone: Version "v1.47.0-123-g939b19c3-beta" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone-beta/.rclone.conf" "move" "/mnt/user0/rclone_upload/google_vfs/" "gdrive_media_vfs:" "-vv" "--drive-chunk-size" "512M" "--checkers" "3" "--fast-list" "--transfers" "2" "--exclude" ".unionfs/**" "--exclude" "*fuse_hidden*" "--exclude" "*_HIDDEN" "--exclude" ".recycle**" "--exclude" "*.backup~*" "--exclude" "*.partial~*" "--delete-empty-src-dirs" "--fast-list" "--bwlimit" "9500k" "--tpslimit" "3" "--min-age" "30m"] 2019/06/13 09:36:44 DEBUG : Using config file from "/boot/config/plugins/rclone-beta/.rclone.conf" 2019/06/13 09:36:44 INFO : Starting bandwidth limiter at 9.277MBytes/s 2019/06/13 09:36:44 INFO : Starting HTTP transaction limiter: max 3 transactions/s with burst 1 2019/06/13 09:36:45 DEBUG : tl1q5apobimhco7a7eht612i0c: Skipping undecryptable file name: Bad PKCS#7 padding - too long 2019/06/13 09:36:45 INFO : Encrypted drive 'gdrive_media_vfs:': Waiting for checks to finish 2019/06/13 09:36:45 INFO : Encrypted drive 'gdrive_media_vfs:': Waiting for transfers to finish 2019/06/13 09:36:45 INFO : Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA - Errors: 0 Checks: 0 / 0, - Transferred: 0 / 0, - Elapsed time: 1.3s 2019/06/13 09:36:45 DEBUG : 6 go routines active 2019/06/13 09:36:45 DEBUG : rclone: Version "v1.47.0-123-g939b19c3-beta" finishing with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone-beta/.rclone.conf" "move" "/mnt/user0/rclone_upload/google_vfs/" "gdrive_media_vfs:" "-vv" "--drive-chunk-size" "512M" "--checkers" "3" "--fast-list" "--transfers" "2" "--exclude" ".unionfs/**" "--exclude" "*fuse_hidden*" "--exclude" "*_HIDDEN" "--exclude" ".recycle**" "--exclude" "*.backup~*" "--exclude" "*.partial~*" "--delete-empty-src-dirs" "--fast-list" "--bwlimit" "9500k" "--tpslimit" "3" "--min-age" "30m"] i think is better isn't it? i tried to run cleanup plugin he is the result: Script location: /tmp/user.scripts/tmpScripts/rclone_custom_plugin/script Note that closing this window will abort the execution of this script 13.06.2019 09:39:28 INFO: starting unionfs cleanup. find: '/mnt/user/mount_unionfs/google_vfs/.unionfs': No such file or directory find: '/mnt/user/mount_unionfs/google_vfs/.unionfs': No such file or directory Seems normal. Also it seems like you still click "run script" instead of "run script in background". If you click run script, it only run while the window is open. If you run in background you can also see the log on the right. Always run these scripts in background. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.