NewDisplayName Posted April 19, 2019 Share Posted April 19, 2019 I dont want to break anything, and unmounting is only working when restarting... Quote Link to comment
NewDisplayName Posted April 19, 2019 Share Posted April 19, 2019 nvm, i fixed it, there were srsly a file in mount rclone UPLOAD... which caused this (i didnt looked there, because i dont use that directory) Quote Link to comment
DZMM Posted April 19, 2019 Author Share Posted April 19, 2019 just stop radarr, sonarr, plex etc - anything that's accessing the mount before you unmount. Shouldn't have any problems Quote Link to comment
NewDisplayName Posted April 19, 2019 Share Posted April 19, 2019 (edited) 1 minute ago, DZMM said: just stop radarr, sonarr, plex etc - anything that's accessing the mount before you unmount. Shouldn't have any problems Sorry for wasting your time, im just stupid. Thanks for your help. Just btw, since you created this post i uploaded all +1 year old files with only 10mbits UL Edited April 19, 2019 by nuhll Quote Link to comment
GorOps Posted April 23, 2019 Share Posted April 23, 2019 I posted a support question on the Rclone forums related to rclone mount causing unraid to get out of memory errors after some time. Need help with my config or mount script... If anyone from here is able to assist it would be greatly appreciated. https://forum.rclone.org/t/rclone-mount-causing-out-of-memory-in-unraid/9695 Quote Link to comment
TheFreemancer Posted April 24, 2019 Share Posted April 24, 2019 GorOps, that's a freaking coincidence. Last night was my first time streaming a remux from the cloud using the script provided here. Result, the entire server crashed and had to be forced shutdown and restarted. I have a very weak computer for unraid, a J1900 with 4GB of ram. I tweaked the script. I removed the buffer size limit off, changed buffer size to 512 instead of 1G. I removed all the upload folder stuff as I don't understand why this is needed, all my uploaded to gdrive is running rclone copy. I don't use a team drive so I removed it. And one thing that was causing problems for me is that the user script was set to run at the start of the array and I changed it to start of the array only (the last option on the drop down list). Also I my crypt mount is called secure: and the mountcheck file I first created was giving me some trouble, so I created a new one called mountcheck2. This is the tweaked version: #!/bin/bash sleep 10 ####### Check if script is already running ########## if [[ -f "/mnt/user/appdata/other/rclone/rclone_mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script already running." exit else touch /mnt/user/appdata/other/rclone/rclone_mount_running fi ####### End Check if script already running ########## sleep 10 ####### Start rclone mounts ########## # create directories for rclone mount and unionfs mount mkdir -p /mnt/user/appdata/other/rclone mkdir -p /mnt/user/mount_rclone/google_vfs mkdir -p /mnt/user/mount_unionfs/google_vfs ####### Start rclone gdrive mount ########## sleep 10 # check if gdrive mount already created if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck2" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check rclone vfs already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: mounting rclone vfs." sleep 10 rclone mount --rc --allow-other --buffer-size 512M --dir-cache-time 72h --drive-chunk-size 256M --fast-list --log-level INFO --vfs-read-chunk-size 128M secure: /mnt/user/mount_rclone/google_vfs & # check if mount successful # slight pause to give mount time to finalise sleep 10 if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck2" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check rclone gdrive vfs mount success." else echo "$(date "+%d.%m.%Y %T") CRITICAL: rclone gdrive vfs mount failed - please check for problems." rm /mnt/user/appdata/other/rclone/rclone_mount_running exit fi fi ####### End rclone gdrive mount ########## ####### Start unionfs mount ########## if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck2" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs already mounted." else unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/mount_rclone/google_vfs=RW /mnt/user/mount_unionfs/google_vfs if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck2" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs mounted." else echo "$(date "+%d.%m.%Y %T") CRITICAL: unionfs Remount failed." rm /mnt/user/appdata/other/rclone/rclone_mount_running exit fi fi ####### End Mount unionfs ########## exit Also I really would like some input from DZMM about the state of my setup here. Quote Link to comment
NewDisplayName Posted April 24, 2019 Share Posted April 24, 2019 (edited) Im not him, but i try to help. I guess it was just running out of RAM. Lowering the buffer is a good idea. Maybe even more, test it out until it starts buffering while watching. 4 gb is very low ram. (i also startet with 4gb, now im at 32). As a tip for lower ram consumption, many dockers have "ram leaks" and use more and more ram longer they run... i use ca auto backup to restart the dockers every day. So that fixes this problem. I dont know why mountcheck makes a problem, but you renamed all the check for the file to mountcheck2 so this should be np. If upload is working, youll be fine. Edited April 24, 2019 by nuhll Quote Link to comment
DZMM Posted April 24, 2019 Author Share Posted April 24, 2019 (edited) 4 hours ago, TheFreemancer said: I have a very weak computer for unraid, a J1900 with 4GB of ram. I tweaked the script. I removed the buffer size limit off, changed buffer size to 512 instead of 1G. With only 4GB you might need to go even smaller if you've got concurrent usage as I think the buffer is per stream. I see you also removed --vfs-read-chunk-size-limit off from your mount script. This defaults to off so removing won't help - if you're still having memory problems I'd set this again to something like 256MB of 512MB, but you're going to struggle to play remux files IMHO if you don't have enough ram to buffer the stream. I don't know what size chunks my setup requests, but I have no issues. 4 hours ago, TheFreemancer said: I removed all the upload folder stuff as I don't understand why this is needed, all my uploaded to gdrive is running rclone copy. I don't use a team drive so I removed it. If you're happy with only being able to play files once they've been uploaded then this is fine. However, if you have a lot of files that are pending upload and you want to Plex/Sonarr/Radarr etc to be able to see them, then this is where the unionfs and rclone_upload folders come into play - e.g. unless you make your setup a lot more complicated you can only upload 750GB/day. There are ways around this e.g. my setup allows me to upload at my full 1G line speed 247, but it took me a few hours to setup, but it means I don't have files waiting to be uploaded for more than a few mins. I still need the unionfs setup though so that Plex doesn't have to rescan thinking the file has changed when it moves, when it hasn't etc etc 4 hours ago, TheFreemancer said: And one thing that was causing problems for me is that the user script was set to run at the start of the array and I changed it to start of the array only (the last option on the drop down list). Check out the new scripts on github as it accounts for this e.g. if it tries to mount and there's no internet connection then it tries again a few mins later. This check is useful if your mount ever drops as it will automatically remount. Edited April 24, 2019 by DZMM Quote Link to comment
francrouge Posted April 25, 2019 Share Posted April 25, 2019 Hi guys my token seem to be expired.. Can someone tell me again how to renew it please Thx Quote Link to comment
francrouge Posted April 26, 2019 Share Posted April 26, 2019 4 hours ago, francrouge said: Hi guys my token seem to be expired.. Can someone tell me again how to renew it please Thx i'm getting this error 2019/04/25 20:41:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:42:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:43:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:44:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError Quote Link to comment
DZMM Posted April 26, 2019 Author Share Posted April 26, 2019 9 hours ago, francrouge said: Hi guys my token seem to be expired.. Can someone tell me again how to renew it please Thx I'm not sure what's going on there or if this will solve your problems: https://rclone.org/drive/#making-your-own-client-id You might need to post on the rclone forums 1 Quote Link to comment
Kaizac Posted April 26, 2019 Share Posted April 26, 2019 6 hours ago, francrouge said: i'm getting this error 2019/04/25 20:41:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:42:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:43:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError 2019/04/25 20:44:38 INFO : Google drive root 'crypt': Failed to get StartPageToken: googleapi: Error 401: Invalid Credentials, authError Did you set up your own client id before (per what DZMM linked)? If so log in to your Google Admin page and see whether these credentials are still valid. If so, rerun the config for your mount(s) and put in the client id and secret again. Don't alter anything else. Then when you get to the end of the config it will ask you if you want to renew your token. Then you can just say yes and run the authorization again. That should do it. 1 Quote Link to comment
NewDisplayName Posted April 30, 2019 Share Posted April 30, 2019 (edited) I found one issue. You have a rlcone unmount script, but its only set to run at array start. I always had the problem that parity check runs on restarts, i found out its because of "unclean shutdowns". So i copied your unmount script and set it to start on array and the other to stop of array.. now unraid restarts just fine without parity check. I have now rclone_unmount array start rclne_unmount array stop Edited April 30, 2019 by nuhll Quote Link to comment
francrouge Posted April 30, 2019 Share Posted April 30, 2019 Hi guys I need help. My script stop working frequently like 1 time per 2-3 days. and I need to unmount and remount or reboot sometimes. Do you have a fix for that ? Also When this happend i lose all my movies in plex like i got over 3000+ and now i got only 3 wish are in my local storage. I need to manually scan the library again.. Did anyone experience this problem ? thx all Quote Link to comment
NewDisplayName Posted May 1, 2019 Share Posted May 1, 2019 2 hours ago, francrouge said: Hi guys I need help. My script stop working frequently like 1 time per 2-3 days. and I need to unmount and remount or reboot sometimes. Do you have a fix for that ? Also When this happend i lose all my movies in plex like i got over 3000+ and now i got only 3 wish are in my local storage. I need to manually scan the library again.. Did anyone experience this problem ? thx all The question is why it stops, it never had happend to me, even i have not the most stable internet. We need logs. For the movie loss, i created a script some pages ago (look thru it) which scans every 1 min if mount still works, if not it disables some dockers (adjust to your needs). Quote Link to comment
DZMM Posted May 1, 2019 Author Share Posted May 1, 2019 12 hours ago, francrouge said: Hi guys I need help. My script stop working frequently like 1 time per 2-3 days. and I need to unmount and remount or reboot sometimes. Do you have a fix for that ? Also When this happend i lose all my movies in plex like i got over 3000+ and now i got only 3 wish are in my local storage. I need to manually scan the library again.. Did anyone experience this problem ? thx all My script should automatically remount if you set it up to run on a cron job say every 5 mins - that's what I use. The problem with Plex is if it scans while the mount is down, if your settings are 'wrong' it will assume the files no longer exist and will remove them from the library. To avoid this make sure your Plex library settings are set to not automatically remove missing content - Plex will then add a trash sign next to content it can no longer see, and will fix itself if that content becomes available again. Doing this does mean you need to manually empty trash every now and then when you know it's safe to do so i.e. when the mount is active. Quote Link to comment
francrouge Posted May 1, 2019 Share Posted May 1, 2019 (edited) 8 hours ago, DZMM said: My script should automatically remount if you set it up to run on a cron job say every 5 mins - that's what I use. The problem with Plex is if it scans while the mount is down, if your settings are 'wrong' it will assume the files no longer exist and will remove them from the library. To avoid this make sure your Plex library settings are set to not automatically remove missing content - Plex will then add a trash sign next to content it can no longer see, and will fix itself if that content becomes available again. Doing this does mean you need to manually empty trash every now and then when you know it's safe to do so i.e. when the mount is active. Hi can you tell me if this is right ? is it correct setting ? rclone mount --allow-other --buffer-size 512M --dir-cache-time 72h --drive-chunk-size 512M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs & Edited May 1, 2019 by francrouge Quote Link to comment
DZMM Posted May 1, 2019 Author Share Posted May 1, 2019 3 hours ago, francrouge said: Hi can you tell me if this is right ? is it correct setting ? Yes Quote Link to comment
francrouge Posted May 1, 2019 Share Posted May 1, 2019 YesWhat version of unraid you run ? Envoyé de mon Pixel 2 XL en utilisant Tapatalk Quote Link to comment
francrouge Posted May 1, 2019 Share Posted May 1, 2019 28 minutes ago, DZMM said: Yes if i check my logs i always got this until i manually reset Script Starting Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:50:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:55:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:58:05 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Quote Link to comment
DZMM Posted May 2, 2019 Author Share Posted May 2, 2019 7 hours ago, francrouge said: if i check my logs i always got this until i manually reset Script Starting Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:50:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:55:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:58:05 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Post your full script Quote Link to comment
Kaizac Posted May 2, 2019 Share Posted May 2, 2019 8 hours ago, francrouge said: if i check my logs i always got this until i manually reset Script Starting Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:50:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:50:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:55:01 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:55:01 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Script Starting Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt 01.05.2019 18:58:05 INFO: Exiting script already running. Script Finished Wed, 01 May 2019 18:58:05 -0400 Full logs for this script are available at /tmp/user.scripts/tmpScripts/test rclone/log.txt Are you hard rebooting/force rebooting your server? If your server doesn't have enough time to shut down array it won't run the unmount script. That's why I added the unmount script before my mount script at boot. Otherwise the "check" files won't be removed and the scripts see it as still running. Quote Link to comment
DZMM Posted May 2, 2019 Author Share Posted May 2, 2019 I'm going to try and do a review of the github content this weekend as I haven't updated it for over 4 months now. 1 Quote Link to comment
TheFreemancer Posted May 2, 2019 Share Posted May 2, 2019 Just wanted to report that I have upgraded my server to 8gb. That's the max this motherboard supports. I can now use the settings DZMM uses without any problem since I don't have any VM, all I have on my unraid server is Sabnzbd, Sonarr, Emby and rclone. Also my internet on my second home got an upgrade from 50mb to 240mb, so this improves how I can access my server. Now I have to ask I tried to ask this on Jriver forums but they had no idea. I have several complete UHD Blu-rays on my google drive, so that's the entire disc, not a single mkv file. If I try to play it from the cloud using anything on Windows 10 (Jriver, DVDFab Media Player) that would display the menus it takes almost 2-3 minutes to show the first menu and 1-2 minutes to show the next. After the movie starts playing is fine. So it's not a Jriver problem since DVDfab performes the same. I'm accessing my files using SMB to share the rclone mount. On the other side of this, the oppo 203 blu-ray player can show the first menu in 15 seconds! So I guess it has something to do with Windows screwing things up. All this is just to ask, I would like to try NFS from unraid to Windows. I got mount_rclone or mount_unionfs to show up as a drive on Windows 10 but inside the folder googleVFS there's nothing. As if NFS could not see the files. Is it possible to use NFS with rclone? Quote Link to comment
francrouge Posted May 2, 2019 Share Posted May 2, 2019 6 hours ago, DZMM said: Post your full script here you go thx mount.txt unmount.txt Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.