DZMM Posted October 22, 2021 Author Share Posted October 22, 2021 (edited) On 10/22/2021 at 10:39 AM, Michel Amberg said: So the mount script as I assumed fails unless there is a json file specifically called sa_gdrive.json. So I renamed one of the _uploadX.json to that and it mounted. But is this the right way to do it? Only for the first run if you've set up your rclone config that way - for the first run you just need to use the value in the script that's in your rclone config. After the first run the script rotates the entry used in rclone config. Edited November 2, 2021 by DZMM Quote Link to comment
Michel Amberg Posted October 22, 2021 Share Posted October 22, 2021 (edited) 58 minutes ago, DZMM said: Only for the first run if you've set up your rclone config that way - for the first run you just need to use the value in the script that's in your rclone config. After the first run the script rotates then entry used in rclone config. I understand I think this can be a bit more clear in the guide since this really confused me. How do I know it actually works? I checked the rclone config while the upload script is running and it seems to keep this line the same?: service_account_file = /mnt/user/appdata/other/rclone/service_accounts/sa_gdrive.json Edited October 22, 2021 by Michel Amberg Quote Link to comment
Michel Amberg Posted October 23, 2021 Share Posted October 23, 2021 So I got the service accounts to work, but I noticed some files was 0KB in the tv folders since the initial upload. So I readded them to the mergerfs folder locally and hoped it would upload them again. It hit the google cloud limit and stopped uploading now something is constantly pegging my upload to 100% and it is not any of these scripts? if I try to download something from the mergerfs folder it is extremely slow like 300KB/s and playback in plex is impossible. What has gone wrong here? Quote Link to comment
DZMM Posted October 23, 2021 Author Share Posted October 23, 2021 28 minutes ago, Michel Amberg said: So I got the service accounts to work, but I noticed some files was 0KB in the tv folders since the initial upload. So I readded them to the mergerfs folder locally and hoped it would upload them again. It hit the google cloud limit and stopped uploading now something is constantly pegging my upload to 100% and it is not any of these scripts? if I try to download something from the mergerfs folder it is extremely slow like 300KB/s and playback in plex is impossible. What has gone wrong here? Logs Quote Link to comment
Michel Amberg Posted October 25, 2021 Share Posted October 25, 2021 On 10/23/2021 at 2:58 PM, DZMM said: Logs I think I figured it out it pinned the my internet completely so if I tried to download something it was just not enough bandwidth. Now that all files are upload everthing works again. However I have one question I did set bwlimits in the upload script but it does not seem to honor it with service accounts? Also downloading files maxes out the internet connection is that what you want really? it kind makes other internet tasks sluggish? Is there a way to limit the download speed also? Quote Link to comment
DZMM Posted October 27, 2021 Author Share Posted October 27, 2021 On 10/25/2021 at 9:33 PM, Michel Amberg said: I think I figured it out it pinned the my internet completely so if I tried to download something it was just not enough bandwidth. Now that all files are upload everthing works again. However I have one question I did set bwlimits in the upload script but it does not seem to honor it with service accounts? Also downloading files maxes out the internet connection is that what you want really? it kind makes other internet tasks sluggish? Is there a way to limit the download speed also? Playing files from the cloud can be bandwidth "intensive" depending on how big the file is you are trying to play, how many concurrent streams etc Vs your line speed. I have a 900/120 connection and re download my usage (at peak about 5-6 streams) never becomes a concern for me, although in the evenings I do make sure any other activity is low so that my friends get a good experience. Re Upload I do have to manage to balance my remote plex demand, seeding, uploading to gdrive etc requirements are all met. You should set bwlimits that are appropriate for your connection - both upload and download. Quote Link to comment
stefan416 Posted October 30, 2021 Share Posted October 30, 2021 Hi all, Im trying to setup services accounts using the instructions found on the Autoclone git but cant for life of me even complete step 1. Im no programmer and have relied on the various guides on setting up different functions for unraid which have been a tremendous help given my lack of programming knowledge. I've installed python3 via nerdpack but where do I go from there? I'd appreciate any help anyone can provide. Quote Link to comment
Heckfy Posted October 30, 2021 Share Posted October 30, 2021 (edited) Hello, thank you very much for creating this script, it's great. But I having some problems :). I have deluge/sonarr combo and it points to mergefs (I have only 1 team drive mount), deluge downloads to mergefs/_d and after that sonarr moves completed files to mergefs/shows. But during download unRAID gets high CPU load (high iowait) and it becomes impossible to watch shows through share. Even without watching shows dashboard shows 70-80 % CPU usage. Are there any ways to combat this behavior? Edited October 30, 2021 by Heckfy Misspelling Quote Link to comment
OsoPolar Posted November 1, 2021 Share Posted November 1, 2021 Is the script on page 1 still valid? I have not got the time to go through 104 pages sorry Quote Link to comment
DZMM Posted November 2, 2021 Author Share Posted November 2, 2021 On 10/31/2021 at 12:19 AM, Heckfy said: Hello, thank you very much for creating this script, it's great. But I having some problems :). I have deluge/sonarr combo and it points to mergefs (I have only 1 team drive mount), deluge downloads to mergefs/_d and after that sonarr moves completed files to mergefs/shows. But during download unRAID gets high CPU load (high iowait) and it becomes impossible to watch shows through share. Even without watching shows dashboard shows 70-80 % CPU usage. Are there any ways to combat this behavior? not sure what's going on - new files should be stored locally until uploaded, so any performance issues shouldn't be down to rclone/mergerfs. Quote Link to comment
DZMM Posted November 2, 2021 Author Share Posted November 2, 2021 19 hours ago, OsoPolar said: Is the script on page 1 still valid? I have not got the time to go through 104 pages sorry Did you at least bother to read the first sentence of the first post? Quote Link to comment
OsoPolar Posted November 2, 2021 Share Posted November 2, 2021 4 hours ago, DZMM said: Did you at least bother to read the first sentence of the first post? I have now cheers i don’t tend to look at text above graphics. But good point Quote Link to comment
Heckfy Posted November 4, 2021 Share Posted November 4, 2021 On 11/2/2021 at 12:48 PM, DZMM said: not sure what's going on - new files should be stored locally until uploaded, so any performance issues shouldn't be down to rclone/mergerfs. It looks like I made an error while setting upload script and didn't add download folder to exceptions in upload script, maybe that's why it stressed system so much. Quote Link to comment
Roudy Posted November 5, 2021 Share Posted November 5, 2021 Anyone recently run into permission issues? I've been running solid for over a year and now Sonarr is having issues importing and is having "Access to the path is denied." issues. Normally I run the Docker Safe New Perms and it fixes small issues like that, but I haven't had much luck. Wanted to see if anyone had a similar issue. Quote Link to comment
DZMM Posted November 5, 2021 Author Share Posted November 5, 2021 41 minutes ago, Roudy said: Anyone recently run into permission issues? I've been running solid for over a year and now Sonarr is having issues importing and is having "Access to the path is denied." issues. Normally I run the Docker Safe New Perms and it fixes small issues like that, but I haven't had much luck. Wanted to see if anyone had a similar issue. no problems here. Quote Link to comment
Roudy Posted November 5, 2021 Share Posted November 5, 2021 (edited) 10 hours ago, DZMM said: no problems here. I got it working. Had to add --dir-perms 0777 and --file-perms 0776 to the rclone mount script and reboot. Weird I've never had an issue before. **Correction, you only need to add -umask 000** Edited November 6, 2021 by Roudy Correcting information 1 Quote Link to comment
T0rqueWr3nch Posted November 5, 2021 Share Posted November 5, 2021 (edited) 7 hours ago, Roudy said: I got it working. Had to add --dir-perms 0777 and --file-perms 0776 to the rclone mount script and reboot. Weird I've never had an issue before. I'm having the same problem, "Group" and "Other" are both missing write permissions. Started with update to rc2. Unfortunately this didn't work for me. I assume you added these arguments to the rclone mount command? This feels like a umask issue... Edited November 5, 2021 by T0rqueWr3nch Quote Link to comment
Roudy Posted November 5, 2021 Share Posted November 5, 2021 (edited) 1 hour ago, T0rqueWr3nch said: I assume you added these arguments to the rclone mount command? Yes, I added them in the "# create rclone mount" section. 1 hour ago, T0rqueWr3nch said: This feels like a umask issue... **Correction, you only need to add the below to the "# create rclone mount" section. ** --umask 000 \ Edited November 6, 2021 by Roudy Removed unneeded lines. Quote Link to comment
Thel1988 Posted November 5, 2021 Share Posted November 5, 2021 (edited) 1 hour ago, Roudy said: Yes, I added them in the "# create rclone mount" section. Try adding --umask 000 as well. I had that in there from a long time ago and just realized it's not in the original script. So try to add the section below and see if it fixes it for you. I also had to reboot because unmounting and remounting didn't seem to solve the problem. --umask 000 \ --dir-perms 0777 \ --file-perms 0776 \ I seem to have the same issue with 6.10 RC2, tried with the above settings. Really weird. UPDATE: Seems the UMASK needed to be corrected on the docker containers as well. Edited November 6, 2021 by Thel1988 Quote Link to comment
Roudy Posted November 6, 2021 Share Posted November 6, 2021 1 hour ago, Thel1988 said: UPDATE: Seems the UMASK needed to be corrected on the docker containers as well. I didn't have to do that for mine. Were you having the same issue until you updated the container? Quote Link to comment
T0rqueWr3nch Posted November 6, 2021 Share Posted November 6, 2021 3 hours ago, Roudy said: **Correction, you only need to add the below to the "# create rclone mount" section. ** --umask 000 \ That's what I ended up doing right after I posted that! Glad to have the independent verification. Thanks! Honestly, I'm not sure why this hasn't always been a problem- rclone says its default umask is "2", but other sources say this is really 0022, which is what I'm seeing. I wonder what changed with rc2. Could it be related to this "Fixed issue in User Share file system where permissions were not being honored."? Unraid OS version 6.10.0-rc2 available - Prereleases - Unraid While troubleshooting this, I also used this as an opportunity to update how I mount rclone. I passed the uid and gid arguments to mount as "nobody" and "users" (UID 99; GID 100). I might go back to just mounting as root again. Thoughts? Quote Link to comment
Roudy Posted November 6, 2021 Share Posted November 6, 2021 2 hours ago, T0rqueWr3nch said: While troubleshooting this, I also used this as an opportunity to update how I mount rclone. I passed the uid and gid arguments to mount as "nobody" and "users" (UID 99; GID 100). I might go back to just mounting as root again. Thoughts? I tried the same thing while troubleshooting. From what I researched, the --allow-other covers the base for other users using the share. I was thinking of making it always use the nobody:users permissions though. I haven't thought through if it would cause any issues I may be overlooking. Have you had any problems with it since running it that way? Quote Link to comment
Thel1988 Posted November 6, 2021 Share Posted November 6, 2021 (edited) 8 hours ago, Roudy said: I didn't have to do that for mine. Were you having the same issue until you updated the container? Yes, the UMASK was set to 002 for that specific docker container I couldn't get working. But yes I think 7 hours ago, T0rqueWr3nch said: I wonder what changed with rc2. Could it be related to this "Fixed issue in User Share file system where permissions were not being honored."? Unraid OS version 6.10.0-rc2 available - Prereleases - Unraid Yes I actually think that this is the cause of the issue, that it is actually never been working as intended. Edited November 6, 2021 by Thel1988 1 Quote Link to comment
T0rqueWr3nch Posted November 6, 2021 Share Posted November 6, 2021 17 hours ago, Roudy said: Have you had any problems with it since running it that way? Nope, I haven't noted any ill effects so far. Unraid's umask seems to be set to wide open (checked by running umask -S which returns 000), so I really don't understand what could've changed. Reviewing this stuff does make me wonder again if I should be doing more for ransomware protection... Quote Link to comment
Heckfy Posted November 8, 2021 Share Posted November 8, 2021 On 11/6/2021 at 1:34 AM, Roudy said: Yes, I added them in the "# create rclone mount" section. **Correction, you only need to add the below to the "# create rclone mount" section. ** --umask 000 \ Could you tell me where it is exactly should be placed? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.