On 10/22/2021 at 10:39 AM, Michel Amberg said:

 

So the mount script as I assumed fails unless there is a json file specifically called sa_gdrive.json. So I renamed one of the _uploadX.json to that and it mounted. But is this the right way to do it?

Only for the first run if you've set up your rclone config that way - for the first run you just need to use the value in the script that's in your rclone config.

 

After the first run the script rotates the entry used in rclone config.

Edited November 2, 20214 yr by DZMM

58 minutes ago, DZMM said:

Only for the first run if you've set up your rclone config that way - for the first run you just need to use the value in the script that's in your rclone config.

 

After the first run the script rotates then entry used in rclone config.

 

I understand I think this can be a bit more clear in the guide since this really confused me. How do I know it actually works? I checked the rclone config while the upload script is running and it seems to keep this line the same?:

service_account_file = /mnt/user/appdata/other/rclone/service_accounts/sa_gdrive.json

 

Edited October 22, 20214 yr by Michel Amberg

So I got the service accounts to work, but I noticed some files was 0KB in the tv folders since the initial upload. So I readded them to the mergerfs folder locally and hoped it would upload them again. It hit the google cloud limit and stopped uploading now something is constantly pegging my upload to 100% and it is not any of these scripts? if I try to download something from the mergerfs folder it is extremely slow like 300KB/s and playback in plex is impossible. What has gone wrong here?

28 minutes ago, Michel Amberg said:

So I got the service accounts to work, but I noticed some files was 0KB in the tv folders since the initial upload. So I readded them to the mergerfs folder locally and hoped it would upload them again. It hit the google cloud limit and stopped uploading now something is constantly pegging my upload to 100% and it is not any of these scripts? if I try to download something from the mergerfs folder it is extremely slow like 300KB/s and playback in plex is impossible. What has gone wrong here?

Logs

On 10/23/2021 at 2:58 PM, DZMM said:

Logs

I think I figured it out it pinned the my internet completely so if I tried to download something it was just not enough bandwidth. Now that all files are upload everthing works again. However I have one question I did set bwlimits in the upload script but it does not seem to honor it with service accounts? Also downloading files maxes out the internet connection is that what you want really? it kind makes other internet tasks sluggish? Is there a way to limit the download speed also? 

On 10/25/2021 at 9:33 PM, Michel Amberg said:

I think I figured it out it pinned the my internet completely so if I tried to download something it was just not enough bandwidth. Now that all files are upload everthing works again. However I have one question I did set bwlimits in the upload script but it does not seem to honor it with service accounts? Also downloading files maxes out the internet connection is that what you want really? it kind makes other internet tasks sluggish? Is there a way to limit the download speed also? 

Playing files from the cloud can be bandwidth "intensive" depending on how big the file is you are trying to play, how many concurrent streams etc Vs your line speed.  I have a 900/120 connection and re download my usage (at peak about 5-6 streams) never becomes a concern for me, although in the evenings I do make sure any other activity is low so that my friends get a good experience.  Re Upload I do have to manage to balance my remote plex demand, seeding, uploading to gdrive etc requirements are all met.

 

You should set bwlimits that are appropriate for your connection - both upload and download.

Hi all,

 

Im trying to setup services accounts using the instructions found on the Autoclone git but cant for life of me even complete step 1. Im no programmer and have relied on the various guides on setting up different functions for unraid which have been a tremendous help given my lack of programming knowledge. I've installed python3 via nerdpack but where do I go from there? I'd appreciate any help anyone can provide.

Hello, thank you very much for creating this script, it's great. But I having some problems :). I have deluge/sonarr combo and it points to mergefs (I have only 1 team drive mount), deluge downloads to mergefs/_d and after that sonarr moves completed files to mergefs/shows. But during download unRAID gets high CPU load (high iowait) and it becomes impossible to watch shows through share. Even without watching shows dashboard shows 70-80 % CPU usage. Are there any ways to combat this behavior? 

Edited October 30, 20214 yr by Heckfy
Misspelling

Is the script on page 1 still valid?

I have not got the time to go through 104 pages sorry

On 10/31/2021 at 12:19 AM, Heckfy said:

Hello, thank you very much for creating this script, it's great. But I having some problems :). I have deluge/sonarr combo and it points to mergefs (I have only 1 team drive mount), deluge downloads to mergefs/_d and after that sonarr moves completed files to mergefs/shows. But during download unRAID gets high CPU load (high iowait) and it becomes impossible to watch shows through share. Even without watching shows dashboard shows 70-80 % CPU usage. Are there any ways to combat this behavior? 

not sure what's going on - new files should be stored locally until uploaded, so any performance issues shouldn't be down to rclone/mergerfs.

19 hours ago, OsoPolar said:

Is the script on page 1 still valid?

I have not got the time to go through 104 pages sorry

Did you at least bother to read the first sentence of the first post?

4 hours ago, DZMM said:

Did you at least bother to read the first sentence of the first post?

I have now cheers i don’t tend to look at text above graphics.

But good point

On 11/2/2021 at 12:48 PM, DZMM said:

not sure what's going on - new files should be stored locally until uploaded, so any performance issues shouldn't be down to rclone/mergerfs.

It looks like I made an error while setting upload script and didn't add download folder to exceptions in upload script, maybe that's why it stressed system so much. 

Anyone recently run into permission issues? I've been running solid for over a year and now Sonarr is having issues importing and is having "Access to the path is denied." issues. Normally I run the Docker Safe New Perms and it fixes small issues like that, but I haven't had much luck. Wanted to see if anyone had a similar issue.

41 minutes ago, Roudy said:

Anyone recently run into permission issues? I've been running solid for over a year and now Sonarr is having issues importing and is having "Access to the path is denied." issues. Normally I run the Docker Safe New Perms and it fixes small issues like that, but I haven't had much luck. Wanted to see if anyone had a similar issue.

no problems here.

10 hours ago, DZMM said:

no problems here.

I got it working. Had to add --dir-perms 0777 and --file-perms 0776 to the rclone mount script and reboot. Weird I've never had an issue before.

 

**Correction, you only need to add -umask 000**

Edited November 6, 20214 yr by Roudy
Correcting information

7 hours ago, Roudy said:

I got it working. Had to add --dir-perms 0777 and --file-perms 0776 to the rclone mount script and reboot. Weird I've never had an issue before.

 

I'm having the same problem, "Group" and "Other" are both missing write permissions. Started with update to rc2. Unfortunately this didn't work for me. I assume you added these arguments to the rclone mount command?

 

This feels like a umask issue...

Edited November 5, 20214 yr by T0rqueWr3nch

1 hour ago, T0rqueWr3nch said:

I assume you added these arguments to the rclone mount command?

 

Yes, I added them in the "# create rclone mount" section. 

 

1 hour ago, T0rqueWr3nch said:

This feels like a umask issue...

 

**Correction, you only need to add the below to the "# create rclone mount" section. **

 

    --umask 000 \

 

 

Edited November 6, 20214 yr by Roudy
Removed unneeded lines.

1 hour ago, Roudy said:

 

Yes, I added them in the "# create rclone mount" section. 

 

 

Try adding --umask 000 as well. I had that in there from a long time ago and just realized it's not in the original script. So try to add the section below and see if it fixes it for you. I also had to reboot because unmounting and remounting didn't seem to solve the problem.

 

    --umask 000 \
    --dir-perms 0777 \
    --file-perms 0776 \

I seem to have the same issue with 6.10 RC2, tried with the above settings. Really weird.

 

UPDATE: Seems the UMASK needed to be corrected on the docker containers as well.

Edited November 6, 20214 yr by Thel1988

1 hour ago, Thel1988 said:

UPDATE: Seems the UMASK needed to be corrected on the docker containers as well.

 

I didn't have to do that for mine. Were you having the same issue until you updated the container? 

3 hours ago, Roudy said:

**Correction, you only need to add the below to the "# create rclone mount" section. **

 

    --umask 000 \

 

That's what I ended up doing right after I posted that! Glad to have the independent verification. Thanks!

 

Honestly, I'm not sure why this hasn't always been a problem- rclone says its default umask is "2", but other sources say this is really 0022, which is what I'm seeing. 

 

I wonder what changed with rc2. Could it be related to this "Fixed issue in User Share file system where permissions were not being honored."? Unraid OS version 6.10.0-rc2 available - Prereleases - Unraid

 

While troubleshooting this, I also used this as an opportunity to update how I mount rclone. I passed the uid and gid arguments to mount as "nobody" and "users" (UID 99; GID 100). I might go back to just mounting as root again. Thoughts?

2 hours ago, T0rqueWr3nch said:

While troubleshooting this, I also used this as an opportunity to update how I mount rclone. I passed the uid and gid arguments to mount as "nobody" and "users" (UID 99; GID 100). I might go back to just mounting as root again. Thoughts?

 

I tried the same thing while troubleshooting. From what I researched, the --allow-other covers the base for other users using the share. I was thinking of making it always use the nobody:users permissions though. I haven't thought through if it would cause any issues I may be overlooking. Have you had any problems with it since running it that way?

8 hours ago, Roudy said:

 

I didn't have to do that for mine. Were you having the same issue until you updated the container? 

Yes, the UMASK was set to 002 for that specific docker container I couldn't get working.

But yes I think 

 

7 hours ago, T0rqueWr3nch said:

I wonder what changed with rc2. Could it be related to this "Fixed issue in User Share file system where permissions were not being honored."? Unraid OS version 6.10.0-rc2 available - Prereleases - Unraid

Yes I actually think that this is the cause of the issue, that it is actually never been working as intended.

Edited November 6, 20214 yr by Thel1988

17 hours ago, Roudy said:

Have you had any problems with it since running it that way?

Nope, I haven't noted any ill effects so far.

 

Unraid's umask seems to be set to wide open (checked by running umask -S which returns 000), so I really don't understand what could've changed. Reviewing this stuff does make me wonder again if I should be doing more for ransomware protection...

On 11/6/2021 at 1:34 AM, Roudy said:

 

Yes, I added them in the "# create rclone mount" section. 

 

 

**Correction, you only need to add the below to the "# create rclone mount" section. **

 

    --umask 000 \

 

 

Could you tell me where it is exactly should be placed?