13 hours ago, Bolagnaise said:

I have 16GB and the buffer is set to 128mb, I did have the low ram issue and fixed it about 5 weeks ago, this feels vastly different. The SMB share becomes entirely inaccessible , whereas when I was running out of memory before I could always access the share and see that the mount had disconnected. Now the only way to access //tower is to unmount and then remount and the server becomes accessible over SMB again. 
 

To further the clue, it seems that running a plex scan does cause it to crash, but I do not have thumbnails turned on. The ram usage during a plex scan right now is only 20%. As I said, this issue has only started in the last 2 days or so and I never had unmount crashes before during scans. 

 

Edit: Ok i have done some more investigating, as everything was pointed to an SMB issue, and i seem to have fixed it. Here's what i think was the issue.

 

1. My network had reverted back to a public profile, by default network sharing is turned off on public and therefore SMB as well, but somehow it was still connecting. I switched back to a private network profile.

2. I enabled this setting https://forums.unraid.net/topic/77442-cannot-connect-to-unraid-shares-from-windows-10/

3. I went to windows credential manager and deleted all saved credentials for mapped drives.

 

That seems to have done the trick, everything is now incredibly much faster

 

 

EDIT EDIT: Problem still not fixed. At my wits end.

post your mount script please.  Have you tried running without dockers on and then turning on one at a time say every hour?

3 minutes ago, DZMM said:

post your mount script please.  Have you tried running without dockers on and then turning on one at a time say every hour?

#!/bin/bash

#######  Check if script is already running  ##########

if [[ -f "/mnt/user/appdata/other/rclone/rclone_mount_running" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Exiting script already running."

exit

else

touch /mnt/user/appdata/other/rclone/rclone_mount_running

fi

#######  End Check if script already running  ##########

#######  Start rclone gdrive mount  ##########

# check if gdrive mount already created

if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check rclone vfs already mounted."

else

echo "$(date "+%d.%m.%Y %T") INFO: mounting rclone vfs."

# create directories for rclone mount and unionfs mount

mkdir -p /mnt/user/appdata/other/rclone
mkdir -p /mnt/user/mount_rclone/google_vfs
mkdir -p /mnt/user/mount_unionfs/google_vfs
mkdir -p /mnt/user/rclone_upload/google_vfs

rclone mount --allow-other --buffer-size 128M --dir-cache-time 72h --drive-chunk-size 512M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs &

# check if mount successful

# slight pause to give mount time to finalise

sleep 5

if [[ -f "/mnt/user/mount_rclone/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check rclone gdrive vfs mount success."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: rclone gdrive vfs mount failed - please check for problems."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End rclone gdrive mount  ##########

#######  Start unionfs mount   ##########

if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs already mounted."

else

unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs

if [[ -f "/mnt/user/mount_unionfs/google_vfs/mountcheck" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Check successful, unionfs mounted."

else

echo "$(date "+%d.%m.%Y %T") CRITICAL: unionfs Remount failed."

rm /mnt/user/appdata/other/rclone/rclone_mount_running

exit

fi

fi

#######  End Mount unionfs   ##########

exit

 

 

 

 

I havent tested dockers yet.

@Bolagnaise I think you've got a rogue docker as script is spot on.  @yendi had similar problems that he resolved by doing some rebuilding, maybe he can help

1 minute ago, DZMM said:

@Bolagnaise I think you've got a rogue docker as script is spot on.  @yendi had similar problems that he resolved by doing some rebuilding, maybe he can help

Yeah I’m going to test 1 by 1 as suggested, I’m only running radarr, Ombi, letsencrypt, and Tautulli on the machine. My next step if none of that works is to stop everything and rebuild unraid into a new machine. I want to switch my plex server and everything over to unraid as it’s all on windows currently. Just need to do it.

@DZMM I just ran the cleanup script and it immediately killed the mount. Could that be an issue?

 

#!/bin/bash

#######  Check if script already running  ##########

if [[ -f "/mnt/user/appdata/other/rclone/rclone_cleanup" ]]; then

echo "$(date "+%d.%m.%Y %T") INFO: Exiting as script already running."

exit

else

touch /mnt/user/appdata/other/rclone/rclone_cleanup

fi

#######  End Check if script already running  ##########

################### Clean-up UnionFS Folder  #########################

echo "$(date "+%d.%m.%Y %T") INFO: starting unionfs cleanup."

find /mnt/user/mount_unionfs/google_vfs/.unionfs -name '*_HIDDEN~' | while read line; do
oldPath=${line#/mnt/user/mount_unionfs/google_vfs/.unionfs}
newPath=/mnt/user/mount_rclone/google_vfs${oldPath%_HIDDEN~}
rm "$newPath"
rm "$line"
done
find "/mnt/user/mount_unionfs/google_vfs/.unionfs" -mindepth 1 -type d -empty -delete

rm /mnt/user/appdata/other/rclone/rclone_cleanup

exit

What did the logs say? 

 

Do you have a lot of files in /mnt/user/mount_unionfs/google_vfs/.unionfs ?? Maybe the script can't cope with the cleanup.

 

I hate this part of unionfs - it looks like rclone union is really coming soon as work resumed last week.

Edited September 23, 20196 yr by DZMM

12 hours ago, DZMM said:

What did the logs say? 

 

Do you have a lot of files in /mnt/user/mount_unionfs/google_vfs/.unionfs ?? Maybe the script can't cope with the cleanup.

 

I gate this part of unionfs - it looks like rclone union is really coming soon as work resumed last week.

Logs said input/output error. But now I’m doing cleanup scripts and it’s working so IDK. When clone union is released, will you write up a new tutorial. I will love you long time if you do 😘

@DZMM im getting these errors appear in the mount log now. 

 

2019/09/23 23:06:40 ERROR : Movies/3 Lives (2019)/3.Lives.2019.WEBDL-1080p.mp4: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes

 

so i added

 

''--vfs-cache-mode writes''

 

 to the mount script (https://github.com/rclone/rclone/issues/961)

 

I got 

 

2019/09/23 23:12:31 INFO : Cleaned the cache: objects 1 (was 4), total size 0 (was 0)

 

i have no idea what ive done, but the error has gone away.

 

No idea if its fixed my error yet.

 

Edited September 23, 20196 yr by Bolagnaise

@Bolagnaise not sure what's going on - are you sure your dockers are writing to /mount_unionfs and not /mount_rclone?  The mentions of vfs-cache-mode writes seems to indicate something is - writing direct to the mount without using rclone upload is risky, as it doesn't recover if something goes wrong

26 minutes ago, DZMM said:

@Bolagnaise not sure what's going on - are you sure your dockers are writing to /mount_unionfs and not /mount_rclone?  The mentions of vfs-cache-mode writes seems to indicate something is - writing direct to the mount without using rclone upload is risky, as it doesn't recover if something goes wrong

Nothing is pointed to mount_rclone, everything is using mount_unionfs. Ive just done a brand new unraid install as well on new hardware.

 

this seems like the issue as you said before https://forums.unraid.net/bug-reports/stable-releases/67x-very-slow-array-concurrent-performance-r605/?do=findComment&comment=5488

 

Edited September 23, 20196 yr by Bolagnaise

Everything has been stable for the past 6 hours, no drops.

 

Things i have done.

1. Downgrade Unraid to 6.6.7

2. Switch to Rclone non beta

3. add --vfs-cache-mode writes to my mount script

i read that when using VFS cache, that the --buffer-size should be set too 0, is that correct?

 

8 hours ago, Bolagnaise said:

Everything has been stable for the past 6 hours, no drops.

 

Things i have done.

1. Downgrade Unraid to 6.6.7

2. Switch to Rclone non beta

3. add --vfs-cache-mode writes to my mount script

i read that when using VFS cache, that the --buffer-size should be set too 0, is that correct?

 

I need to re-review adding --vfs-cache-mode writes, as a quick glance now makes sense and it might help with when I occasionally write direct to the mount.

 

I don't think buffer-size should be set to 0, but again I'll research as I haven't touched my settings for almost a year.

2 minutes ago, DZMM said:

I need to re-review adding --vfs-cache-mode writes, as a quick glance now makes sense and it might help with when I occasionally write direct to the mount.

 

I don't think buffer-size should be set to 0, but again I'll research as I haven't touched my settings for almost a year.

No worries man, that the way it is, everything works completely fine...until it doesnt. You have done everyone a service so I don’t mind a few late nights troubleshooting, your literally saving me money with this script. 
 

anyway, 12 hours uptime now, zero dismounts and I successfully moved my everything onto a brand new unraid build in new hardware. looks like it’s fixed.

@DZMM So I tried running the upload script last night and the mount immediately disconnected and was throwing errors in the mount log saying it couldn’t pull the api key, which made me realise exactly what the original issue was, I had BW set to 9000 in the script, and I use a 5G router to perform the uploads but it only has a 4G uplink speed of around 45mbps. So basically everytime the script ran it would cause my router to crash and the mount would disconnect, as soon as I stopped the upload/rebooted it would work again.

 

maybe a warning to everyone to change the BW limit as a must. @jamesac2 only has a 10Mbps upload so maybe that’s why he’s also getting disconnects. 

Edited September 25, 20196 yr by Bolagnaise

doesnt really make sense, you cant transfer faster then your line is, and if you transfer at full speed (thats what happens when u click and download something) then u might get lag but no disconnect.

 

And which router disconnects when you use ur line? (i also have 2 4g "lines")

 

I use since years my full bw... 

Edited September 25, 20196 yr by nuhll

2 hours ago, Bolagnaise said:

@DZMM So I tried running the upload script last night and the mount immediately disconnected and was throwing errors in the mount log saying it couldn’t pull the api key, which made me realise exactly what the original issue was, I had BW set to 9000 in the script, and I use a 5G router to perform the uploads but it only has a 4G uplink speed of around 45mbps. So basically everytime the script ran it would cause my router to crash and the mount would disconnect, as soon as I stopped the upload/rebooted it would work again.

 

maybe a warning to everyone to change the BW limit as a must. @jamesac2 only has a 10Mbps upload so maybe that’s why he’s also getting disconnects. 

How are you running the scripts? As "run now" or "run in background"?

4 hours ago, Kaizac said:

How are you running the scripts? As "run now" or "run in background"?

Background using cron jobs.

1 minute ago, Bolagnaise said:

Background using cron jobs.

Then I think the issue is with your router being 5 or 4G. It's sort of having wifi non stop which just isn't stable. So when it drops, you also lose your mount.

I also have a not stable connection, if one of the lines drop, my mounts automatic remount?

41 minutes ago, Kaizac said:

Then I think the issue is with your router being 5 or 4G. It's sort of having wifi non stop which just isn't stable. So when it drops, you also lose your mount.

Yeah it’s defiantly related to the router, but as I said, I just reduced the BW limit to 3000 and it no longer drops anymore. 

1 hour ago, Bolagnaise said:

Yeah it’s defiantly related to the router, but as I said, I just reduced the BW limit to 3000 and it no longer drops anymore. 

Hopefully it stays that way for you. Could be that your ISP or router just can't handle the full load/demands. Especially when you also start streaming while uploading.

So now i also found the issue where i can only access local files, and remote load forever (via SMB) and in Plex. 

 

Are there any new settings i could adjust? Or is it just unraid bug?

I had to format and reinstall

I had this working great for a few months, but hten the power went out and my unraid server turned off unexpectedly.. since then I haven't been able to get this thing to work well.

 

I can see and view the files, but then when I use plex to watch a movie, after about 3 minutes it just freezes up. I can see the directories of the files, but can't actually view and of the files. Any idea what is going on? I tried to find some logs but I am having a really hard time finding anything with rclone logs.

 

On the main unraid page, the CPU's look to be maxed out.. but opening up a terminal and running htop, they are at about 10%. Not sure if this has anything to do with it.

 

 

 

My scripts are taken directly from the github. 

Edited October 5, 20196 yr by mestep

16 hours ago, mestep said:

I had this working great for a few months, but hten the power went out and my unraid server turned off unexpectedly.. since then I haven't been able to get this thing to work well.

 

I can see and view the files, but then when I use plex to watch a movie, after about 3 minutes it just freezes up. I can see the directories of the files, but can't actually view and of the files. Any idea what is going on? I tried to find some logs but I am having a really hard time finding anything with rclone logs.

 

On the main unraid page, the CPU's look to be maxed out.. but opening up a terminal and running htop, they are at about 10%. Not sure if this has anything to do with it.

 

 

 

My scripts are taken directly from the github. 

What does the mount log say?