I'm having a problem where my /mnt/user directory is unmounting. I have looked through the logs and can not find where or what is causing this problem. I have tried several large file transfers, running multiple vm's, and activating the mover. None of these seems to be the cause. The directory just randomly drops. The only way I have found to fix the problem is to reboot the system. I have attached to log file. What else can I try to fix the problem.

tower-diagnostics-20200125-1416.zip

Power down and plug your USB into your PC and run chkdsk.  Fix any errors if found.  Also install your USB into a USB 2 port on your server.  I have had a USB port go wonky and changed it to my other USB port and it fixed my issues

There are some out of memory errors and a fuse errors a few minutes after that, that would be when /mnt/user stopped working, so it could be related, fix the out of memory errors first and see if it helps, if not post new diags.

I did the chkdsk and non errors reported.

The memory is surprising as I try to keep about 1/2 free unless I'm playing games on the VM. I'll give that a look. thanks.

I did a memory test a couple of days ago and it reported no errors. I also shutdown several docker containers and stopped a VM from starting. I also placed the thumb drive into a different USB port. Finally did tried to update the BIOS, but found out it was already up to date. I still am having the problem of the /mnt/user directory unmounting. I looked through the logs and did not see any errors with memory. What else can I look at or try.

tower-diagnostics-20200129-0823.zip

Edited January 29, 20206 yr by ralieghnet

Why do you have 50G allocated for docker image? Have you had problems filling it? Also Fix Common Problems says a number of your dockers are missing templates. Any idea what that is about?

13 minutes ago, ralieghnet said:

I did a memory test a couple of days ago and it reported no errors.

I mentioned running out of memory, not memory errors, but these last diags don't have that and still there's the same error:

Jan 29 00:31:46 Tower shfs: shfs: ../lib/fuse.c:1450: unlink_node: Assertion `node->nlookup > 1' failed.

I have no idea what this error means, don't remember seeing it before, maybe @limetechknows what this is about?

For the docker image. A while ago I installed a container that was hungry in its size. Since then I have removed the container as it was affecting general performance. I also found that Zoneminder was getting rather large until I made some changes to the container. I just haven't scaled it back down.

 

Just found this post.

They report the same problem. However, I don't see what the fix was.

 

Yes, same error, another thing you could try would be running the server in safe mode to rule out any plugin.

I'll give that a try when I get home. I can't make that happen remotely.

2 hours ago, johnnie.black said:

I mentioned running out of memory, not memory errors, but these last diags don't have that and still there's the same error:

Jan 29 00:31:46 Tower shfs: shfs: ../lib/fuse.c:1450: unlink_node: Assertion `node->nlookup > 1' failed.

I have no idea what this error means, don't remember seeing it before, maybe @limetechknows what this is about?

It means somehow it tried to delete a file that was already deleted.  Shouldn't happen.

I suggest reboot in Safe Mode to rule out app interference.

BTW: noticed 'nginx' is installed as a plugin - @Squid wouldn't this interfere with nginx natively installed?

I'm looking at my system. I don't see nginx installed as a plugin. I've looked in the plugins tab and inside nerd pack. I do have nginx installed inside a docker. Where else can I look?

19 hours ago, limetech said:

It means somehow it tried to delete a file that was already deleted.  Shouldn't happen.

I suggest reboot in Safe Mode to rule out app interference.

BTW: noticed 'nginx' is installed as a plugin - @Squid wouldn't this interfere with nginx natively installed?

Don't see it as a plugin.  There is an nginx container running which shouldn't have any ill effects.

So for I have not had any more memory errors or dismounting of the directory. I took the suggestion of Gragorg and moved the thumb drive to another USB port. I also installed from Nerd Pack sshfs-fuse.

Last night was the first time in a while the problem happened again. I completely forgot to get a log dump. The only thing that I had running external of the system was an SSH session from work that I forgot to close. But, that shouldn't of caused the /mnt/user to stop working.

 

I did however notice on one of my desk a couple of days ago one of my disks had a "Current Pending sector" problem. Then yesterday we had a power outage and the system preformed like it should of and shutdown before the ups died. However, when the system came back online a new error on the same disk now read "Offline uncorrectable." I already have a new disk on its way. I don't think these two are related as the disk in question is empty and only has about 100M of data. All of which I have no problem loosing.

 

Any more suggestion to fixing the problem. I have already upgraded to the newest stable release of 6.8.2. That did not fix the problem.

On 2/3/2020 at 2:56 PM, ralieghnet said:

I also installed from Nerd Pack sshfs-fuse.

Why?

 

  

10 minutes ago, ralieghnet said:

I don't think these two are related

It is possible for disk or filesystem problems to also break user shares. Is the problem disk showing as Unmountable?

The disk is mounted and I can look through the contents with both the GUI and CLI. Here is a screen shot of the disk states. The thing that surprises me is the SMART test short and extended completed without error. Granted this is a WD Blue hard drive. I have a WD Red arriving today.

I've upgraded the disk that was failing to a newer disk. and still the system /mnt/user is unmounting.

One thing I'm noticing is the memory. When a fresh reboot occurs the required amount of RAM is used. Then half of the available RAM is used as a cache. The rest remains unused. But, when I checked while at work the amount of available RAM was almost gone and was consumed by the cache portion. What might be causing this.

 

Any suggestions.

tower-diagnostics-20200212-1744.zip

Edited February 13, 20206 yr by ralieghnet

Your cache pool isn't mounting, that is likely the cause of your user shares problem.

 

 

21 minutes ago, ralieghnet said:

One thing I'm noticing is the memory. When a fresh reboot occurs the required amount of RAM is used. Then half of the available RAM is used as a cache. The rest remains unused. But, when I checked while at work the amount of available RAM was almost gone and was consumed by the cache portion. What might be causing this.

Where are you seeing this? It is normal for Linux to use all available memory for I/O buffer. Probably this is what you are seeing.

 

I'm pretty sure my cache pool is mounting. My windows vm's run from the cache pool. I look using the top command and under stats page for a quicker overview. My other Ubuntu system at work never fills up the memory to capacity. This is why I was wondering if that might be the problem.

Did you try booting in safe mode?

8 hours ago, ralieghnet said:

I'm pretty sure my cache pool is mounting

Sorry, you're right.

 

On 2/3/2020 at 2:56 PM, ralieghnet said:

I also installed from Nerd Pack sshfs-fuse.

On 2/10/2020 at 2:32 PM, trurl said:

Why?

 

4 hours ago, johnnie.black said:

Did you try booting in safe mode?

Since the upgrade to 6.8.3 I've had no problems with the /mnt/user unmounting. Thanks for all the help it was appreciated.