[6.8.3] shfs error results in lost /mnt/user

@primeval_god I have installed netdata, but I have never used it and it is completely unknown to me. I'm looking for how to create the configuration you write about, but I can't find anything sensible.

Can you help? Any link? The netdata documentation is an ocean.

17 hours ago, jaclas said:

@primeval_god I have installed netdata, but I have never used it and it is completely unknown to me. I'm looking for how to create the configuration you write about, but I can't find anything sensible.

Can you help? Any link? The netdata documentation is an ocean.

You will need to obtain a copy of the apps_groups.conf file from the netdata container. You can use the docker cp command for this. From the command line, in the appdata folder where you store your netdata configuration, run 

docker cp netdata:/etc/netdata/orig/apps_groups.conf apps_groups.conf

assuming your container is named netdata. Add the line 

shfs: shfs

to the end of the apps_groups.conf file. Then all you need to do is bind mount the file back into the container at the path /etc/netdata/apps_groups.conf (add the entry to the docker template, make sure both the host and container paths are to the file itself).

Edited May 8 by primeval_god

On 5/8/2024 at 2:39 AM, primeval_god said:

You will need to obtain a copy of the apps_groups.conf file from the netdata container. You can use the docker cp command for this. From the command line, in the appdata folder where you store your netdata configuration, run 

docker cp netdata:/etc/netdata/orig/apps_groups.conf apps_groups.conf

assuming your container is named netdata. Add the line 

shfs: shfs

to the end of the apps_groups.conf file. Then all you need to do is bind mount the file back into the container at the path /etc/netdata/apps_groups.conf (add the entry to the docker template, make sure both the host and container paths are to the file itself).

 

Thx, I did it, restarted the container.... and now where to look for this value in netdata webgui?

 

5 hours ago, jaclas said:

 

Thx, I did it, restarted the container.... and now where to look for this value in netdata webgui?

 

Under the Applications section, the graphs should now have a shfs line. For this specific issue the subsection apps.files will give you the number of open files for the shfs process.

55 minutes ago, primeval_god said:

Under the Applications section, the graphs should now have a shfs line. For this specific issue the subsection apps.files will give you the number of open files for the shfs process.

End of the config file:

 

 

and I can't find the metrics in netdata gui:

 

 

 

I'm probably not looking where I need to. Please show a screenshot if you have it configured.

 

ps. in fact I think I have a way of triggering this error, I need to check it a few more times and I will publish it

 

3 hours ago, jaclas said:

I'm probably not looking where I need to. Please show a screenshot if you have it configured.

If you select the "file descriptor limit" graph labeled fds in the side bar, then one of the dimensions on the graph should be shfs. I use an older version of the ui so i just annotated your screen shot with where to look.

 

@primeval_god

 

Unfortunately, there is no this item, here they are all on two screenshots:

 

 

 

There's another one at the end: VMs, and that's it. 
Perhaps this configuration from the apps_groups.conf file was not loaded?

 

In fact, I didn't strictly do what you suggested, i.e. mount the file in docker, because docker already has this whole directory mapped to a directory in Unraid: 
 

 

So I just copied it there from inside the container and added what you pointed out. But that's probably not the problem.

 

6 hours ago, jaclas said:

So I just copied it there from inside the container and added what you pointed out. But that's probably not the problem.

That shouldnt be a problem. Perhaps its a permissions issue? Do the permissions on the apps_groups.conf file match those of the other files in /etc/netdata? I assume you restarted the container after adding the config file, but if not that should be done as well.

On 5/10/2024 at 3:21 PM, primeval_god said:

That shouldnt be a problem. Perhaps its a permissions issue? Do the permissions on the apps_groups.conf file match those of the other files in /etc/netdata? I assume you restarted the container after adding the config file, but if not that should be done as well.

 

Eh, silly me.... I got the file name wrong, I had apps_group instead of apps_groups 🙂

Now it works, by the way I show a screenshot with a graph of the number of open files (all of them) on which you can see four big jumps. The jumps are just happening in the shfs area (lower, yellow) when I start compiling the application in node. The number of open descriptors in total reaches 18k, and the ulimit is set to about 40k, so there was no error this time. But I just happen to have most of the services disabled on the server, and just for a test I wanted to see if netdata would respond correctly. I will do tests under higher load soon.

 

Edited May 12 by jaclas

I have just encountered this. Caused my Plex installation to completely wipe all of the media out of its database.