mergerFS using 1500%-1600% CPU


Recommended Posts

Hey All, 

 

Hopefully this is going to be an easy issue to curb. I'm loving Unraid, and it's been working GANG Busters for the past few weeks. Starting this weekend, I ran into an issue where my rclone mount was disconnected and giving me an "Endpoint Transport" disconnect error. My SABnzbd downloader kept downloading and I believe thats' whats bringing us to my current issue.

 

(Initial setup completed using the guide below)

 

When I try accessing my mergerfs folder /mnt/user/mount_mergerfs/gdrive_vfs/[Contents of my GDrive Team Drive], my terminal is intermittently able to list the contents of this folder, other times it just freezes and the terminal window needs to be closed/reopened to regain control. When I AM able to list the contents of my Gdrive_vfs folder, it looks like some folders were duplicated (books, movies, tv) and it shows 2 entries for these folders.

ie. 

/mnt/user/mount_mergerfs/gdrive_vfs/books

/mnt/user/mount_mergerfs/gdrive_vfs/books

/mnt/user/mount_mergerfs/gdrive_vfs/movies

/mnt/user/mount_mergerfs/gdrive_vfs/movies

/mnt/user/mount_mergerfs/gdrive_vfs/tv

/mnt/user/mount_mergerfs/gdrive_vfs/tv

 

(Is it possible to collapse 2 folders with an identical name into just 1 folder using one command?)

To mount the rclone/mergerfs I'm using this script https://github.com/BinsonBuzz/unraid_rclone_mount

 

When I check the Dashboard, it shows that my CPU (Ryzen 3700) has all 16 threads maxed out at 100% usage (48GB of RAM idling around 15% usage).

https://imgur.com/a/JA8n3kX

 

When I check in Glances, it reveals that there is 1 command (mergerfs) which is currently using... 1,578.0% of my available CPU. 

https://imgur.com/a/WsDCTBn

 

This is affecting the normal usage of my server (as you can imagine), the biggest issue is that when I send reboot commands, unraid seems to ignore it. The server needs to be hard rebooted in order for it to come back up following a reboot command being sent. 

 

At the moment I'm unsure on how to proceed, rclone/mergerfs is still uploading data to my Gdrive at a slowish rate, and I'm hoping that it'll figure itself out eventually after its able to upload all of the data currently local on my machine. 

 

Let me know if you have any thoughts for solutions. 

Yours Truly, 

TechnicallyConfused

Edited by TechnicallyConfused
added imgur link for cpu/ram usage view from Dashboard tab
Link to comment

Read the last couple of posts as there was a problem with a mergerfs build that the author resolved

On 3/1/2020 at 7:49 PM, teh0wner said:

I finally managed to get this working. Try and make a list of all docker images/containers you have, and nuke all the mergerfs ones. That will force pull the latest one, which seems to be working.

 

Link to comment

Thanks for the reply!

 

I don't think that my mergerfs is running inside of a docker client. Its running as a user script when the computer boots, and then it re-runs itself every 10min - 6 hours. 

 

I'm almost at the point where I'm going to nuke my dockers and just deal with the headache of setting everything back up.

 

My concern is, the dockers only reference the mergerfs folders that contain my media, specifically the main folder where my GDrive gets mounted

/mnt/user/mount_mergerfs/gdrive_vfs/ 

but the docker containers don't have any role to play in setting up the mergerfs connection, other than peering into it. 

 

I'll try turning off all my dockers that get started with the user script, and nuke 1 container and rebuild from scratch to see if that makes a difference with the duplicated folders. 

 

 

 

In my research I've come across another tool (https://github.com/trapexit/mergerfs-tools) mergerfs.dedup which is apparently designed to fix this exact issue, but I'm running into a hell of a time trying to get it running on my UnRaid box. 

Link to comment

Don't bother trying to compile mergerfs-tools.

Trapexit created the docker for Unraid just so we can compile the mergerfs binary precisely because it wasn't possible to compile the binary out of Unraid.

 

What you can try is to just reboot. That should rebuild the latest mergerfs binary and hopefully it will resolve your problem.

Link to comment

My dude! Thats awesome!

 

Where can I find the docker for mergerfs binary? Is it in Dev Tools? Nerd Tools? Apps Store?

 

edit: I think I found it. https://hub.docker.com/repository/docker/trapexit/mergerfs-static-build

 

I will do the thing you suggested as soon as I can find it! 

VERY EXCITED!

TechnicallyConfused

Edited by TechnicallyConfused
wait... I think I found it
Link to comment

You big beautiful individual! It works! I could kiss you. 

 

Think I had myself confused there for a minute. 

 

Just like you said, run these 2 commands and everything is OK: 

 

mkdir -p /mnt/user/appdata/other/test/mergerfs

docker run -v /mnt/user/appdata/other/test/mergerfs:/build --rm trapexit/mergerfs-static-build

 

I feel so blessed to have access to so many smart people.

Thanks for the help!

That Voodoo that you do, makes everything so much easier for schlubs like me :) 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.