"Out Of Memory errors detected on your server" - Which one(s)?


Recommended Posts

  • 2 weeks later...

Why is your system share on the array? Also domains and appdata. Usually you want these to be all on cache and configured to stay on cache so your dockers / VMs won't keep the array spunup with their always open files, and so dockers / VMs won't have performance impacted by slower parity writes.

 

Why have you given 40G to docker.img? Have you had problems filling it? 20G is usually more than enough.

 

The usual reason for filling docker.img is an application writing to a path that isn't mapped.

 

The usual reason for docker filling RAM is mapping a host path that isn't actual storage.

Link to comment
4 hours ago, trurl said:

Why is your system share on the array? Also domains and appdata. Usually you want these to be all on cache and configured to stay on cache so your dockers / VMs won't keep the array spunup with their always open files, and so dockers / VMs won't have performance impacted by slower parity writes.

 

Why have you given 40G to docker.img? Have you had problems filling it? 20G is usually more than enough.

 

The usual reason for filling docker.img is an application writing to a path that isn't mapped.

 

The usual reason for docker filling RAM is mapping a host path that isn't actual storage.

I haven't done that intentionally if it is. I'll change that though.
The docker image got full, so I just said fk it, and gave it twice as much.
To my knowledge, I'm not writing to the docker image.

Link to comment

Also notice many of your shares are using Most Free allocation. That may be less efficient than Highwater since it could cause constant switching to other disks and keeping disks spunup just because one disk temporarily has more free than another.

 

Making docker.img larger won't fix problems with filling it. It will only make it take longer to fill.

 

How many dockers do you have? I am running 18 dockers and they are taking only about half of 20G docker.img

 

To get appdata, domains, system shares off the array you will have to

  1. Set them to cache-prefer.
  2. Go to Settings and disable both Docker and VM Manager.
  3. Run mover and wait for it to finish.

Then post new diagnostics.

Link to comment
1 hour ago, trurl said:

Also notice many of your shares are using Most Free allocation. That may be less efficient than Highwater since it could cause constant switching to other disks and keeping disks spunup just because one disk temporarily has more free than another.

 

Making docker.img larger won't fix problems with filling it. It will only make it take longer to fill.

 

How many dockers do you have? I am running 18 dockers and they are taking only about half of 20G docker.img

 

To get appdata, domains, system shares off the array you will have to

  1. Set them to cache-prefer.
  2. Go to Settings and disable both Docker and VM Manager.
  3. Run mover and wait for it to finish.

Then post new diagnostics.

32. But only 29 is running atm.
I used Most Free Allocation so I could spread it out more. I guess my idea was when one of the  dies, I'll be without that data for shorter.

 

I'll run the mover in the morning.
I've set it to cache only atm. should I still change it to prefer?

Link to comment

There is a Help (?) toggle for the whole webUI on the menu bar. You can also toggle help for a specific setting by clicking on its label. Take a look at the help for the cache setting for that User Share.

 

Long story short, Mover ignores cache no or only. To get Mover to move from array to cache, it must be cache prefer.

 

The reason you have to disable dockers / VMs is because Mover (or anything else) can't move open files. Mover also won't move duplicates (files that already exist on the destination).

 

Diagnostics after moving will tell us if everything got moved. But you can also see how much of each disk is used by each User Share by clicking Compute... for the share, or use the Compute All button.

Link to comment
1 hour ago, trurl said:

There is a Help (?) toggle for the whole webUI on the main bar. You can also toggle help for a specific setting by clicking on its label. Take a look at the help for the cache setting for that User Share.

 

Long story short, Mover ignores cache no or only. To get Mover to move from array to cache, it must be cache prefer.

 

The reason you have to disable dockers / VMs is because Mover (or anything else) can't move open files. Mover also won't move duplicates (files that already exist on the destination).

 

Diagnostics after moving will tell us if everything got moved. But you can also see how much of each disk is used by each User Share by clicking Compute... for the share, or use the Compute All button.

Yeah, I just didn't get my head around it properly.
I've done the changes now, and added new diagnostics. 
I'm also using unbalance to start moving files one drive at a time.

nanostorage-diagnostics-20210330-0141.zip

Link to comment
27 minutes ago, Nanobug said:

using unbalance

I haven't used that in a long time. Does it work with cache?

 

The builtin Mover is the usual way to get things moved to/from cache according to the settings for each user share. You can directly run it from Main - Array Operation.

Link to comment
6 hours ago, trurl said:

I haven't used that in a long time. Does it work with cache?

 

The builtin Mover is the usual way to get things moved to/from cache according to the settings for each user share. You can directly run it from Main - Array Operation.

I did just the regular mover.

Just using unbalance to move from one disk to the rest, just to get the "high water" usage on all of the disks.

Link to comment
5 minutes ago, Squid said:

The logs are being filled because you've got atop installed via NerdPack.  It's notorious for that.

Dang... I didn't know that, and I thought I uninstalled most of them again, since I looked through what could be used. Must have forgotten that one.
Uninstalling it right away. Thanks!
I'll let you guys know tomorrow if it continues.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.