Mover Tuning; not moving from cache based on capacity during parity

[cities]

Had a power outage yesterday and needed to perform a 16 Tb parity rebuild (1-1.5 days). I'm still actively using my NAS, and yesterday I filled up the cache (240 Gb) twice.

 

The first time, I was at the computer and kicked off a force move.

 

The second time, I was sleeping, and when the cache filled up, my VMs stopped along with all file transfers (parity rebuild is still going).

 

I know the mover (by default) does not run concurrently with a parity build. This can be configured with the "Let scheduled mover run during a parity check / rebuild" setting in Mover Tuner.

 

Mover settings:

• schedule: weekly
• day of week: sunday
• day of month: N/A
• Time of day: 03:40
• Mover logging: Enabled

Mover tuning w age:

• All "No" "0" or "Normal" (default settings) unless otherwise specified
• Let scheduled mover run during a parity check / rebuild: Yes
• Move All from Cache-Yes shares when disk is above a certain percentage: Yes
• Move All from Cache-yes shares pool percentage: 70%

 

I also checked my `/var/log/syslog`, but the only lines related to the mover were during install (ie: when searching "mover" I didn't see any lines about failed moves, which I would have expected if a capacity move attempted -- and failed -- to start).

 

Is this a bug? Perhaps there is a missing "Let capacity mover run during a parity check / rebuild" setting.

 

Or, do I need to do something else to ensure my cache is cleared in time to avoid it filling up?

[trurl]

You can go directly to the correct support thread for any of your plugins by clicking its Support link on your Plugins page.

[itimpi]

Have you made sure that you have a sensible Minimum Free Space setting for the cache to avoid it getting completely filled which can cause problems?

[trurl]

You probably need to reconsider how you are using cache. Here is a recent post where I discuss caching.

 

 

 

[trurl]

If you really want help with the plugin, go to its thread.

 

If you want more general advice on how to change your configuration to make things work better,

 

attach diagnostics to your NEXT post in this thread.

[cities]

40 minutes ago, itimpi said:

Have you made sure that you have a sensible Minimum Free Space setting for the cache to avoid it getting completely filled which can cause problems?

 

It seems I misunderstood how the mover tuner plugin worked. I thought the % fill moving condition fired independent of the schedule (ie: I could configure mover to clear the cache whenever it reached x capacity). I had calculated my average network throughput, and the 70% figure I landed on was more than enough.

That is... assuming moves off cache started at a certain capacity. Which they didn't.

Someone on Reddit replied to my cross-post there and answered my question: "The Mover Tuning plugin doesn't automatically run the mover when one of the conditions is met. The mover still just runs on the schedule you set."

  

43 minutes ago, trurl said:

You can go directly to the correct support thread for any of your plugins by clicking its Support link on your Plugins page.

Thanks, first time posting here and I was unaware of that feature. I'll move the discussion there to confirm what I heard on reddit.

 

Edit: Someone else had the same misunderstanding as me, and the explanation from Reddit was confirmed in the plugin thread: 

 

 

Guess I'll either setup the tuner to run on a faster schedule or write a bash script of my own to move at a certain filled %.

Edited June 22, 2022 by cities

[trurl]

Mover tuning, bash script, whatever.

 

It is impossible to move from cache to slower array as fast as you can write to cache.

 

See my link above for more