Invoke Mover when cache is full


Recommended Posts

3 minutes ago, Jaybau said:

When the cache drive becomes full, automatically invoke the Mover to move files from cache to array.

 

I don't want to wait for the schedule to move files, nor manually move files.  Automatic move can be an option.

 

Have you tried the Mover Tuning plugin to see if that satisfies your needs?

 

BTW:  You should always set a sensible Minimum Free Space setting for any pool as file systems tend to misbehave if they get too full.

  • Like 1
  • Upvote 1
Link to comment

1. First off make sure all your shares are set to use cache:yes not only unless there are specific ones you want to stay on cache

2. Click on your Cache and look for Minimum free space. it will be grey'd out until you stop the array to change its value. Make sure its set to at least 2x the size of your largest file so if you edit a file it has room to move. I personally have mine set to 50GB, but whatever suits you. If your going to cache a lot more I'd recommend upping the Minimum free space. 

3. I use a 1TB SSD and I never come close to filling it. Maybe you need a larger SSD unless your constantly filling it with content and size isn't the issue. 

 

You can use the Mover Plugin which you can program to monitor your cache and move files based on specific parameters. Keep in mind it follows the scheduler for the regular mover, but does extra checks prior to moving. For example mine checks daily at 5:45AM and then fires off the plugin. I don't need it to run hourly. 

 

 

Link to comment

it very well could be. I don't recall I changed mine years. ago. 

 

Should say.    Minimum free space:

If you click on it, there should be a help box that gives your some examples. I have mine set to 50GB, but your needs might need to be more or less depending on you usage. 

 

 

Link to comment
2 hours ago, JonathanM said:

At the moment, yes, because we have no way of knowing what a good value would be for any specific setup. It's up to you to set a sane value.

 

IMO it should default to something like 10% of the drive capacity. Better to default to a safe setup even if that may mean losing access to some space by default and need changing to less to reclaim some space if that much isn't needed than risking filesystem corruption because it was set to 0 and filled to the brim...

Edited by Kilrah
Link to comment
42 minutes ago, Kilrah said:

 

IMO it should default to something like 10% of the drive capacity. Better to default to a safe setup even if that may need losing access to some space by default and need changing to less to reclaim some space if that much isn't needed than risking filesystem corruption because it was set to 0 and filled to the brim...

I’ve been suggesting this for some time if it is spotted that the value is 0, and leave it alone if it is any other value (I.e. explicitly set by the user).

Link to comment
  • 1 year later...
On 8/12/2022 at 11:37 AM, trurl said:

It is impossible to move from fast cache to slow array as fast as you can write to cache. You are doing it wrong.

i mean, that's just not true.  I'm currently running backups from my array on one machine to the cache on another (because my IOwait times were near 50% if i wrote direct to the backups array).   That can only move as fast as the disk can read (and as fast as a 1Gbps network can transfer) but if its moving 5TB of stuff, it will fill up before its done.
I'm getting ~115MB/s writes to the cache thats 1TB. that means it will fill up in a 2.3 hours, so i have mover run every 2 hours. when mover runs it writes to the array at 160-190MB/s and slowly empties the cache. and that is while the cache is still being written to at 115MB/s
It would be cool if you could just set it to also invoke the mover if the ssd reaches ~90% full

Link to comment
On 8/12/2022 at 4:37 PM, trurl said:

It is impossible to move from fast cache to slow array as fast as you can write to cache. You are doing it wrong.

 

I have to say, this leaves a bit of a bad taste in my mouth. It's a shit comment. Constant writes were never mentioned, but you assumed so for some reason, then decided to go full Apple with a "You're holding it wrong" variant.

 

Regardless of it's feasability, usefulness, or anything else, it is a legitimate feature request, and one I've suggested before.

+1 from me. The Mover Tuning plugin can do this, but it would be a nice option to have native.

 

 

Link to comment
13 minutes ago, -Daedalus said:

 

Regardless of its feasability, usefulness, or anything else, it is a legitimate feature request, and one I've suggested before.

+1 from me. The Mover Tuning plugin can do this, but it would be a nice option to have native.

 

 


just for interest why not write directly to the array?    You array write speeds seem much higher than I normally expect to see (unless running without parity) so the advantage of going via a cache pool may be marginal.

Link to comment
2 minutes ago, -Daedalus said:

I wasn't the one above quoting disk speeds, that was @PassTheSalt. It is weird that he's getting such high speeds to array though. I only get around 90 or so sustained.

I wonder if @PassTheSalt does not have a parity drive?     In such a case the benefit from the cache is far smaller as individual drives can then be written as fast as the drive can handle.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.