When the cache drive becomes full, automatically invoke the Mover to move files from cache to array.

 

I don't want to wait for the schedule to move files, nor manually move files.  Automatic move can be an option.

 

3 minutes ago, Jaybau said:

When the cache drive becomes full, automatically invoke the Mover to move files from cache to array.

 

I don't want to wait for the schedule to move files, nor manually move files.  Automatic move can be an option.

 

Have you tried the Mover Tuning plugin to see if that satisfies your needs?

 

BTW:  You should always set a sensible Minimum Free Space setting for any pool as file systems tend to misbehave if they get too full.

It is impossible to move from fast cache to slow array as fast as you can write to cache. You are doing it wrong.

1. First off make sure all your shares are set to use cache:yes not only unless there are specific ones you want to stay on cache

2. Click on your Cache and look for Minimum free space. it will be grey'd out until you stop the array to change its value. Make sure its set to at least 2x the size of your largest file so if you edit a file it has room to move. I personally have mine set to 50GB, but whatever suits you. If your going to cache a lot more I'd recommend upping the Minimum free space. 

3. I use a 1TB SSD and I never come close to filling it. Maybe you need a larger SSD unless your constantly filling it with content and size isn't the issue. 

 

You can use the Mover Plugin which you can program to monitor your cache and move files based on specific parameters. Keep in mind it follows the scheduler for the regular mover, but does extra checks prior to moving. For example mine checks daily at 5:45AM and then fires off the plugin. I don't need it to run hourly. 

 

 

1 minute ago, kizer said:

specific ones you want to stay on cache

appdata, domains, system

4 minutes ago, kizer said:

2. Click on your Cache and look for Minimum free space. it will be grey'd out until you stop the array to change its value.

 

Is the default 0?

it very well could be. I don't recall I changed mine years. ago. 

 

Should say.    Minimum free space:

If you click on it, there should be a help box that gives your some examples. I have mine set to 50GB, but your needs might need to be more or less depending on you usage. 

 

 

2 minutes ago, Jaybau said:

 

Is the default 0?

At the moment, yes, because we have no way of knowing what a good value would be for any specific setup. It's up to you to set a sane value.

I ran into a different mess when with the default of zero (I don't think the default should be zero since zero is the most insane value...at least for me it was). 

2 hours ago, JonathanM said:

At the moment, yes, because we have no way of knowing what a good value would be for any specific setup. It's up to you to set a sane value.

 

IMO it should default to something like 10% of the drive capacity. Better to default to a safe setup even if that may mean losing access to some space by default and need changing to less to reclaim some space if that much isn't needed than risking filesystem corruption because it was set to 0 and filled to the brim...

Edited August 12, 20223 yr by Kilrah

42 minutes ago, Kilrah said:

 

IMO it should default to something like 10% of the drive capacity. Better to default to a safe setup even if that may need losing access to some space by default and need changing to less to reclaim some space if that much isn't needed than risking filesystem corruption because it was set to 0 and filled to the brim...

I’ve been suggesting this for some time if it is spotted that the value is 0, and leave it alone if it is any other value (I.e. explicitly set by the user).

On 8/12/2022 at 11:37 AM, trurl said:

It is impossible to move from fast cache to slow array as fast as you can write to cache. You are doing it wrong.

i mean, that's just not true.  I'm currently running backups from my array on one machine to the cache on another (because my IOwait times were near 50% if i wrote direct to the backups array).   That can only move as fast as the disk can read (and as fast as a 1Gbps network can transfer) but if its moving 5TB of stuff, it will fill up before its done.
I'm getting ~115MB/s writes to the cache thats 1TB. that means it will fill up in a 2.3 hours, so i have mover run every 2 hours. when mover runs it writes to the array at 160-190MB/s and slowly empties the cache. and that is while the cache is still being written to at 115MB/s
It would be cool if you could just set it to also invoke the mover if the ssd reaches ~90% full

On 8/12/2022 at 4:37 PM, trurl said:

It is impossible to move from fast cache to slow array as fast as you can write to cache. You are doing it wrong.

 

I have to say, this leaves a bit of a bad taste in my mouth. It's a shit comment. Constant writes were never mentioned, but you assumed so for some reason, then decided to go full Apple with a "You're holding it wrong" variant.

 

Regardless of it's feasability, usefulness, or anything else, it is a legitimate feature request, and one I've suggested before.

+1 from me. The Mover Tuning plugin can do this, but it would be a nice option to have native.

 

 

13 minutes ago, -Daedalus said:

 

Regardless of its feasability, usefulness, or anything else, it is a legitimate feature request, and one I've suggested before.

+1 from me. The Mover Tuning plugin can do this, but it would be a nice option to have native.

 

 

just for interest why not write directly to the array?    You array write speeds seem much higher than I normally expect to see (unless running without parity) so the advantage of going via a cache pool may be marginal.

I wasn't the one above quoting disk speeds, that was @PassTheSalt. It is weird that he's getting such high speeds to array though. I only get around 90 or so sustained.

2 minutes ago, -Daedalus said:

I wasn't the one above quoting disk speeds, that was @PassTheSalt. It is weird that he's getting such high speeds to array though. I only get around 90 or so sustained.

I wonder if @PassTheSalt does not have a parity drive?     In such a case the benefit from the cache is far smaller as individual drives can then be written as fast as the drive can handle.

That would goes a way towards explaining it. Cache still useful if it's NVME/SSD vs spinners, but certainly less so alright.

I dont see how Mover Tuning plugin can acomplish this

There should be an option to fire mover as soon as cache hits 100%

I regularly hit 100% when torrenting, though i have 10GB set as minimum free space on my 256GB cache array. Im currently forced to do a manual move or wait for scheduled move and my torrents just stop with error untill the move is sompleted, though its obvious that move should happen as soon as cache is full. I literally cant belive this is not implemented, its clear as sky it should be this way.

To clarify, i only use cache as data pre-cache, and dont use it for appdata or other system shares

Edited June 10, 20251 yr by Journeyman1