How about a multi-factor mover trigger:

 

Example:

 

The mover function triggers when

 

Disk space below 20%

<or>

All necessary drives currently spun-up

<and>

Array activity subsided for X minutes

<or>

If it's been X-days since the last move

<if>

specific share

<or>

cache drive's SMART characteristic changes

...etc.

 

 

That sort of flexibility could potentially allow an array to sleep for days before being spun up.  I know for myself, the files most recently uploaded ARE the most likely to be viewed/reviewed/replaced.  That could cut down on disk fragmentation.  And for those worried about increased dwell time on the cache drive, having the action vary depending on which share is involved could help ensure that while last night's code updates would be saved nightly, last night's PBS special could hang around on the cache drive for a few days- long enough for you to watch it (or maybe delete/replace it) before being moved to the array.  Just a thought.

 

 

 

I was going to suggest something much less sophisticated - just for the mover to be (optionally) triggered when the cache drive gets close to being full - but I like this idea very much.

 

There is a min free space setting for the cache drive that will already cause the written files to go directly to the array.  It doesn't invoke the mover, but will prevent an out of space condition on the cache if properly configured.

Yes, thanks. I knew about that. You need to set it to a value larger than the biggest file you're likely to copy otherwise you get an out of space error. But I'd like the option to run the mover automatically whenever the cache approaches being full, though the original post contains a much better idea.

I was thinking something much less involved. But... I like this.

 

Move when X% full

Don't move when array has errors

Daily Overnight Move.

I would love it to be even more intelligent..

 

I would love it if the mover could be setup to not be time-based at all.. I am more than happy to have files remain on my cache pool as long as there is enough space.. The moment there is no longer enough space the oldest placed files could be moved off..

 

For a media server that should be nice behaviour, I almost always watch newly added content within a few days.. So given a cache pool large enough I would be using the cached version in most circumstances keeping the drives spun down. Ideally this would be configurable on a per-user share basis, so the tactic could be different for media and, for example, photos..

A couple of things.

 

Mover runs every night. I think for sanity / saftey reasons that it's important that mover at minmum runs at a regular scheduled time....

 

I really don't want to be in the spot where a file I put on my array has been setting on the cache for 3 days (when I thought it was added to the array 3 days ago..) when the cache disk failes. Anything that makes it harder to know when files are protected or not is not a good change.

 

I can see adding logic to trigger mover at additional times for logical reasons (but I'm not sure I agree with some of the uses cases here because how often do you plan to fill your Cache during the course of 24 hours... if you are having this problem perhap syou need a bigger cache, or to invoke it manually.).

 

A couple of things.

 

Mover runs every night. I think for sanity / saftey reasons that it's important that mover at minmum runs at a regular scheduled time....

 

I really don't want to be in the spot where a file I put on my array has been setting on the cache for 3 days (when I thought it was added to the array 3 days ago..) when the cache disk failes. Anything that makes it harder to know when files are protected or not is not a good change.

 

I can see adding logic to trigger mover at additional times for logical reasons (but I'm not sure I agree with some of the uses cases here because how often do you plan to fill your Cache during the course of 24 hours... if you are having this problem perhap syou need a bigger cache, or to invoke it manually.).

 

This is a personal choice.. I amusing a BTRFS cachepool (standard functionlity) based on 3 ssd drives, this is actually a more robust storage location then the array itself..

 

Imho opinion, its time to think another way, the cache drive is no longer a scary temporary location..

A couple of things.

 

Mover runs every night. I think for sanity / saftey reasons that it's important that mover at minmum runs at a regular scheduled time....

 

I really don't want to be in the spot where a file I put on my array has been setting on the cache for 3 days (when I thought it was added to the array 3 days ago..) when the cache disk failes. Anything that makes it harder to know when files are protected or not is not a good change.

 

I can see adding logic to trigger mover at additional times for logical reasons (but I'm not sure I agree with some of the uses cases here because how often do you plan to fill your Cache during the course of 24 hours... if you are having this problem perhap syou need a bigger cache, or to invoke it manually.).

 

This is a personal choice.. I amusing a BTRFS cachepool (standard functionlity) based on 3 ssd drives, this is actually a more robust storage location then the array itself..

 

Imho opinion, its time to think another way, the cache drive is no longer a scary temporary location..

 

A couple of thoughts on this. 1) BTRFS cache pools are really only for advanced users currently. You are correct that it is a game changer when it comes to adding RAID like protection to your cache. I can see a case for a multi-factor mover if and only if they are running a BTRFS cache set to add loss of drive protection. (You can also use BTRFS cache pooling to make 4 small SSDs into one large SSD... if that is what they are doing they risk a loss of data.)

 

2) I've seen a lot of reports in the past 2-3 months about BTRFS cache drives and pools becoming corrupt.