Add "Both" to "Use Cache"

[mgutt]

I like to use only my SSD cache for some Shares, but by that I would need to create manually a backup routine. Instead I like to choose an option called "Both" where the Mover syncs between the Cache and the Share.

 

Example:

Edited April 28, 2020 by mgutt

[testdasi]

1 hour ago, mgutt said:

I like to use only my SSD cache for some Shares, but by that I would need to create manually a backup routine. Instead I like to choose an option called "Both" where the Mover syncs between the Cache and the Share.

 

Example:

+1

Good idea. Sort of an online backup functionality.

Probably name it "cloned" is a better name since "both" can be interpreted differently.

 

This would be a great use case for high speed NAS (e.g. for video editing). We could have a RAID0 cache pool for max speed for WIP files. These are automatically backed up to the slow array nightly using the mover so (some) protection against RAID0 failure. Once done, WIP files can be moved to the archive (again on the slow array).

 

Potential complication with shfs as moving operation will be more complicated. Given the arrangement can easily be achieved using User Scripts plugin to cron a nightly bash script, it probably isn't a high priority but it would be useful nonetheless.

Edited April 28, 2020 by testdasi

[itimpi]

It is not clear to me what the advantage the proposed “Both” option has?    In fact it seems to have the disadvantage that new files start failing when there is no free space on the cache, rather than succeeding by by-passing the cache and going directly to the array as happens with the current Yes option.

[mgutt]

I'm having a 1TB SSD installed. So the cache is big enough for my needs. If not I will install a bigger one.

 

Why I need it:

1.) I want to add my music collection to the SSD as it is more energy efficient compared to the HDD and the response time is much better between a sleeping SSD and HDD (a press on the play button on a sonos speaker plays almost directly if I use the SSD).

2.) I want to add my video transcoding folder to the cache. This is the data I'm working on the weekends. I move the videos to an other share if I finished my work (= long time archive). At the moment I can only take advantage of the SSD as long the mover did not move the videos to the array. Of course a defective SSD kills the work of multiple hours, but after the mover did his work the data would be safe.

 

At the moment I'm using rsync to copy both data to a backup folder on a daily basis. This perfectly solves my problem, but maybe its interesting having it as a builtin feature of Unraid.

[itimpi]

Hmm - I think I see - you want files to exist in two places - on the cache AND on the array?

 

Things might get a bit messy as Unraid has a built in assumption that a file only exists either on the cache or on the array.   You might start getting some unexpected behaviour if the file existed in two places.    As an example deleting a file would initially appear to not work as you would be deleting the cache copy, and when that was gone the array copy (which may have different contents) would have taken its place.

[mgutt]

Yes, with such a setting Unraid must totally ignore the content of the array else it would not work.

[itimpi]

5 hours ago, mgutt said:

Yes, with such a setting Unraid must totally ignore the content of the array else it would not work.

I suspect that makes such a change a large amount of effort for Limetech and this not likely to appear any time in the near future (if at all).  I would like to be proved wrong though.

[-Daedalus]

I would like this as well. +1

 

Currently, I have my documents and music shares as cache-prefer, because I don't have to wait for spinners to spin up. +1 to responsiveness, +1 to energy efficiency.

 

But, I manually back these shares up to a backup share on the array via rsync because, to be frank, while the cache pool is nice, I don't trust BTRFS to keep my data safe; It's probably fine in reality, but I've read a few too many corruption stories with the result of "format cache drives and restore from backup".

 

 

Could something like this be done behind the scenes?

Have a "Clone/both/whatever" setting function like cache-only, but run an rsync once the mover is done?