A proposal for smart read/write caching

[droctakan]

I've used Synology, TrueNAS (core and scale), and Unraid and overall Unraid is my favorite.

 

But I continue to be surprised by the lack of sophistication in the cache system. I'm constantly choosing between cache: yes, where the caches sit mostly empty unless I just wrote a lot to the share, or cache: prefer, where the mover will happily try to shove everything into the cache, bringing the NVMEs or SSDs to almost completely full and taking data off the parity protected drives. Now, I can fix some of the above with tuning and with the mover plugin. But it would be fairly easy to have this all work a lot better.

 

So my proposal would be to implement cache: smart.

 

In this mode Unraid will monitor file opens/reads/writes and populate a small database. This can either be done by actually watching for open/read/write system calls, or can periodically poll processes for open file descriptors. Then the mover can periodically update a priority queue and create a "plan" for what files it prefers to have on the cache, how much room it wants to preserve for incoming writes, etc. Once a plan is created, the mover can move data to the proper place.

 

In my ideal world, data on the cache is also replicated to the array, since it worries me to have cached data not parity protected. Maybe this is hard to do in a consistent way, but a COW filesystem should make this quite a bit easier.

[marco_yang]

Agree

Unraid is my favorite due to it's ease on add different volume disks but I was always wondering why it didn't have a smart cache system. In my synology nas there's a screen display the hit rate of my ssd cache and I really wish the same on my unraid server.