KRunyon Posted June 21, 2020 Posted June 21, 2020 Can anyone think of a method to have two cache drives basically mirror each other without being in RAID1? My first thought is Syncthing, but I haven't used it in years, and I'm not 100% sure it can be a source and destination from a container. Backstory: I just splurged on a 16TB SAS SSD, and I already have an 8TB SAS SSD. Would love to use them as a cache drive and have some sort of tolerance against a drive failure, but RAID1 would leave 8TB inaccessible. I currently use the 8TB drive as a UD with my Steam library on it (10GbE, getting 1GB/s+). If a method to sync the two caches is found, I figured I would create a '.Steam' folder on the cache drive to exclude it from the sync, and create a custom share script to retain it as my Steam storage location. I know its all exceptional overkill, but I suspect I'll eventually commercialize my server. Quote
KRunyon Posted June 21, 2020 Author Posted June 21, 2020 (For reference, these SSD's get 2000MB/s throughput locally, so syncing them after/during use will be trivial since my local network caps at half that, and the internet writes won't exceed 1Gb/s (125MB/s) ) Quote
JorgeB Posted June 21, 2020 Posted June 21, 2020 You can use rsync if it's XFS or rsync/snapshots if it's btrfs. 1 Quote
KRunyon Posted June 22, 2020 Author Posted June 22, 2020 3 hours ago, johnnie.black said: You can use rsync if it's XFS or rsync/snapshots if it's btrfs. Gave it some thought, probably going to use rsync, and do ranges of sync based on file size (<10MB, 10-30MB, 30-200MB, 200-1500MB, >1500MB), ignoring duplicates based on name + file size + last modified, and delete if not found in source. I think that will cover my bases pretty well Guessing that if mover comes around to move the files and they are "open" by rsync to copy them elsewhere, they'll get passed until the next mover sweep? Or does a copy not invoke exclusive access? Quote
KRunyon Posted June 22, 2020 Author Posted June 22, 2020 (edited) (Just checked the release notes for Unraid 6.9beta, looks like I'll want to wait until that goes at least RC since the way cache drives are handled is going to change a bit: "A multiple-device pool may only be formatted with btrfs.", which hopefully can still be JBOD...) (Might have to leave one disk in Cache, and one disk as UD then rsync between them) Edited June 22, 2020 by KRunyon Quote
JorgeB Posted June 22, 2020 Posted June 22, 2020 The main change is that now you can have multiple pools, any pool with more than one device must be btrfs, like always. Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.