Question: Cache Drive one way syncing to another cache drive


Recommended Posts

Can anyone think of a method to have two cache drives basically mirror each other without being in RAID1? My first thought is Syncthing, but I haven't used it in years, and I'm not 100% sure it can be a source and destination from a container.

 

Backstory: I just splurged on a 16TB SAS SSD, and I already have an 8TB SAS SSD. Would love to use them as a cache drive and have some sort of tolerance against a drive failure, but RAID1 would leave 8TB inaccessible. I currently use the 8TB drive as a UD with my Steam library on it (10GbE, getting 1GB/s+). If a method to sync the two caches is found, I figured I would create a '.Steam'  folder on the cache drive to exclude it from the sync, and create a custom share script to retain it as my Steam storage location.

 

I know its all exceptional overkill, but I suspect I'll eventually commercialize my server.

Link to comment
3 hours ago, johnnie.black said:

You can use rsync if it's XFS or rsync/snapshots if it's btrfs.

Gave it some thought, probably going to use rsync, and do ranges of sync based on file size (<10MB, 10-30MB, 30-200MB, 200-1500MB, >1500MB), ignoring duplicates based on name + file size + last modified, and delete if not found in source. I think that will cover my bases pretty well

 

Guessing that if mover comes around to move the files and they are "open" by rsync to copy them elsewhere, they'll get passed until the next mover sweep? Or does a copy not invoke exclusive access?

Link to comment

(Just checked the release notes for Unraid 6.9beta, looks like I'll want to wait until that goes at least RC since the way cache drives are handled is going to change a bit: "A multiple-device pool may only be formatted with btrfs.", which hopefully can still be JBOD...) (Might have to leave one disk in Cache, and one disk as UD then rsync between them)

Edited by KRunyon
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.