RT87 Posted May 14, 2021 Share Posted May 14, 2021 Hi, I would love for an additional "cache copier/syncer" as an addition to the existing mover. What exactly do I mean by that: Say I only want to use one large SSD cache drive (instead of 2 or more to get an array-like redundancy, since large SSDs are still somewhat pricey and, of course, take up space and ports). I would like to have all my docker/vm/often used data ond the cache drive, but I would also care to have that cache-data backed up on the standard array/data drives on a regular basis. Hence, I would like for a nightly backup to the array to occur, but not with moving, but copying/syncing instead. Ideally, the share which is used for this would be read-only to all users (a "cache-sync-share", if you will), so only the syncer can write to it (root definitely shouldn't, but could). However, the users/root CAN use it to restore the data from this share (in case of SSD failure), or, of course, delete it entirely, This would be very helpful, because I could save on SSDs without sacrificing redundancy. I personally do not care about the performance of the syncer, because this job would occur during low-to-none-load times. I also know that I can probably build this very feature on my own, using the "cache: only" setting + rsync/rclone or something similar (most basic: cp -uvr + crontab + remove data-singles without cache-equivalent), but I really feel like this is something many people would benefit from. Maybe this request will get a few "+1"s, which would be nice, but otherwise I would also be happy to just script-kiddie implement this on my own ;). Quote Link to comment
-Daedalus Posted May 14, 2021 Share Posted May 14, 2021 +1 I'm doing this manually at the moment, but it would be nice to have it built-in as a mover option. Quote Link to comment
Andiroo2 Posted December 28, 2021 Share Posted December 28, 2021 +1 for the H O T C A C H E. Quote Link to comment
-Daedalus Posted December 30, 2021 Share Posted December 30, 2021 Please yes. I'm rsyncing this at the moment, but I'd love a native option for it. Quote Link to comment
ich777 Posted January 2, 2022 Share Posted January 2, 2022 On 5/14/2021 at 11:38 AM, RT87 said: I also know that I can probably build this very feature on my own, using the "cache: only" setting + rsync/rclone or something similar (most basic: cp -uvr + crontab + remove data-singles without cache-equivalent), but I really feel like this is something many people would benefit from. Is CA Backup maybe the solution here in combination with luckyBackup or any other sync application for you other important data? Quote Link to comment
Monster1290 Posted January 9, 2022 Share Posted January 9, 2022 +1 But I'm thinking about slightly different logic. In share setting add checkbox "Sync cache with main pool" (or smth like this) when cache mode either Only or Prefer. In normal operation data are read/write to cache and synced with main pool when Mover is operating. Obviously this will lead to misunderstanding of word "move", so might be added another action "Sync" similar to "Mover" with it's own button, settings, etc (in settings might be checkbox to schedule syncing alongside with mover). When cache drive fails then data operation remains normal, but read/writes should be manually redirected to main pool. I think automatic redirection is not possible because of possible data consistency damage by process which will try to use data from main pool while they state based on data from cache pool. To restore normal operation user might change cache pool (in case of additional cache pools) or replace cache drive in current cache pool. In both cases syncing must be performed right away to eliminate uncertainty of where to write/read from data. I think this logic is the best from user perspective. You setup couple of settings and in case of failure just replace failed drive. Just like main pool operation. IMHO this feature will be very helpful because some of the users don't want to protect data on cache drive with another cache drive when main pool does the same. Quote Link to comment
RT87 Posted March 1, 2022 Author Share Posted March 1, 2022 On 1/2/2022 at 8:09 AM, ich777 said: Is CA Backup maybe the solution here in combination with luckyBackup or any other sync application for you other important data? For now I'm using Vorta, but sure, any reliable backup solution will do... but still, I can't be the "only one" with this use-case, so an "official" way would be beneficial. Quote Link to comment
ich777 Posted March 1, 2022 Share Posted March 1, 2022 35 minutes ago, RT87 said: For now I'm using Vorta, but sure, any reliable backup solution will do... but still, I can't be the "only one" with this use-case, so an "official" way would be beneficial. But what about CA Backup, if you configure it like this: It should be what you are trying to accomplish and you could also exclude folders, use a compression if you want to, set schedules,... I know it is not designed for this but it should do the job just fine. I really don't think that Unraid was designed for that use case, I also totally understand why you want this, I also have such disks that I backup frequently through luckyBackup with a schedule and snapshots. Quote Link to comment
RT87 Posted March 1, 2022 Author Share Posted March 1, 2022 I guess both pretty much do the same thing, featurewise, I just went with Vorta because it uses borg and it's something I can use more flexible, i.e. it's not unraid specific. With "official" I meant somehintg along the lines of "another cache use-type", just plain and simple, no versioning or anything, just hold-on-cache-and-sync-to-array-via-cronjob Quote Link to comment
JonathanM Posted March 1, 2022 Share Posted March 1, 2022 10 minutes ago, RT87 said: With "official" I meant somehintg along the lines of "another cache use-type", just plain and simple, no versioning or anything, just hold-on-cache-and-sync-to-array-via-cronjob That introduces a whole can of worms, because of the way Unraid merges the file systems across all array and pool drives. Multiple files with the same name and path on different drives show up as a single file in the user share filesystem, with only one actually being the target of reads and writes. Quote Link to comment
RT87 Posted March 2, 2022 Author Share Posted March 2, 2022 23 hours ago, JonathanM said: That introduces a whole can of worms, because of the way Unraid merges the file systems across all array and pool drives. Multiple files with the same name and path on different drives show up as a single file in the user share filesystem, with only one actually being the target of reads and writes. It wouldn't need to be the same path, I could simply backup the appdata cache-path to /mnt/user/appdata_cache_backup or somthing similar. Quote Link to comment
trurl Posted March 2, 2022 Share Posted March 2, 2022 35 minutes ago, RT87 said: It wouldn't need to be the same path, I could simply backup the appdata cache-path to /mnt/user/appdata_cache_backup or somthing similar. That is what the CA Appdata Backup plugin lets you do. Quote Link to comment
RT87 Posted March 3, 2022 Author Share Posted March 3, 2022 11 hours ago, trurl said: That is what the CA Appdata Backup plugin lets you do. well almost, I still would like to just have it run with the mover-job. Just as a regular no-plugin-or-config-required setup, simply by using a new pool-usage-type "prefer-cache-but-sync-to-array-during-moving" I'm a little worried that people will neglect this because they may not fully realise the danger of running everything on your cache without a cache-array/backup. But maybe I'm a little off here, I don't know, as I said, I'm happy with my solution, I just think a more elegant way might be nice. Quote Link to comment
trurl Posted March 3, 2022 Share Posted March 3, 2022 6 hours ago, RT87 said: would like to just have it run with the mover-job More flexible to have it run as its own thing with its own schedule. Looks like Mover Tuning plugin will let you add scripts to run before/after mover. Seems to be a work in progress, I don't use it. Quote Link to comment
raven42 Posted March 23, 2022 Share Posted March 23, 2022 +1 for native solution for this. Quote Link to comment
Hammy Havoc Posted September 7, 2022 Share Posted September 7, 2022 On 3/23/2022 at 4:24 PM, raven42 said: +1 for native solution for this. I'd like to see this too. TrueNAS Scale has some very good backup options out of the box. https://www.truenas.com/docs/core/gettingstarted/databackups/ Feels like the missing piece of the puzzle in Unraid. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.