12 minutes ago, raptorjr said:

Ok, then it is my fault. Then it doesn't really help if the cache get filled faster than the cron is about to execute.

But you can set cron to be every hour or 5 min to execute and set mover tuning setting about when to move or not (thresholds etc)

30 minutes ago, Masterwishx said:

Just checked again, for sync you need to make files will be keep on primary :

movingThreshold="60" -> will move files after that pool threshold until freeingThreshold

freeingThreshold="40" -> files until this pool threshold will be synced

So if sync enabled files will be synced when before freeing threshold.

Seems it need to be changed to more easy way and set shares to choose for sync :)

that would be awesome .. for the movie collection share create a duplicate for everything on the array to the cache except .mkv and .mp3 :) that would be the best possible filter setting. in dependent of any threshold ... i see when my cache is getting full :D

Edited July 24, 20251 yr by Grobalt

11 hours ago, Grobalt said:

the cache except .mkv and .mp3 :)

Ignore files already available by settings

For users who using synchronise :

1. how do use it? If using already

2.what should be done in addition to choosing shares for synchronization?

16 minutes ago, Masterwishx said:

Ignore files already available by settings

i need the opposite ... a whitelist and not a blacklist.

it would be great to use the mover tuning to make unraid a tiered storage solution .. small, quick access files on nvme (but a redundant copy on parity protected array) and large, streaming files on array.

Edited July 25, 20251 yr by Grobalt

13 minutes ago, Grobalt said:

i need the opposite ... a whitelist and not a blacklist.

you want white list for files that will be moved ?

1 minute ago, Masterwishx said:

you want white list for files that will be moved ?

white list for files that should be duplicated on cache and array

10 minutes ago, Grobalt said:

white list for files that should be duplicated on cache and array

I will check, I wanted to add shares names, maybe will add files also, you mean files by extensions?

yeah, file extensions would be sufficient.

example .. a movie collection usually has the movies in folders with moviename, in the folder then JPG, PNG, MKV, AVI, MP4 etc. To show the movies in the player GUI it is great to have the small text files and pictures on fast cache and the large files on array. but for reliability, copies on cache, not cache only.

19 hours ago, Masterwishx said:

You can see in log line 18 : 01:00:01.364 Warning: Moving thresholds are equal. No action will be taken because the thresholds do not provide a gap.

So if you want files to move you need to set gap between moving and freeing thresholds.

But also you have many overrides for shares, check them if they configured exactly how you wanted or remove them.

Do you mean syncing not working from primary to secondary ?

Hi, thanks for your lightning fast reply. I removed the custom overrides to have one central configuration for the plugin.

What I meant regarding syncing: Files, which should reside mainly on primary, are moved to secondary and live only there. No copy on primary. For example, a couple of .jpeg thumbnails were created two days ago and exist only on secondary. My suspicion is that something is amiss between mover, mover tuner and the ZFS datasets. I had to manually create datasets, because unraid only created them on secondary or vice versa. So, for example, a share was created as a ZFS dataset with unraid, but only created on secondary. So I created its sibling on primary. I am not sure if this could be a problem, the GUI does not give any indication that it is a problem.

It seems random, where files end up, and it does mean that the disks spin up for files a few kilobytes in size, when they should only spin up when files beyond a configured threshold and age are accessed via a user share.

4 hours ago, Masterwishx said:

For users who using synchronise :

1. how do use it? If using already

2.what should be done in addition to choosing shares for synchronization?

1. Create parity protection for all files primarily stored on a fast storage medium which should only be accessed by consumers (SMB/NFS mounts, docker shares) on this fast storage medium

2. Continuous synchronization with backfill, aka making sure that files residing at a configured path, match of a regex, part of a whitelist, share, size etc. are synced to a designated storage (sync target), where they are accessed by consumers. This sync should be independent of modification time and trump mover configuration, aka be higher in priority, if sync settings contradict mover settings. All changes should occur on the sync target and propagate to sources. The sync target should therefore act as master data location, also propagating deletion.

3 hours ago, Gymnae said:

This sync should be independent of modification time and trump mover configuration, aka be higher in priority, if sync settings contradict mover settings. All changes should occur on the sync target and propagate to sources. The sync target should therefore act as master data location, also propagating deletion.

Yep this sound cool but don't forget this plugin called Mover tuning (not syncing), so author of plugin added synchronization some time befor leaved project.

So I'm still trying to understand how to update it to be more easy to synchronise if it possible with time and ability I have.

How it's working now for you? With your testing synchronise or Resynchronize maybe?

@alturismo do you think it's worth to update the synchronise if you tested maybe from perspective of how unraid should work at all?

3 hours ago, Gymnae said:

The sync target should therefore act as master data location, also propagating deletion.

Not sure what is meaning?

1 hour ago, Masterwishx said:

@alturismo do you think it's worth to update the synchronise if you tested maybe from perspective of how unraid should work at all?

this would be more something as a sep tool in my opinion ...

the actual "buildin" sync function (also the array > cache function) in mover tuning confusing more then helping actually ...

so we would end up in

cache > array (the main point from this plugin) which already likes to collide with

array > cache Shares ... which you solved somehow now with the target and so on ...

syncing ... already in there and also starting to shoot in from a side ...

cache > array ... i assume main goal to keep files on cache and create some sort of backup now ... that was the idea from the author before you ...

array > cache sync ... syncing which will now also collide with main function ... (will fill up cache now too ...)

so as mentioned, if you ask me, its way over the purpose of this ... and also NOT the way Unraid likes to work, roughly i also abuse it with my cache mover plugin to keep files on cache when watching "older movies, shows, ..." but its not the way i would suggest for here at all, creating dublette files on cache and array ... as a main super helpful plugin like mover tuning, most users would love to have this native implemented ...

30 minutes ago, alturismo said:

that was the idea from the author before you

Yep as I tested and wrote it's working a little strange now.

31 minutes ago, alturismo said:

so as mentioned, if you ask me, its way over the purpose of this ... and also NOT the way Unraid likes to work,

Yep that's why asked

33 minutes ago, alturismo said:

but its not the way i would suggest for here at all, creating dublette files on cache and array ... as a main super helpful plugin like mover tuning, most users would love to have this native implemented ...

As it's already avalibe maybe(not sure) I will make a little enchantments, but yes it's very confusing for this plugin. I'm still trying to understand how it should work right at all.

9 minutes ago, Masterwishx said:

I'm still trying to understand how it should work right at all.

the idea was

here was "crap"

backup the cache > array files which are kept on cache ...

lets say i have a treehold of 80 %, sync will always copy the files anyhow then to array BUT keep the files on cache instead moving them ... so they are already backed up ... BUT, this will spinup disks always and so on ...

22 minutes ago, alturismo said:

lets say i have a treehold of 80 %, sync will always copy the files anyhow then to array BUT keep the files on cache instead moving them ... so they are already backed up ... BUT, this will spinup disks always and so on ...

I got the idea from the code but not really worked for me when tested, only after many manipulations..

Not sure many users want it, not using it personally but if someone want I can make a little enchantment . Also someone opened issue in github for add synchronise to share settings...

Will see...

14 hours ago, alturismo said:

backup the cache > array files which are kept on cache ...

Wow i forgot what i explained some time ago to some users : Only modified files since last sync are synchronized. :)

17 hours ago, Masterwishx said:

Not sure what is meaning?

With master storage I meant that if a file gets deleted or manipulated on secondary storage, then the state of the file on primary storage is restored to secondary storage during a sync. Thus you avoid collisions or undeletion.

Yet having read your exchange above, I concur that convoluting plugins with feature creep creates confusion. This superb plugin could focus on tuning the vanilla mover with additional tweaks. But functionality that extends beyond the scope of mover behavior might best become part of an extra plugin

A separate sync plugin might be a good idea, it could contain the functionality which we discussed above:

1. Offers users to define affected shares, file types, file sizes, regex, paths etc. to be covered by the plugin

2. Plugin then controls on which storage data covered by the plugin configuration gets accessed, i.e. stop spinning up disks to access small files present on fast primary storage (e.g. cache) but also parity protected on slow storage (array)

3. Controls which storage acts as master storage for affected files, see above what I mean by that, aka one location replicates its state to the second location no matter what happens on that second location - this is required because we work with copies instead of vanilla moved files

4. Tells mover to create copies of data covered by the user configuration for parity protection on a secondary storage (e.g. array or different ZFS pool)

5. Supersedes mover if mover wants to move files off a storage but the configuration of the sync plugin is against it

6. Offers bidirectional sync to achieve desired state also for per-existing data from before plugin installation, but with the caveat that the mater storage location will define the state of files affected from now on. i.e. after activating a configuration and the plugin and following the initial sync, changes made to files falling under the configuartion in the secondary location will be overwritten by sync operations of the plugin

7. Could be a complementary plugin to mover tuner

Edited July 26, 20251 yr by Gymnae

21 minutes ago, Gymnae said:

A separate sync plugin might be a good idea, it could contain the functionality which we discussed above:

Yep you right, we also need developer who can make it and maintain.

I just took this plugin to maintain temporary at this year Becouse it was not working in Unraid 7.0

And was need to be fixed and no one want to get it Becouse it veeery big plugin to maintain.

So I fixed and added some futures for availability and time I have. Will try add some additional to sync if will be possible...

Also some additional plugins added and will be added in future related to moving and syncing maybe. :)

just wanted to randomly drop in and say thanks for keeping this plugin going. the development over the last year or whatever has been excellent.

2025.07.27 version Update

- new: Added a "Moved Only" notification option that allows notifications only when files are actually moved. **_(masterwishx)_**
- new: Debug added for synchronize and notification modes.
- new: Introduced a "Synchronize Primary files to Secondary" setting for shares override with cache enabled, enabling selective synchronization of modified files for backup and parity protection.

Edited July 27, 20251 yr by Masterwishx

14 hours ago, Masterwishx said:

- new: Introduced a "Synchronize Primary files to Secondary" setting for shares override with cache enabled, enabling selective synchronization of modified files for backup and parity protection.

Does this feature affect the entire share, meaning that all files of the share will reside then on both primary and secondary, or does it obey configured skipped files types and skipped file lists? Aka would this feature enable the use case of having files with certain parameters like age, size, file type, path etc. moved to secondary but keep others on both primary and secondary?

31 minutes ago, Gymnae said:

Does this feature affect the entire share, meaning that all files of the share will reside then on both primary and secondary

This sync should work exactly as for global was but can be sync for share only when global sync is off

Edited July 28, 2025Jul 28 by Masterwishx