Did a quick search and didn't find anything.

 

"Let scheduled mover run during a parity check / rebuild:" is set to "No" for me, but the mover still ran during my parity check. It all worked fine so no big deal, but just strange.

In Mover Tuning, I also have "Pause array operations while mover running: Yes". Wonder if that's affecting it. I'll mess around with the settings and see if that changes anything.

 

Thanks for keeping this updated

@Masterwishx

 

Any chance we could get a fix or option to change how the plugin handles hardlinks? When running mover tuning plugin I frequently get messages like this for files originating from Qbittorrent:

 

Expected 2 files, got 1. Not moving "/mnt/m2cache/./media/Library/*RANDOM_FILE_NAME*"  to prevent breaking hardlinks

 

And this happens even to files that are no longer being monitored or used in Qbit, and files that originate from usenet are handled and moved to array without issue 100% of the time. Furthermore, this is only an issue with the plugin's mover operation, a standard system mover operation moves the above mentioned files from cache to array just fine.

 

Thanks for picking up the torch and keeping this plugin alive and well. Your time and effort are massively appreciated by the community.

Seems there are three versions of this plugin in the CA apps. I assume we should use the latest one but perhaps the other two need to be removed?

 

 

10 hours ago, applesauce9 said:

Thanks for keeping this updated

I will check it

 

6 hours ago, Notwist said:

change how the plugin handles hardlinks?

Will check it

 

 

The ignore list doesn't seem to work properly

First the files are skipped and then they are copied anyway.

 

My mover ignore list:

/mnt/cache/nextcloud/appdata_XXXXX
/mnt/cache/nextcloud/.htaccess
/mnt/cache/nextcloud/.ncdata
/mnt/cache/nextcloud/index.html
/mnt/cache/nextcloud/nextcloud.log
/mnt/cache/Downloads

 

 

 

/tmp/ca.mover.tuning/Filtered_files_2025-02-19T124049.list

 

cache|cache|Downloads|skipped|9999999999|0|57344|0|0|0|/mnt/cache/Downloads/*
cache|cache|nextcloud|skipped|9999999999|0|1024|0|0|0|/mnt/cache/nextcloud/index.html
cache|cache|nextcloud|skipped|9999999999|0|1024|0|0|0|/mnt/cache/nextcloud/nextcloud.log
cache|cache|nextcloud|skipped|9999999999|0|4080218932|0|0|0|/mnt/cache/nextcloud/appdata_XXXXX/*
cache|cache|nextcloud|skipped|9999999999|0|5120|0|0|0|/mnt/cache/nextcloud/.htaccess
cache|cache|nextcloud|skipped|9999999999|0|5120|0|0|0|/mnt/cache/nextcloud/.ncdata

 

cache|user0|nextcloud|yes|1739608682|0|0|0|1|145562|/mnt/cache/nextcloud/nextcloud.log
cache|user0|nextcloud|yes|1739522147|0|0|0|1|145236|/mnt/cache/nextcloud/index.html
cache|user0|nextcloud|yes|1739522147|0|542|9|1|145482|/mnt/cache/nextcloud/.htaccess
cache|user0|nextcloud|yes|1739522147|0|52|98|1|145792|/mnt/cache/nextcloud/.ncdata

 

Feb 19 12:55:45 Server move: ----------------------------------------------------------- Processing nextcloud share ----------------------------------------------------------
Feb 19 12:55:45 Server move: Primary storage: cache - size: 1.8TiB - used: 32 % (599GiB)
Feb 19 12:55:45 Server move: Secondary storage: user0
Feb 19 12:55:45 Server move: Share Information: Name: nextcloud - Path: /mnt/cache/nextcloud
Feb 19 12:55:45 Server move: Moving threshold: 10% (180GiB) ; Freeing threshold: 0% (0B)
Feb 19 12:55:45 Server move: Mover action: cache->user0 (cache:yes). Pool is above moving threshold percentage:  32% >= 10%.
Feb 19 12:55:45 Server move: => Will smart move old files from cache to user0. Nothing will be moved from user0 to cache
Feb 19 12:55:45 Server move: Skipping Files from List. File size are taken into account in the calculation of the threshold
Feb 19 12:55:45 Server move: List Path: /mnt/cache/appdata/mover_ignore.txt:
Feb 19 12:55:46 Server move:     Ignored files are using 3.9GiB
Feb 19 12:55:48 Server move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-02-19T125544.list for nextcloud
Feb 19 12:55:48 Server move: ------------------------------------------------------------- Processing system share -----------------------------------------------------------

Feb 19 12:55:48 Server move: *********************************************************** LET THE MOVING SHOW BEGIN ! *********************************************************
Feb 19 12:55:48 Server move: Moving "/mnt/cache/./nextcloud/nextcloud.log"  to  /mnt/user0/ 
Feb 19 12:55:48 Server move: 3 files remaining from caches to array  594B
Feb 19 12:55:48 Server move: Moving "/mnt/cache/./nextcloud/.htaccess"  to  /mnt/user0/ 
Feb 19 12:55:48 Server move: 2 files remaining from caches to array  52B
Feb 19 12:55:48 Server move: Moving "/mnt/cache/./nextcloud/index.html"  to  /mnt/user0/ 
Feb 19 12:55:48 Server move: 1 files remaining from caches to array  52B
Feb 19 12:55:48 Server move: Moving "/mnt/cache/./nextcloud/.ncdata"  to  /mnt/user0/ 
Feb 19 12:55:48 Server move: 0 files remaining from caches to array  0B
Feb 19 12:55:48 Server move: ------------------------------------------------------------------- Cleaning up -----------------------------------------------------------------

 

Edited February 19, 20251 yr by Tufinger

16 hours ago, Gex2501 said:

Seems there are three versions of this plugin in the CA apps. I assume we should use the latest one but perhaps the other two need to be removed?

 

 

Thanks for pointing this out! apparently i had the oldest one installed. I never realized there were multiple apps in the store. I haven't had any issues until i updated to unRaid 7.0, now the log was filling up regularly. I just changed the one installed, but that has seemed to fix my issue so far.

 

Thanks again for pointing this out!

It doesn't seem like the primer move is working at all. I see this message in the summary file:

 

16:52:57.037 Primary storage: cache_download - size: 239GiB - used:   1 % (5.8MiB)
16:52:57.038 Secondary storage: user0
16:52:57.041 Share Information: Name: media - Path: /mnt/*/media
16:52:57.044 Moving threshold: 80% (191GiB) ; Freeing threshold: 50% (120GiB)
16:52:57.045 Mover action: user0->cache_download (cache:prefer). Pool is below priming threshold percentage:   1% < 50%.
16:52:57.046 => Will smart move newest files from user0 to cache_download until threshold. Older files will be moved from cache_download to user0.
16:52:57.048 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-02-19T165256.list for media

 

But no files were moved to the cache_download pool.

 

I did notice it has a tiny bit more in the syslog message:

Feb 19 17:17:08 falcon move: ------------------------------------------------------------- Processing media share ------------------------------------------------------------
Feb 19 17:17:08 falcon move: Primary storage: cache_download - size: 239GiB - used:  32 % (75GiB)
Feb 19 17:17:08 falcon move: Secondary storage: user0
Feb 19 17:17:08 falcon move: Share Information: Name: media - Path: /mnt/*/media
Feb 19 17:17:08 falcon move: Moving threshold: 80% (191GiB) ; Freeing threshold: 80% (191GiB)
Feb 19 17:17:08 falcon move: Mover action: user0->cache_download (cache:prefer). Pool is below priming threshold percentage:  32% < 80%.
Feb 19 17:17:08 falcon move: => Will smart move newest files from user0 to cache_download until threshold. Older files will be moved from cache_download to user0.
Feb 19 17:17:08 falcon move: find: '/mnt/*/media': No such file or directory
Feb 19 17:17:08 falcon move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-02-19T171708.list for media

 

Note the error about finding the media folder. Maybe it isn't getting the right output of files?

Edited February 20, 20251 yr by tehg
more info

45 minutes ago, tehg said:

Feb 19 17:17:08 falcon move: ------------------------------------------------------------- Processing media share ------------------------------------------------------------
Feb 19 17:17:08 falcon move: Primary storage: cache_download - size: 239GiB - used:  32 % (75GiB)
Feb 19 17:17:08 falcon move: Secondary storage: user0
Feb 19 17:17:08 falcon move: Share Information: Name: media - Path: /mnt/*/media
Feb 19 17:17:08 falcon move: Moving threshold: 80% (191GiB) ; Freeing threshold: 80% (191GiB)
Feb 19 17:17:08 falcon move: Mover action: user0->cache_download (cache:prefer). Pool is below priming threshold percentage:  32% < 80%.
Feb 19 17:17:08 falcon move: => Will smart move newest files from user0 to cache_download until threshold. Older files will be moved from cache_download to user0.
Feb 19 17:17:08 falcon move: find: '/mnt/*/media': No such file or directory
Feb 19 17:17:08 falcon move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-02-19T171708.list for media

 

Note the error about finding the media folder. Maybe it isn't getting the right output of files?

Sorry for double posting.

 

It looks like the issue is on line 525 in the age mover: https://github.com/masterwishx/ca.mover.tuning/blob/06a0061ecbe915c7afc8d508de2480fc3fe8e311/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover#L525C1-L527C11

 

If you have prefer cache or rebalance shares, it sets the SHAREPATH to "/mnt/*/$SHARENAME". That never seems to actually get changed to a valid find format that I can tell. I'm far from a good shell scripter though, so I could be wrong.

6 hours ago, tehg said:

Sorry for double posting.

 

It looks like the issue is on line 525 in the age mover: https://github.com/masterwishx/ca.mover.tuning/blob/06a0061ecbe915c7afc8d508de2480fc3fe8e311/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover#L525C1-L527C11

 

If you have prefer cache or rebalance shares, it sets the SHAREPATH to "/mnt/*/$SHARENAME". That never seems to actually get changed to a valid find format that I can tell. I'm far from a good shell scripter though, so I could be wrong.

 

I will check it

20 hours ago, Tufinger said:

The ignore list doesn't seem to work properly

First the files are skipped and then they are copied anyway.

i know need to fix it

On 2/20/2025 at 4:02 AM, tehg said:

Sorry for double posting.

 

It looks like the issue is on line 525 in the age mover: https://github.com/masterwishx/ca.mover.tuning/blob/06a0061ecbe915c7afc8d508de2480fc3fe8e311/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover#L525C1-L527C11

 

If you have prefer cache or rebalance shares, it sets the SHAREPATH to "/mnt/*/$SHARENAME". That never seems to actually get changed to a valid find format that I can tell. I'm far from a good shell scripter though, so I could be wrong.

It has another if check after this line

But still checking code... 

On 2/18/2025 at 2:56 PM, Masterwishx said:

If you set threshold to 0% and set age it's not working? 

Going to try this now and will let you know... However if you could implement something like Simple Mover where we can have both but prefer age over space that would be great... I know someone will say just use simple mover but I use other things this has as well.

On 2/16/2025 at 11:22 PM, alturismo said:

because there was a extra feature added ... which is very useful ...

 

before >> only run when treehold (fillrate from your cache) was upper lets say 50 % (whatever you have chosen) and move ...

change >> also only run when treehold (fillrate) was upper lets say 50 % but you set a 2nd value to stop moving !

 

before >> just move everything when limit was reached ...

later >> just move the "difference" to keep your other (most common newer) files still on cache

 

what is so hard to understand on this feature ? in your upper sample. 95 95 ... you will play ping pong and its not mentioned to be setup like this

 

WHEN > 95 % is reached >> start moving ...

WHEN < 95 % is reached >> stop moving ...

 

so a single file could start the mover and stop it again ...

 

what is a real usecase

 

WHEN > 90 % >> start moving (based on age as sample)

WHEN < 60 % >> stop moving

 

so ~ 30% (90 - 60) will then get moved from cache to array, the ~ 60 % will be kept on cache ... based on age of files ...

 

now, just set a low value in terms you want to move almost anything to the array always, like the 5 % ((yes, i know its 5 and not 0), but hey ... dont you have anything kept on cache like Dockers, Isos, Domain Images, ... which are easily already filling up those 5 % anyhow ? and set your start treehold also low like 10, 20, 30, ... whatever ... whenever you want the mover to start ...

 

 

I have nothing but downloads on mine... I have 2 cache pools... one for download and one for everything else.  So that is not always the case.

1 minute ago, slikone27 said:

I have nothing but downloads on mine... I have 2 cache pools... one for download and one for everything else.  So that is not always the case.

then set it like described above to 5% and your age setting and you should be fine

2 hours ago, alturismo said:

then set it like described above to 5% and your age setting and you should be fine

We can set now 0% also... 

2 hours ago, slikone27 said:

However if you could implement something like Simple Mover where we can have both but prefer age over space that would be great... I know someone will say just use

Sorry I don't know how this version of plugin was worked on 6.12, used the old one (2023). Only using it after updated to unraid 7.0 about some weeks ago. 

 

I am setting up the mover feature in Unraid for the first time and want to configure it to fit my setup. I have 4x 4TB SSDs in a ZFS RAIDZ1 and multiple user shares (TV, Movies, Documentaries, etc.).  

 

I’d like to:  

- Only include specific folders within shares in the cache (avoid caching random old episodes).  

- Move data off the cache once it reaches a certain %, prioritizing older files.  

 

I understand you can exclude files from being moved off the cache via exclude file list but is there a way to exclude them from being moved onto the cache instead? Can this setup be achieved with the current plugin by Masterwishx?

What is the difference that simple mover offers.

Basically I want to only include certain shows / movies on the cache. How are you guys prioritizing your Plex/ Jellyfin content.

 

Thanks

2 hours ago, KeyBoardDabbler said:

but is there a way to exclude them from being moved onto the cache instead? Can this setup be achieved with the current plugin by Masterwishx?

 

nope, a mover runs only on a certain time (schedule) and does its job to free cache / pool drives, then you have mover tuning where you can define some rules when it shell do this , which and how much and ...

 

but how your files are coming into the cache, is a different story

 

share setup while uploading files to your Shares

other plugins like my cache mover to handle media files for watching

whatever ...

 

3 hours ago, KeyBoardDabbler said:

- Only include specific folders within shares in the cache (avoid caching random old episodes).  

 

so No, thats your part how you handle uploads for those files ...

 

3 hours ago, KeyBoardDabbler said:

- Move data off the cache once it reaches a certain %, prioritizing older files.  

 

so Yes, thats what mover tuning is mainly about ...

 

sense of a onboard mover is to free the cache again, plugins like mover tuning to "fine tune" to fit your needs how long you want the already cached files keep on there ...

 

3 hours ago, KeyBoardDabbler said:

What is the difference that simple mover offers.

 

well, actually i wrote myself a "simple mover" plugin, which also does basically the same but doesnt run on the buildin schedules, doesnt include into the standard UI mover page > standalone setup page

 

why, as this plugin had many issues and i needed a working solution to play fine with my cache mover plugin (caching Media Files from array to cache for watching pretty silent when playing older content from array disks)

 

there is no intention from my side (simple mover) for a race condition with mover tuning plugin

1st, this plugin is more advanced and has pretty neaat extra features then simple mover

2nd, once @Masterwishx has reverse read up all the code and fixed the issues you have all you need inside here

3rd, simple mover was since yesterday only working properly "side by side" with mover tuning as

- i never disabled the onboard mover

- therefore "abused" the mover tuning exclusion feature to let handle simple mover Shares its stuff and exclude them from mover tuning ... i kept the exclusion list updated like cache mover also does

 

yes, i added a option now to disable onboard mover, but my setup is pretty simple

i use for moving features only Media Files ... so my code is way more simple then this here ...

 

so may rather wait a little until this is on point ... then we have a more "official" integration then a standalone plugin, and, it also wont handle your incoming files which you may dont even want on your Cache

How are we handling the Unraid Patch 1.0.0 when we have the Mover plugin installed? As the offical patch fails to install right now.

30 minutes ago, nebb00 said:

How are we handling the Unraid Patch 1.0.0 when we have the Mover plugin installed? As the offical patch fails to install right now.

Reboot

Hello,

 

I've been using the plugin for a few months now, so far everything has been great. Today, however, I had the interesting problem that the plugin filled up my rootfs until nothing worked anymore. In the logs I found the hint that the plugin moved data from "/mnt/user/media" to "/mnt/none". However, there is no share with the name "none". The logs showed that the secondary target is "none", which is correct. However, I don't know how the plugin comes up with the idea of creating a "none" folder under /mnt/.

 

As I was still able to access the server via SSH and move the data back to the correct location, there was fortunately no data loss. I was also able to restart nginx and access the WebUI directly again. I then carried out a clean restart to be on the safe side and unfortunately forgot to save the logs.

 

I would like to use the plugin again. Does anyone have any idea what the problem could be?

33 minutes ago, Gorosch said:

Hello,

 

I've been using the plugin for a few months now, so far everything has been great. Today, however, I had the interesting problem that the plugin filled up my rootfs until nothing worked anymore. In the logs I found the hint that the plugin moved data from "/mnt/user/media" to "/mnt/none". However, there is no share with the name "none". The logs showed that the secondary target is "none", which is correct. However, I don't know how the plugin comes up with the idea of creating a "none" folder under /mnt/.

 

As I was still able to access the server via SSH and move the data back to the correct location, there was fortunately no data loss. I was also able to restart nginx and access the WebUI directly again. I then carried out a clean restart to be on the safe side and unfortunately forgot to save the logs.

 

I would like to use the plugin again. Does anyone have any idea what the problem could be?

What version of the plugin? 

Hi, I recently installed the latest version of this from @Masterwishx (2025.02.18.1752) after upgrading to Unraid 7.  However, I am having an issue when the script processes my ignore list. 

I have a script that runs every couple of hours to check Plex OnDeck and Watchlists, and moves those files to the cache, and then adds the file names to an ignore list, so they dont get moved back. 

However, this version of mover can't deal with filenames that have single quotes in them.  For example, 

/mnt/hddcache/media/TV/8 Out of 10 Cats Does Countdown (2013) {imdb-tt2345459}/Season 07/8 Out of 10 Cats Does Countdown (2013) - S07E11 - Mel Giedroyc Lee Mack David O'Doherty Adam Buxton [WEBDL-1080p][AAC 2.0][x264]-NAF.mkv

To resolve this, I have changed the following lines in my age_mover file:

Line 666:

 

FINDSTR="$FINDSTR -not -path '${skipped_path}'"

to

FINDSTR="$FINDSTR -not -path \"${skipped_path}\""

 

Line 670:

FINDSTR="$FINDSTR -not -name '${skipped_path}'"

to

FINDSTR="$FINDSTR -not -name \"${skipped_path}\""

This seems to resolve the issue, as the strings in the subsequent find command are now double quotes.

I don't know if this is something that can be incorporated into the plugin itself, or not, but I thought I'd fire some feedback over.

Edited February 23, 20251 yr by JayBriers

5 hours ago, Masterwishx said:

What version of the plugin? 

 

The latest. On Unraid V7

 

Edited February 23, 20251 yr by Gorosch

On 2/22/2025 at 2:31 AM, Masterwishx said:

We can set now 0% also... 

I have set it to 0% and 10 days and it isn't running at all now.  I think setting it to 0% is basically disabling mover right now.  Would have been great if it did work but if anyone can make this one function w/ an option to prefer age and then run on % next, that would be amazing... I sometimes download complete old shows that are very large... so basic functionality I like leaving anything under 10 days on the cache drive for fast starts (great for new episodes of shows).  Then when downloading multiple big shows with lots of seasons, having the mover run when the drive hits 80% so it won't completely fill up and run out of room.

 

Just to clarify I had it set at 0% to 0%, to try and run just on age for now.

Edited February 23, 20251 yr by slikone27