[Plugin] Mover Tuning

alturismo · January 11

21 minutes ago, MadMatt337 said:

I had the setting in mover tuning setup to "Move All from Cache-yes shares pool percentage:" at 80% it did not enable the move function automatically.

the mover is scheduled, mover tuning is a "filter" WHEN the mover is running ...

so when you use a 80 % filter ... this will jump in ONLY when the mover is triggered ...

it WONT watch the fillrate in real time and trigger the mover ... so rather increase your schedule triggers (like hourly) and the mover will start each hour, IF the fillrate is 80+ it ll move, otherwise not ...

huggie · January 11

Just wanted to say what a great plugin this is

I see you can run scripts before and after the mover is enabled, does anyone have a decent script that stops torrents or the container before the move so they can be moved and then restart them after?

Thankyou

huggie · January 16

me again, i set up Mover Tuning to move files that are over 20days old. That way I can keep new stuff on the cache and move it off after its been watched and seeded.

Problem is it doesn't seem to be working. I'm getting this error in the logs.

root: Log Level: 1
root: /tmp/Mover Directory does not exist. Creating it
root: mover: started
root: df: /mnt/cache: No such file or directory
root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 687: [: -gt: unary operator expected
root: mover: finished
emhttpd: spinning down /dev/sdb
emhttpd: read SMART /dev/sdb

Any idea what is going on here? I've tried deleting this plugin and installing again but it still does it.

brandonhowlett · January 16

Haven't made any changes recently. Noticed this spamming the syslog today and my cache drives were full.

Move files off cache based on age has always been set to no.

move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")

Edited January 16 by brandonhowlett

Swarles · January 17

14 hours ago, huggie said:

root: df: /mnt/cache: No such file or directory
root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 687: [: -gt: unary operator expected

Any idea what is going on here? I've tried deleting this plugin and installing again but it still does it.

Can you confirm if you have a cache pool called "cache" and if the location "/mnt/cache" exists?

My first thoughts are that your cache pool is not called that but you may have some left over cache pool config files located at "/boot/config/pools". Check that location and see if all your pools are correct.

Swarles · January 17

2 hours ago, brandonhowlett said:

Haven't made any changes recently. Noticed this spamming the syslog today and my cache drives were full.

Move files off cache based on age has always been set to no.

move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")

Not 100% sure but it seems like you might be getting an invalid file size for some of the files that are performing an array>cache move. Could you check the contents of your "/tmp/Mover/Custom_Cache_Tuning_$NOW.list" file, you may have some entries that are missing a number at the end.

huggie · January 17

16 hours ago, Swarles said:

Can you confirm if you have a cache pool called "cache" and if the location "/mnt/cache" exists?

My first thoughts are that your cache pool is not called that but you may have some left over cache pool config files located at "/boot/config/pools". Check that location and see if all your pools are correct.

My cache pool was renamed to cache_nvme. would this mess it up?

Swarles · January 17

4 hours ago, huggie said:

My cache pool was renamed to cache_nvme. would this mess it up?

The mover tuning plugin uses the cache pool configuration files located in "/boot/config/pools" to determine what pools exist and their settings, check to see if you have a file called "cache.cfg" in there in addition to your "cache_nvme.cfg" file.

Let me know if you don't have one, or if you only have "cache.cfg" check inside it to see what the cache name reports back.

huggie · January 18

11 hours ago, Swarles said:

The mover tuning plugin uses the cache pool configuration files located in "/boot/config/pools" to determine what pools exist and their settings, check to see if you have a file called "cache.cfg" in there in addition to your "cache_nvme.cfg" file.

Let me know if you don't have one, or if you only have "cache.cfg" check inside it to see what the cache name reports back.

Right theres a cache.nvme.cfg file but no cache.cfg file.

Swarles · January 18

2 hours ago, huggie said:

Right theres a cache.nvme.cfg file but no cache.cfg file.

Can you confirm if the path that exists is,

/mnt/cache.nvme
or
/mnt/cache_nvme

It could be that perhaps the code is treating the ".nvme.cfg" as the full extension and removing it, but I have checked this and it should not be the case. But for a better understanding you should provide some logs.

Go to Settings>Scheduler>Mover Settings
Set "Mover Logging" to Enabled.
Go to the Mover Tunning settings and set "Test Mode" to "Yes"
Set "Move Now button follows plug-in filters" to "Yes".
On the Main tab press the "Move" button to run the mover. Running in test mode will generate logs but not make moves.
Go to "/tmp/Mover" and get the most recent file ending in ".log"
You can revert test mode and logging now.

You can supply the .log contents here in a spoiler or send them to me privately. This might help me understand why you are experiencing behaviour that is unexpected.

huggie · January 18

23 minutes ago, Swarles said:

its cache_nvme.cfg

The logs are exactly the same that I posted a few posts up. But I'll run it again and send it to you in a PM

ApfelBirneKreis · January 18

I want to shutdown docker services with a script and enable it after the mover ran again.

how do i run the script? Is it in bash? whats the filename? Is there any documentation?

Texot · January 20

On 1/17/2024 at 8:25 AM, Swarles said:

Not 100% sure but it seems like you might be getting an invalid file size for some of the files that are performing an array>cache move. Could you check the contents of your "/tmp/Mover/Custom_Cache_Tuning_$NOW.list" file, you may have some entries that are missing a number at the end.

There is a bug in age_mover line 368 reading CUSTOMLINE while in 370 it's referring to CUSTOMFILE

Swarles · January 21

9 hours ago, Texot said:

There is a bug in age_mover line 368 reading CUSTOMLINE while in 370 it's referring to CUSTOMFILE

Oh yep, that'll do it, fixed now. Should be fixed in the next update. Thank you for spotting that

Autchirion · January 25

Somehow it seems to me, that moving when the cache is full isn't working.

I've got a share with the "Move All from Cache-Yes shares when disk is above a certain percentage" yes.
Two caches (Naming Cache and Datacache), where for this share the Datacache (~120GB) is used
I'm downloading ~100 files with ~2GB each, files are beeing pre generated (~1KB) and then after this filled with data. -> Mover isn't being activated when Datacache is 80% full.

I'm not sure if I set up something wrong or if this usecase isn't covered by mover tuning.

[EDIT] since it sounded similar, cache.cfg and datacache.cfg exist, I'm not sure if it is case sensitive, because the filename is lowercase, but the first letter of the Cache Name in the UI is uppercase.

Mover Tuning - Share settings:

Mover Tuning Settings:

image.png.f5bc5702bb412015b87447c7324ab2ee.png

Edited January 25 by Autchirion

alturismo · January 25

34 minutes ago, Autchirion said:

Mover isn't being activated when Datacache is 80% full.

the mover is running scheduled ... so the trigger is ONLY time based ...

WHEN its triggered it ll follow the rules

so, did you set the schedule and waited for it to run (supposed behaviour) ?

or did you expect the mover kicks in automatically while reaching the fill rate (wont happen) ?

Autchirion · January 25

3 minutes ago, alturismo said:

or did you expect the mover kicks in automatically while reaching the fill rate (wont happen) ?

yes, I expected this, damn it. 🙂

alturismo · January 25

2 minutes ago, Autchirion said:

yes, I expected this, damn it. 🙂

then may read a littl ebackwards or overall how it works.

sample (same page ... here)

so yes, you where not alone ... but may also read the manual again

in my case, i just inreased the scheduler to run hourly ... as i control all by mover tuning.

Autchirion · January 25

Yeah, my Problem is, cache is only 120GB, if I set it to e.g. move only at 80% I'd need to run it every 3.4 Minutes (assuming 1Gbit/s upload rate which is my home network speed) to make sure it never fills up.

Kilrah · January 25

14 minutes ago, Autchirion said:

I'd need to run it every 3.4 Minutes (assuming 1Gbit/s upload rate which is my home network speed) to make sure it never fills up.

Then disable cache for those shares, it's completely pointless to use it.

It's useful if you're using it to absorb bursts of data that are coming in faster than the array can take or to store things on for a few days, but if you have to empty it continuously you don't gain anything.

Edited January 25 by Kilrah

itimpi · January 25

3 hours ago, Autchirion said:

Yeah, my Problem is, cache is only 120GB, if I set it to e.g. move only at 80% I'd need to run it every 3.4 Minutes (assuming 1Gbit/s upload rate which is my home network speed) to make sure it never fills up.

It is worth pointing out that if you have Turbo Write enabled you can write directly to the array at speeds of that order of magnitude. Probable a better way to handle your requirements anyway.

Autchirion · January 25

It is made to handle bursts, but sometimes these bursts are to big in size to be handled by the cache. I was hoping to find a solution without throwing money (aka bigger cache, not reusing my first SSD) at it.

Kilrah · January 25

Well it's either accept the temporary slowdown or shell out for bigger drives, no miracle solution...

Schaka · January 28

First of all, I'm sorry for hijacking this thread. I tried to find a solution in other threads, but everything seems outdated. I'm still on 6.11.5.

I was looking to manually trigger the mover via cli/script and expected it to be blocking, so that I may shut down docker containers before running it so files won't be in use.

There seem to be two problems where calling mover via ssh just seems to do nothing (no disk activity), using the GUI it does run but then calling mover stop does nothing again.

What is the correct way (maybe the way this plugin does it) to call the mover? Can I just call one of the scripts the plugin uses and it will be blocking? I'd love to utilize the cache again, but unless I can get the mover to run after my nightly backup, I don't see another way of doing this atm.

Edited January 28 by Schaka

Revan335 · January 28

58 minutes ago, Schaka said:

it to be blocking

Any Error/Message do you become?

[Plugin] Mover Tuning

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid

hugenbdd

hugenbdd

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation