[Plugin] Mover Tuning


Recommended Posts

16 hours ago, Nodiaque said:

Oh, I guess I understood this plugin wrong. Is that just a set of rule that dictate what happen when the miover is ran? Because I though that pool percentage was a trigger, like when that is hit, the mover is ran... Is there a plugin that does that?

No plug-in that I know that does that. but....  You can get pretty close with this plug-in.  Set mover to run every hour or so, and set your percentage.  It checks percentage on each run and then determines if it's over, to move files.

Link to comment
15 hours ago, HDT said:

Is there a way for MOVER to move files by oldest first until x % of space is free?

 

Not by age, like any file over 9 days old, but by moving the oldest file first out of all the files in the cache pool until the cache pool has  x % of space free.

No.

The closest you can come with this plug-in is to play with your days old and % free to get close to the amount of space you want to keep filled up on the cache.

Link to comment
  • 2 weeks later...
On 2/4/2021 at 10:56 PM, hugenbdd said:

Few things.

There might be a hidden ^M character in your exclusion file if you created/edited it in windows.  Either remove inside vi editor, try dos2unix (may not work in unraid), or  sed -e "s/^M//" filename > newfilename

 

Try just the find command on it's own from the console and see what it returns. (It should not have the files in the exclude file in the output).

find "/mnt/cache/Backup/" -depth | grep -vFf '/boot/moverignore.txt'

Hi,

 

I have a similar problem as he had and tried to solve it with these tips, but nothing worked and I don't have any idea what the problem could be.

 

I've configured a moverignore.txt that contains the following:

/mnt/cache/Docker/Logging/Pi-Hole/pihole.log
/mnt/cache/Docker/Logging/TeamSpeak/ts3server_0.log
/mnt/cache/Docker/Logging/TeamSpeak/ts3server_1.log
/mnt/cache/Docker/Logging/Traefik/access.log
/mnt/cache/Docker/Logging/Traefik/traefik.log
/mnt/cache/Docker/Logging/Filebot/filebot.log
/mnt/cache/Backups/Docker/Wordpress/Uploads/uploads.ini
/mnt/cache/Downloads/Temp/*
/mnt/cache/Docker/Logging/SmartHome/home-assistant.log
/mnt/cache/Docker/Logging/Matrix/matrix-server.log
/mnt/cache/Docker/Compose/Matrix/shared_secret_authenticator.py
/mnt/cache/Docker/Logging/Paperless/paperless.log

 

That's the out output form the log file:

Jun 23 05:35:25 Media-Server root: mover: started
Jun 23 05:35:27 Media-Server move: file: /mnt/cache/Backups/Docker/Wordpress/Uploads/uploads.ini
Jun 23 05:35:29 Media-Server move: skip: /mnt/cache/Docker/Logging/TeamSpeak/ts3server_0.log
Jun 23 05:35:29 Media-Server move: skip: /mnt/cache/Docker/Logging/TeamSpeak/ts3server_1.log
Jun 23 05:35:30 Media-Server move: skip: /mnt/cache/Docker/Logging/Pi-Hole/pihole.log
Jun 23 05:35:30 Media-Server move: skip: /mnt/cache/Docker/Logging/SmartHome/home-assistant.log
Jun 23 05:35:30 Media-Server move: skip: /mnt/cache/Docker/Logging/Traefik/access.log
Jun 23 05:35:31 Media-Server move: skip: /mnt/cache/Docker/Logging/Traefik/traefik.log
Jun 23 05:35:31 Media-Server move: skip: /mnt/cache/Docker/Logging/Matrix/matrix-server.log
Jun 23 05:35:31 Media-Server move: file: /mnt/cache/Docker/Logging/Paperless/paperless.log
Jun 23 05:35:32 Media-Server move: file: /mnt/cache/Docker/Compose/Matrix/shared_secret_authenticator.py
Jun 23 05:35:32 Media-Server root: mover: finished

 

Why is the mover ignoring just some files and not all? Does anyone have an idea what I can try?

moverignore.txt

Link to comment

Hi, thanks for the plugin,

 

I was wandering if it was possible as a future release feature to custom choose the "Move files that are larger than this size (In M)." option, 1024 MB is a little low for me, I'd like to move files larger than 2 GB.

 

If not than it's no problem.

 

Thanks

Link to comment
On 6/27/2022 at 3:07 AM, giosann said:

Hi, thanks for the plugin,

 

I was wandering if it was possible as a future release feature to custom choose the "Move files that are larger than this size (In M)." option, 1024 MB is a little low for me, I'd like to move files larger than 2 GB.

 

If not than it's no problem.

 

Thanks

+1 for this feature.  I would like to always move my UHD video files.  <20GB would be my request.

 

Thanks,

Jason

Link to comment
On 6/9/2022 at 2:11 PM, HDT said:

Is there a way for MOVER to move files by oldest first until x % of space is free?

 

Not by age, like any file over 9 days old, but by moving the oldest file first out of all the files in the cache pool until the cache pool has  x % of space free.

 

I have my system setup to check every day in the AM, look for at least 10% used and move files that are 20 days old. 

 

So in a sense it does move the oldest files first the 20day files and then keeps moving the oldest every day

  • Thanks 1
Link to comment

Do the 'share' specific move buttons follow the mover tuner settings when they are run manually and the 'Move Now button follows plug-in filters' is enabled? I have a downloads/incomplete folder filtered out with a text file, so files are never moved out while they are downloading and processing, but using the 'share' move button seems to try and move the incomplete files over regardless. I assumed it would follow the plugin filters, since the setting was turned on, but does only the 'Move Now' button on the  'Main' tab follow the rules?

Link to comment
On 7/1/2022 at 2:13 PM, kizer said:

 

I have my system setup to check every day in the AM, look for at least 10% used and move files that are 20 days old. 

 

So in a sense it does move the oldest files first the 20day files and then keeps moving the oldest every day

 

May you please share a screenshot of your settings?

Link to comment
On 7/7/2022 at 3:42 PM, HDT said:

 

May you please share a screenshot of your settings?

 

Sure when I get a chance later tonight when I get home I'll take a snap of what I have going on. Its nothing really special, but I know sometimes a little bit of info goes a long way when your trying to explain something or understand something. 

Link to comment

For some reason, I can't get the mover to run at 70 % threshold, and if the files are more than 10 days old.
I'm sure I made a mistake somewhere, I just can't find it.

 

Here's a picture of my settings:

image.thumb.png.f4d6f4c8ef2f2d02d6787fc72a1a5931.png

 

Can someone tell me where I got it wrong?

Link to comment
2 hours ago, alturismo said:

it is generally looking all fine, we talk about a manual test scenario ? as you disabled automation more or less ...

 

image.thumb.png.f9282aa6f3e7f692053ea98de0281ef2.png

I've tried both Yes and No in the one you highlited, it doens't move it when it's 70 % full :/

Link to comment
2 hours ago, Nanobug said:

Which one? The original mover, or the mover tuner?

 

I posted my exact settings up above a bit. I also fixed what I posted a couple days ago that you qouted me on. You do not want the top line to be YES, Yes means disable the plugin. The last line setting it to yes means enable TEST mode. You want things to move, you have to use NO on the last line too. 

 

 

On a side note you can enable Logging and click the button to move and watch files move while watching the logs using Test mode. Or you can disable TEST mode and watch the logs as well. I normally enable TEST and Logging so I can see what the script would do and when I'm happy turn off TEST and let it go. 

 

I used to keep logging and check things, but I've since disabled Logging and have my files move once a day vs every hour because I have my server setup to move files, do Plex maintenance and other things between the hours of 6AM and 8AM every morning. Otherwise my drives sleep all day. 

 

Link to comment
On 7/18/2022 at 12:31 AM, kizer said:

 

I posted my exact settings up above a bit. I also fixed what I posted a couple days ago that you qouted me on. You do not want the top line to be YES, Yes means disable the plugin. The last line setting it to yes means enable TEST mode. You want things to move, you have to use NO on the last line too. 

 

 

On a side note you can enable Logging and click the button to move and watch files move while watching the logs using Test mode. Or you can disable TEST mode and watch the logs as well. I normally enable TEST and Logging so I can see what the script would do and when I'm happy turn off TEST and let it go. 

 

I used to keep logging and check things, but I've since disabled Logging and have my files move once a day vs every hour because I have my server setup to move files, do Plex maintenance and other things between the hours of 6AM and 8AM every morning. Otherwise my drives sleep all day. 

 

 

I tried setting it up with the picture you referred to, but now the mover is not moving at all, not even manually.

 

I can't figure out why I'm having problems with it. Everything else is running perfectly, Unraid is super stable for me and it's all good dancing on rainbows and all that, but the mover/cache just refuses to work...

Link to comment

Is there actually something left to move?

Not totally clear how the Age thing works (if it's AND or OR), but if it's "only move files that are more than X days old" and behaves as AND then your setup to move if more than 70'% is used would only do something if there are files that are >10 days old to move. 

Edited by Kilrah
Link to comment
2 hours ago, Kilrah said:

Is there actually something left to move?

Not totally clear how the Age thing works (if it's AND or OR), but if it's "only move files that are more than X days old" and behaves as AND then your setup to move if more than 70'% is used would only do something if there are files that are >10 days old to move. 

I've disabled it, and tried to set it to every hour from the regular mover, and also tried manually. And yes, there's 700 GB of data to move.

 

Link to comment
58 minutes ago, Nanobug said:

I've disabled it, and tried to set it to every hour from the regular mover, and also tried manually. And yes, there's 700 GB of data to move.

 

 

Any of these shares set to Cache Only? That for sure will not let anything move. 

Link to comment
59 minutes ago, kizer said:

Any of these shares set to Cache Only? That for sure will not let anything move. 

cache:only and cache:no shares are ignored.

 

cache:yes are moved from the designated pool to array and cache:prefer are moved from array to designated pool. Files on other pools are ignored. At least, that is how builtin mover works, not sure if the plugin version of mover considers all pools or not. Doesn't seem to be a setting for that so I assume it works like builtin mover.

 

Builtin mover won't move duplicates, not sure about the plugin version of mover. I assume it won't either since there doesn't appear to be a setting to tell it to. I don't use the plugin.

 

Nothing can move open files.

Link to comment

how can i configure mover to run if cache hits 80%  regardless of what time it is?

 

i had a massive 3 drive failure due to bad cables. lost 7 tb.  its all downloading now and my 1 TB cache drive is half full of cache-only stuff.  so the remaining 600GB fills up quick,  i have to run mover manually every day mid-day when cache hits 71-80%. mover runs every night at 4am as well.

 

i guess i could just set it on hourly - every 8 hours....

Edited by TRusselo
Link to comment
1 hour ago, TRusselo said:

its all downloading now

It would be better if you could set things so it downloads directly to the array. As it is right now you are wasting writes for no good reason. Better to just write once directly to the array, then once your download pattern gets back to normal you can use the cache to keep the array spun down.

  • Like 2
  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.