[Plugin] Mover Tuning

Squid

By Squid
in Plugin Support

Recommended Posts

Recommended

Posted by hugenbdd,

How to ignore a SINGLE file 1.) Find the path of the file you wish to ignore.     ls -ltr /mnt/cache/Download/complete/test.txt         root@Tower:/# ls -ltr /mnt/cache/Download/complete/test.txt         -rwxrwxrwx 1 root root 14 Oct 27 11:32 /mnt/cache/Download/complete/test.txt* 2.) Copy the complete path used.       /mnt/cache/Download/complete/test.txt 3.) Create a text file to hold the ignore list.     vi /mnt/user/appdata/mover-ignore/mover_ignore.txt 4.) Add a file pa
Recommended by Squid
  • Like

4 reactions

Go to this post
Recommended

Posted by Swarles,

FAQ - Frequently Asked Questions I'm putting these here for some of the common problems, and will try to update it as more questions/issues come along. How do I enable logs / provide logs? Mover wont run after installing Mover Tunning? Mover isn't ignoring files or folders? Issue Reporting Template  
Recommended by JonathanM
  • Upvote
  • Thanks

2 reactions

Go to this post
Vader666 Noob

Vader666

Posted
Posted
9 minutes ago, hugenbdd said:

Mover Logging : Enabled. (Up top, schedule area)

 

You can see in your last pic the "mover started.      Mover Finished". at the bottom.  That will expand to a lot more.

 

You will need to browse the individual cache drive or use the console and issue ls commands on the cache.

https://www.tecmint.com/15-basic-ls-command-examples-in-linux/

 

also

https://www.techrepublic.com/article/linux-directory-navigation-tips-and-tricks/#:~:text=To change to your home,and then press [Enter].

ok cool i am trying to figure this out 

thank you 

Link to comment
  • Replies 1.4k
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid
Squid

CA Mover Tuning       A simple little plugin (mainly created for my own purposes) that will let you fine tune the operation of the mover script On scheduled runs of mover, o

hugenbdd
hugenbdd

New Version 07-03-2023   HUGE thanks/Shout-out to @Swarles for the help with updating code fixes and adding a great new feature.  You can now have some customization per share.  It's great t

hugenbdd
hugenbdd

How to ignore a SINGLE file 1.) Find the path of the file you wish to ignore.     ls -ltr /mnt/cache/Download/complete/test.txt         root@Tower:/# ls -ltr /mnt/cache/Download/complete/test

Posted Images

jcato Rookie

jcato

Posted
Posted
On 3/16/2021 at 9:33 AM, hugenbdd said:

It's because I left ECHO statements in the script.  These help me catch bugs for the new release.

 

I will release a new version in the next few days that will comment out the echo statements.

 

Hello! Has this happened again? I'm getting emails when the daily cron job runs that say "Mover not Needed." I don't get one every day, I guess only when the mover is not needed.😃

 

Thanks

 

Link to comment
polishprocessors Rookie

polishprocessors

Posted
Posted

Hey there! I have this plugin installed and set to run the mover at 60% cache utilization with a run frequency of every 3 hours. However, my cache keeps getting over 60% and failing to move. If I run the mover manually it works fine so I changed the 'move now button follows plugin fliters’ option to ‘yes’ and ran it again. Here’s my output:

  

Sep 27 19:05:59 unRaid root: mvlogger: *********************************MOVER START*******************************
Sep 27 19:05:59 unRaid root: mvlogger: Age supplied
Sep 27 19:05:59 unRaid root: mvlogger: Size supplied
Sep 27 19:05:59 unRaid root: mvlogger: Sparness supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Skipfiles Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Skipfiles Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Before Script Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No After Script Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: CTIME Argument: no
Sep 27 19:05:59 unRaid root: mvlogger: CACHE THRESH to Move all Cache-Yes shares to array: 65
Sep 27 19:05:59 unRaid root: mvlogger: No Test Mode Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Ignore Hidden Files Argument Supplied
Sep 27 19:05:59 unRaid root: mover: started
Sep 27 19:05:59 unRaid root: mvlogger: No Script to Run.
Sep 27 19:05:59 unRaid root: mvlogger: CACHETHRESH: 65
Sep 27 19:05:59 unRaid root: mvlogger: Checking to see if we should all files from share.
Sep 27 19:05:59 unRaid root: mvlogger: Move All Cache Threshold: 65
Sep 27 19:05:59 unRaid root: mvlogger: /mnt/cache PCT USED: 85
Sep 27 19:05:59 unRaid root: mvlogger: Pool /mnt/cache is above 'move all' percentage, moving all cache-yes shares to array
Sep 27 19:05:59 unRaid root: mvlogger: Not in Test mode
Sep 27 19:05:59 unRaid root: find: unknown predicate `-depth+= | /usr/local/sbin/move -d 1'
Sep 27 19:05:59 unRaid root: mvlogger: Complete Mover Command: find /mnt/cache/media -depth
Sep 27 19:05:59 unRaid root: mvlogger: Done checking Move All from Share
Sep 27 19:05:59 unRaid root: mvlogger: Share Name Only: Media
Sep 27 19:05:59 unRaid root: mvlogger: Cache Pool Name: cache 
Sep 27 19:05:59 unRaid root: mvlogger: cache Threshold Pct: 
Sep 27 19:05:59 unRaid root: mvlogger: OVERALL Threshold: 60
Sep 27 19:05:59 unRaid root: mvlogger: Share Path: /mnt/cache/Media
Sep 27 19:05:59 unRaid root: mvlogger: Pool Pct Used: 85 %
Sep 27 19:05:59 unRaid root: mvlogger: DFTPCT LIMIT USED FOR SETTING: 60
Sep 27 19:05:59 unRaid root: mvlogger: Threshold Used: 60
Sep 27 19:05:59 unRaid root: mvlogger: Adding Age
Sep 27 19:05:59 unRaid root: mvlogger: Age 1
Sep 27 19:05:59 unRaid root: mvlogger: Skipfiletypes string: find "/mnt/cache/Media" -depth -mtime +1
Sep 27 19:05:59 unRaid root: mvlogger: Not in Test mode
Sep 27 19:05:59 unRaid root: mvlogger: Complete Mover Command: find "/mnt/cache/Media" -depth -mtime +1
Sep 27 19:05:59 unRaid root: find: '/mnt/cache/Media': No such file or directory
Sep 27 19:05:59 unRaid root: mover: finished
Sep 27 19:05:59 unRaid root: mvlogger: No After Script to Run.

 

and some settings for reference:
 

image.thumb.png.a2db75cabd438a95cb065274c001873a.png

 

Any thoughts on what might be happening?

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
9 hours ago, jcato said:

 

Hello! Has this happened again? I'm getting emails when the daily cron job runs that say "Mover not Needed." I don't get one every day, I guess only when the mover is not needed.😃

 

Thanks

 

I'll check the code but should only be happening when logging is enabled....

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
5 hours ago, polishprocessors said:

Hey there! I have this plugin installed and set to run the mover at 60% cache utilization with a run frequency of every 3 hours. However, my cache keeps getting over 60% and failing to move. If I run the mover manually it works fine so I changed the 'move now button follows plugin fliters’ option to ‘yes’ and ran it again. Here’s my output:

  

Sep 27 19:05:59 unRaid root: mvlogger: *********************************MOVER START*******************************
Sep 27 19:05:59 unRaid root: mvlogger: Age supplied
Sep 27 19:05:59 unRaid root: mvlogger: Size supplied
Sep 27 19:05:59 unRaid root: mvlogger: Sparness supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Skipfiles Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Skipfiles Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Before Script Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No After Script Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: CTIME Argument: no
Sep 27 19:05:59 unRaid root: mvlogger: CACHE THRESH to Move all Cache-Yes shares to array: 65
Sep 27 19:05:59 unRaid root: mvlogger: No Test Mode Argument Supplied
Sep 27 19:05:59 unRaid root: mvlogger: No Ignore Hidden Files Argument Supplied
Sep 27 19:05:59 unRaid root: mover: started
Sep 27 19:05:59 unRaid root: mvlogger: No Script to Run.
Sep 27 19:05:59 unRaid root: mvlogger: CACHETHRESH: 65
Sep 27 19:05:59 unRaid root: mvlogger: Checking to see if we should all files from share.
Sep 27 19:05:59 unRaid root: mvlogger: Move All Cache Threshold: 65
Sep 27 19:05:59 unRaid root: mvlogger: /mnt/cache PCT USED: 85
Sep 27 19:05:59 unRaid root: mvlogger: Pool /mnt/cache is above 'move all' percentage, moving all cache-yes shares to array
Sep 27 19:05:59 unRaid root: mvlogger: Not in Test mode
Sep 27 19:05:59 unRaid root: find: unknown predicate `-depth+= | /usr/local/sbin/move -d 1'
Sep 27 19:05:59 unRaid root: mvlogger: Complete Mover Command: find /mnt/cache/media -depth
Sep 27 19:05:59 unRaid root: mvlogger: Done checking Move All from Share
Sep 27 19:05:59 unRaid root: mvlogger: Share Name Only: Media
Sep 27 19:05:59 unRaid root: mvlogger: Cache Pool Name: cache 
Sep 27 19:05:59 unRaid root: mvlogger: cache Threshold Pct: 
Sep 27 19:05:59 unRaid root: mvlogger: OVERALL Threshold: 60
Sep 27 19:05:59 unRaid root: mvlogger: Share Path: /mnt/cache/Media
Sep 27 19:05:59 unRaid root: mvlogger: Pool Pct Used: 85 %
Sep 27 19:05:59 unRaid root: mvlogger: DFTPCT LIMIT USED FOR SETTING: 60
Sep 27 19:05:59 unRaid root: mvlogger: Threshold Used: 60
Sep 27 19:05:59 unRaid root: mvlogger: Adding Age
Sep 27 19:05:59 unRaid root: mvlogger: Age 1
Sep 27 19:05:59 unRaid root: mvlogger: Skipfiletypes string: find "/mnt/cache/Media" -depth -mtime +1
Sep 27 19:05:59 unRaid root: mvlogger: Not in Test mode
Sep 27 19:05:59 unRaid root: mvlogger: Complete Mover Command: find "/mnt/cache/Media" -depth -mtime +1
Sep 27 19:05:59 unRaid root: find: '/mnt/cache/Media': No such file or directory
Sep 27 19:05:59 unRaid root: mover: finished
Sep 27 19:05:59 unRaid root: mvlogger: No After Script to Run.

 

and some settings for reference:
 

image.thumb.png.a2db75cabd438a95cb065274c001873a.png

 

Any thoughts on what might be happening?

Looks like it might be a share name issue.

 

Sep 27 19:05:59 unRaid root: mvlogger: Not in Test mode Sep 27 19:05:59 unRaid root: find: unknown predicate `-depth+= | /usr/local/sbin/move -d 1'

 

Is the share name media or Media. (lower versus upper).  Seems also maybe a configuration file may have a mismatch in it to cause the issue above.

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
On 9/27/2021 at 8:51 AM, jcato said:

 

Hello! Has this happened again? I'm getting emails when the daily cron job runs that say "Mover not Needed." I don't get one every day, I guess only when the mover is not needed.😃

 

Thanks

 

There was an Echo in there for that statement.  Next release will have it updated to follow the logging settings. (But that may be a while...)

 

If you want to fix it now.

Line 396 of /usr/local/emhttp/plugins/ca.mover.tuning/age_mover

Change

echo "mover not Needed."

 

to

 

mvlogger "Mover not Needed."

 

  • Thanks 1
Link to comment
jcato Rookie

jcato

Posted
Posted
On 9/28/2021 at 11:26 AM, hugenbdd said:

There was an Echo in there for that statement.  Next release will have it updated to follow the logging settings. (But that may be a while...)

 

If you want to fix it now.

Line 396 of /usr/local/emhttp/plugins/ca.mover.tuning/age_mover

Change

echo "mover not Needed."

 

to

 

mvlogger "Mover not Needed."

 

I made this change and emails stopped. But, then they started again. I rechecked the file and it's back to an echo statement.

 

I think it's because I rebooted. Does this change not survive a reboot? If so, is there a way to make it permanent?

 

 

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
5 hours ago, jcato said:

I made this change and emails stopped. But, then they started again. I rechecked the file and it's back to an echo statement.

 

I think it's because I rebooted. Does this change not survive a reboot? If so, is there a way to make it permanent?

 

 

Probably doesn't survive reboot.

 

I'll work on a release in the next few days.

  • Thanks 1
Link to comment
bschaeff18 Noob

bschaeff18

Posted
Posted

Hello,

I'm trying to run a simple stop then start script with the user.scripts plugin but it doesn't what to run the scripts it looks like. When I got to check if the dockers have been stopped they have not. So my question is, where does this go to look for the before/after scripts? As well as how can I point this to the user scripts?

Link to comment
Tune Noob

Tune

Posted
Posted

I have a problem setting up a text file with excluded paths. Whenever i set it up to exclude a single path, it work fine, but i just tried to make a file that has multiple lines in it, and it only excludes the last line in the file whenever it starts moving.

 

These are the three lines that i have in the file currently:

/mnt/cache/Media/Downloads
/mnt/cache/Syncthing/Telefoonbackup
/mnt/cache/Syncthing/Overdracht

 

But when running the mover, it only excludes /mnt/cache/Syncthing/Overdracht

 

I am positive that there are no spelling errors in the paths used, should i be using a different way to seperate them in the text file, instead of putting each path on a seperate line?

 

Any help would be appreciated.

 

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
5 hours ago, Tune said:

I have a problem setting up a text file with excluded paths. Whenever i set it up to exclude a single path, it work fine, but i just tried to make a file that has multiple lines in it, and it only excludes the last line in the file whenever it starts moving.

 

These are the three lines that i have in the file currently:

/mnt/cache/Media/Downloads
/mnt/cache/Syncthing/Telefoonbackup
/mnt/cache/Syncthing/Overdracht

 

But when running the mover, it only excludes /mnt/cache/Syncthing/Overdracht

 

I am positive that there are no spelling errors in the paths used, should i be using a different way to seperate them in the text file, instead of putting each path on a seperate line?

 

Any help would be appreciated.

 

Make sure logging is on.

Move Now button follows plug-in filters (Yes)

Turn on test mode (Very bottom)

 

 

Test each directory one at a time in your file.

 

It's possible there are control characters for the end of line if you created the file in windows.  If you created it with vi editor or from the command line this should not be the case.

 

Link to comment
Tune Noob

Tune

Posted
Posted (edited)
2 hours ago, hugenbdd said:

Make sure logging is on.

Move Now button follows plug-in filters (Yes)

Turn on test mode (Very bottom)

 

 

Test each directory one at a time in your file.

 

It's possible there are control characters for the end of line if you created the file in windows.  If you created it with vi editor or from the command line this should not be the case.

 

Hey, thank you for taking the time to try and help me with this, i did have the settings on  that you mentioned here, and i used the test mode to come to the results I have at the moment.

I'll try recreating the file via command line as i have indeed created it in windows, I will report back later how this goes, thank you!

 

Edit: I just recreated the file using nano in the command line, and jus retyping the lines instead of copying them. Using the test mode of mover tuning it all seems to work now. So it seems that using Windows to create the file and edit it was indeed the issue, thank you for helping me out!

Edited by Tune
Reporting back
  • Like 1
Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
1 hour ago, Tune said:

Edit: I just recreated the file using nano in the command line, and jus retyping the lines instead of copying them. Using the test mode of mover tuning it all seems to work now. So it seems that using Windows to create the file and edit it was indeed the issue, thank you for helping me out!

Great!

In normal Linux dos2unix will take care of this.  There is a "end of line" character difference between Windows & Unix. Sometimes they show ^M if you print the file from the command line.

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
On 10/5/2021 at 8:49 AM, jcato said:

I made this change and emails stopped. But, then they started again. I rechecked the file and it's back to an echo statement.

 

I think it's because I rebooted. Does this change not survive a reboot? If so, is there a way to make it permanent?

 

 

New Release to fix this issue.

Link to comment
wgstarks Experienced

wgstarks

Posted
Posted

I’m trying to get this plugin configured but can’t seem to get it right. My goal is to get this plugin to run mover if the cache is above a set percentage and also at a set time every 24 hours regardless of used space.

 

Settings-

430243269_ScreenShot2021-10-17at8_00_22AM.thumb.png.f05e8f650fea0732b7cb7b9dde8f9cbb.png

308958511_ScreenShot2021-10-17at8_00_47AM.thumb.png.d99997c4bb0bed9020be500bdc9cf62e.png

 

@hugenbdd I believe we discussed this in a previous post and at that time it wasn't possible to use both features together but I saw there have been changes and wanted to try again. The screenshots show both features enabled. Move files with disk usage above 40% and force move of all files at 3:30am. I have filled the cache disk to about 65% but mover doesn't run after waiting about 3 hours. I tried disabling "force move of all files" and waited again but still nothing. Have I gotten something else mis-configured?

Link to comment
hugenbdd Enthusiast

hugenbdd

Posted
Posted
11 minutes ago, wgstarks said:

I’m trying to get this plugin configured but can’t seem to get it right. My goal is to get this plugin to run mover if the cache is above a set percentage and also at a set time every 24 hours regardless of used space.

 

Settings-

430243269_ScreenShot2021-10-17at8_00_22AM.thumb.png.f05e8f650fea0732b7cb7b9dde8f9cbb.png

308958511_ScreenShot2021-10-17at8_00_47AM.thumb.png.d99997c4bb0bed9020be500bdc9cf62e.png

 

@hugenbdd I believe we discussed this in a previous post and at that time it wasn't possible to use both features together but I saw there have been changes and wanted to try again. The screenshots show both features enabled. Move files with disk usage above 40% and force move of all files at 3:30am. I have filled the cache disk to about 65% but mover doesn't run after waiting about 3 hours. I tried disabling "force move of all files" and waited again but still nothing. Have I gotten something else mis-configured?

 

Looks right..

Can you enable logging and post the log entries?  Then I can track down the issue.

Link to comment
wgstarks Experienced

wgstarks

Posted
Posted
1 hour ago, hugenbdd said:

 

Looks right..

Can you enable logging and post the log entries?  Then I can track down the issue.

I may have found the issue-

 

2015932895_ScreenShot2021-10-17at8_00_22AM.thumb.png.79b169ae38229be7697975f362397969.png

 

Does the traditional mover schedule effect when the actions set in the tuning settings are performed? Seem to remember that it does. I set this to hourly and mover ran on the hour. Still testing to be sure the filters are being respected though. I've enabled logging and will include them in my next post if I'm still having issues.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing