[Plugin] Mover Tuning


Recommended Posts

3 hours ago, hugenbdd said:

New update fixing the two issues above.  I hope removing the 0 from the days old drop down does not cause issues.   However, setting it to 1 will move with a "find -mtime 0" now.

 

11-07-2021

- Updated "move all" code to fix wrong variable and script error.
- Removed 0 from days old drop down
- Updated age_mover to day - 1 to be able to move files 24 hours old instead of 48. (See post from captaincu on 11/3/2021)

 

I updated to version 2021.11.07 and unfortunately I'm still having trouble moving files greater than 24 hours old.

 

Setting "Move files that are greater than this many days old" to 1 results in the following command being run.

find "/mnt/downloads/Media" -depth -type f -mtime +1 -print0 | du -shc --files0-from - | tail -n1

 

I believe it should be using the following option.

-mtime +0

 

Link to comment
9 minutes ago, captaincu said:

 

I updated to version 2021.11.07 and unfortunately I'm still having trouble moving files greater than 24 hours old.

 

Setting "Move files that are greater than this many days old" to 1 results in the following command being run.

find "/mnt/downloads/Media" -depth -type f -mtime +1 -print0 | du -shc --files0-from - | tail -n1

 

I believe it should be using the following option.

-mtime +0

 

Yup agree... Okay, will look at it again tonight/tomorrow.

 

Link to comment
21 hours ago, captaincu said:

 

I updated to version 2021.11.07 and unfortunately I'm still having trouble moving files greater than 24 hours old.

 

Setting "Move files that are greater than this many days old" to 1 results in the following command being run.

find "/mnt/downloads/Media" -depth -type f -mtime +1 -print0 | du -shc --files0-from - | tail -n1

 

I believe it should be using the following option.

-mtime +0

 

Updated... had some issues with testing so there were actually 3 releases today.. Hopefully I didn't mess to much stuff up.  I fully expect to have some issues, but I tested as best I could...

 

###2021.11.08

- Actually subtract 1 from the age of files for 3rd item in 11-07 release.

- uncomment commented out eval string for mover. (To actually move files.)

  • Like 1
Link to comment

Finaly i added some cache pools and parity drive to array.

having to question:

 

1. all our shares that prefer cache is marked as unprotected is it OK?

 

2. when mover regular run having some files exist also not exist on some apps [ appdata is (prefer ssd) that was moved after stopped docker services ] but now like this:

any ideas guys?

image.thumb.png.b36e7c174c03e95e0f43f96e277b2cc7.png

 

image.png.46ab4ce10071895c50b863fb583b4745.png

 

 

 

Edited by Masterwishx
Link to comment
22 hours ago, Masterwishx said:

Finaly i added some cache pools and parity drive to array.

having to question:

 

1. all our shares that prefer cache is marked as unprotected is it OK?

 

2. when mover regular run having some files exist also not exist on some apps [ appdata is (prefer ssd) that was moved after stopped docker services ] but now like this:

any ideas guys?

image.thumb.png.b36e7c174c03e95e0f43f96e277b2cc7.png

 

image.png.46ab4ce10071895c50b863fb583b4745.png

 

 

 

Hi Masterwishx

I don't think this is a plug-in issue.  This is a general mover issues.  You would probably have better luck posting a separate thread with debug log attached. 

  • Like 1
Link to comment

File ignore settings are not working anymore.

 

I had some .anchor files which are now being moved despite being in the "Ignore file types".

 

I tried the "Ignore files listed inside of a text" setting, and was not working either.

 

I finally got around the problem create a script to run after the mover (which is not ideal)

Edited by J OBrien
  • Like 1
Link to comment
14 minutes ago, J OBrien said:

File ignore settings are not working anymore.

 

I had some .anchor files which are now being moved despite being in the "Ignore file types".

 

I tried the "Ignore files listed inside of a text" setting, and was not working either.

 

I finally got around the problem create a script to run after the mover (which is not ideal)

Can you turn on logging and post the entries from the syslog?

 

makes it easier for me to reproduce.

Link to comment
33 minutes ago, J OBrien said:

File ignore settings are not working anymore.

 

I had some .anchor files which are now being moved despite being in the "Ignore file types".

 

I tried the "Ignore files listed inside of a text" setting, and was not working either.

 

I finally got around the problem create a script to run after the mover (which is not ideal)

Think I was able to reproduce.

Nov 11 21:26:33 Tower root: mvlogger: find "/mnt/cache/TV" -depth -type f -mtime +29 -print0 -e '.anchor' | du -shc --files0-from - | tail -n1
Nov 11 21:26:33 Tower root: find: unknown predicate `-e'

 

Will look at this tomorrow.

Link to comment
15 hours ago, J OBrien said:

File ignore settings are not working anymore.

 

I had some .anchor files which are now being moved despite being in the "Ignore file types".

 

I tried the "Ignore files listed inside of a text" setting, and was not working either.

 

I finally got around the problem create a script to run after the mover (which is not ideal)

Update is out. 

(No more sizing of total files in test mode)

 

Seemed to work with my changes, as I was only getting an error when in test mode.

 

Please post your settings and the log entries if it still errors for you.

Link to comment

Something change recently? I have my Mover set to move everything 15days old.

Everything Moved. 

 

I haven't changed a setting in months. 

 

Is this New?

 

Move All from Cache-Yes shares when disk is above a certain percentage: Yes

Move All from Cache-yes shares pool percentage: 10%

 

 

Link to comment
10 minutes ago, kizer said:

Something change recently? I have my Mover set to move everything 15days old.

Everything Moved. 

 

I haven't changed a setting in months. 

 

Is this New?

 

Move All from Cache-Yes shares when disk is above a certain percentage: Yes

Move All from Cache-yes shares pool percentage: 10%

 

 

Yes, can you post your log entries? I tried to test as best I could.

Link to comment

Well if that is new then I had files bigger than 10% of my drive so it moved them per the settings. 

I changed them to NO for the future and I'll see what happens. ;)

 

Not a big deal I just didn't know those settings existed and moved files off my Cache. Lol

 

Is it possible to add new features and have them default to NO just to avoid future questions/issues. I'm not mad or anything, it just perplexed me when I fired up a Movie I ripped earlier and my drive spun up when it normally doesn't for 15 days. 

 

Looking at my Logs it looks like it did what was intended, I just didn't know about it. Lol

 

Link to comment
27 minutes ago, kizer said:

Well if that is new then I had files bigger than 10% of my drive so it moved them per the settings. 

I changed them to NO for the future and I'll see what happens. ;)

 

Not a big deal I just didn't know those settings existed and moved files off my Cache. Lol

 

Is it possible to add new features and have them default to NO just to avoid future questions/issues. I'm not mad or anything, it just perplexed me when I fired up a Movie I ripped earlier and my drive spun up when it normally doesn't for 15 days. 

 

Looking at my Logs it looks like it did what was intended, I just didn't know about it. Lol

 

There was some issue with it that was causing it to not move in the past.  I fixed it, and tested as best I could...

 

Move All from Cache-Yes shares when disk is above a certain percentage: Yes

Move All from Cache-yes shares pool percentage: 10%

 

That should move everything if the pool is above 10% (I would recommend this be above 60% or more if using it.)

Link to comment
On 11/12/2021 at 12:41 PM, hugenbdd said:

Update is out. 

(No more sizing of total files in test mode)

 

Seemed to work with my changes, as I was only getting an error when in test mode.

 

Please post your settings and the log entries if it still errors for you.

 

Hello, this is the beginning of my log (I am on 2021.11.12). I Tried with .anchor, anchor, and both.

 

Nov 15 15:50:44 Tower root: mvlogger: *********************************MOVER START*******************************
Nov 15 15:50:44 Tower root: mvlogger: Age supplied 0
Nov 15 15:50:44 Tower root: mvlogger: Size supplied
Nov 15 15:50:44 Tower root: mvlogger: Sparness supplied
Nov 15 15:50:44 Tower root: mvlogger: No Skipfiles Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: SKIP FILE TYPES: .anchor,anchor
Nov 15 15:50:44 Tower root: mvlogger: No Before Script Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No After Script Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No CTIME Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: CACHE THRESH to Move all Cache-Yes shares to array: 5
Nov 15 15:50:44 Tower root: mvlogger: No Test Mode Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No Ignore Hidden Files Argument Supplied
Nov 15 15:50:44 Tower root: mover: started
Nov 15 15:50:44 Tower root: mvlogger: No Script to Run.
Nov 15 15:50:44 Tower root: mvlogger: CACHETHRESH: 5
Nov 15 15:50:44 Tower root: mvlogger: Checking to see if we should all files from share.
Nov 15 15:50:44 Tower root: mvlogger: Move All Cache Threshold: 5
Nov 15 15:50:44 Tower root: mvlogger: /mnt/cache PCT USED: 31
Nov 15 15:50:44 Tower root: mvlogger: Pool /mnt/cache is above 'move all' percentage, moving all cache-yes shares to array
Nov 15 15:50:44 Tower root: mvlogger: Not in Test mode
Nov 15 15:50:44 Tower root: mvlogger: Complete Mover Command: find /mnt/cache/media.movies.4k -depth | /usr/local/sbin/move -d 1
Nov 15 15:50:44 Tower move: move: file /mnt/cache/media.movies.4k/local.anchor

 

 

Link to comment
1 hour ago, J OBrien said:

 

Hello, this is the beginning of my log (I am on 2021.11.12). I Tried with .anchor, anchor, and both.

 

Nov 15 15:50:44 Tower root: mvlogger: *********************************MOVER START*******************************
Nov 15 15:50:44 Tower root: mvlogger: Age supplied 0
Nov 15 15:50:44 Tower root: mvlogger: Size supplied
Nov 15 15:50:44 Tower root: mvlogger: Sparness supplied
Nov 15 15:50:44 Tower root: mvlogger: No Skipfiles Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: SKIP FILE TYPES: .anchor,anchor
Nov 15 15:50:44 Tower root: mvlogger: No Before Script Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No After Script Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No CTIME Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: CACHE THRESH to Move all Cache-Yes shares to array: 5
Nov 15 15:50:44 Tower root: mvlogger: No Test Mode Argument Supplied
Nov 15 15:50:44 Tower root: mvlogger: No Ignore Hidden Files Argument Supplied
Nov 15 15:50:44 Tower root: mover: started
Nov 15 15:50:44 Tower root: mvlogger: No Script to Run.
Nov 15 15:50:44 Tower root: mvlogger: CACHETHRESH: 5
Nov 15 15:50:44 Tower root: mvlogger: Checking to see if we should all files from share.
Nov 15 15:50:44 Tower root: mvlogger: Move All Cache Threshold: 5
Nov 15 15:50:44 Tower root: mvlogger: /mnt/cache PCT USED: 31
Nov 15 15:50:44 Tower root: mvlogger: Pool /mnt/cache is above 'move all' percentage, moving all cache-yes shares to array
Nov 15 15:50:44 Tower root: mvlogger: Not in Test mode
Nov 15 15:50:44 Tower root: mvlogger: Complete Mover Command: find /mnt/cache/media.movies.4k -depth | /usr/local/sbin/move -d 1
Nov 15 15:50:44 Tower move: move: file /mnt/cache/media.movies.4k/local.anchor

 

 

Looks like it's working as intended.

Nov 15 15:50:44 Tower root: mvlogger: Checking to see if we should all files from share.
Nov 15 15:50:44 Tower root: mvlogger: Move All Cache Threshold: 5
Nov 15 15:50:44 Tower root: mvlogger: /mnt/cache PCT USED: 31
Nov 15 15:50:44 Tower root: mvlogger: Pool /mnt/cache is above 'move all' percentage, moving all cache-yes shares to array

 

Move All settings move everything in the share if it's on the cache and percent is above the Move All Cache Threshold.

  • Thanks 1
Link to comment
16 hours ago, J OBrien said:

Thank you.

 

I don't know why that setting changed all of a sudden.

 

Thank you its working now.

 

 

 

 

There was an update recently that added some additional settings along the bottom of the settings page. 

To Move files if the source is above 10%. Just set it to NO should fix it right up. Or you could adjust the settings to 60% or something else as a fall back. 

 

I personally just set it to NO for now. I might tweak it later. 

Link to comment

So, I recently installed this plugin, and have it set to Move All from Cache-Yes shares when disk is above a certain percentage:Yes and the threshold set to 75%

 

I manually invoked the mover, so the cache was empty. I then transferred media to the server, and it filled the cache drive to 88%, and the mover wasnt initialised by the plugin. What have I missed?

Untitled.png

Link to comment
46 minutes ago, alturismo said:

 

how often do you let the mover scheduled run ?

 

if you use it like me with a treshold, i let it run hourly.

 

image.thumb.png.6594b56a253fbd68625d833d78fd1682.png

OK, that might have been it. I had it scheduled to run daily (at 1am). Changed it to hourly. So effectively that is how frequently the mover tuner will check those thresholds?

Link to comment
3 hours ago, alturismo said:

Yes

Gesendet von meinem SM-G981B mit Tapatalk
 

Thanks for the help. So what are the two different thresholds for then. One to ignore move if the cache is at least than say 50% then the other to move if the cache is more than say 70%... What's the difference?

 

As an example, how could I set the mover to run every night at 2am, or if the cache disk is more than 70% full?

Link to comment
1 hour ago, eatoff said:

how could I set the mover to run every night at 2am, or if the cache disk is more than 70% full?

I use a user script to move all the files at a certain time and use this plugin to move all files at a certain percentage.

#!/bin/bash
/usr/local/sbin/mover.old start

 

Edited by wgstarks
  • Thanks 1
Link to comment

New Release

###2021.11.23

- Add GUI button in individual shares to move just that shares files from the cache. Must be cache yes

- Fixed bug where original mover would overwrite syslog instead of append.

 

Suggested from Tomr

 

image.thumb.png.99f454b8fe8870fdf1916eee2ef9dd65.png 

*Guess my wording could be a bit better.  Basically sends all files on the cache pool name it find in the share path (example: /mnt/cache/TV) to the array.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.