[Plugin] Mover Tuning


474 posts in this topic Last Reply

Recommended Posts

16 minutes ago, hugenbdd said:

Can you post a pic of your schedule page or let me know what you have in the file ignore box so I can try and reproduce on my end.

Sure. Screen attached. The contents of moverignore.txt is (probably redundant):

/mnt/cache/Share1/.DOWNLOADS/
/mnt/cache/Share1/.DOWNLOADS/*

 

Capture.PNG

Link to post
  • Replies 473
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

CA Mover Tuning       A simple little plugin (mainly created for my own purposes) that will let you fine tune the operation of the mover script On scheduled runs of mover, o

I would suggest removing the plug in, then moving everything.   I'm working on getting a Dev box up and running so I can continue working on this for the new Beta.

Haven't forgotten about this.  Will circle back in a few weeks once things settle down with the new release and try and add this in.

Posted Images

With this plugin,

Is the only move files that is a spesifc amount of days old looking at when it was first written there or when it was modified/read from?

Can you make an spesific folder(s) in a share be moved to array while the rest dont?

Link to post
21 hours ago, hugenbdd said:

Can you post a pic of your schedule page or let me know what you have in the file ignore box so I can try and reproduce on my end.

Hey, so I figured out what my issue was.

 

My share config file didn't have the shareCachePool parameter at all. Setting 'use cache pool' in share settings in Unraid from yes to no, then back to yes added this entry and script now works. (I only have one cache drive)

Link to post
6 minutes ago, classiccrime said:

Hey, so I figured out what my issue was.

 

My share config file didn't have the shareCachePool parameter at all. Setting 'use cache pool' in share settings in Unraid from yes to no, then back to yes added this entry and script now works. (I only have one cache drive)

That's odd.

 

This file (/boot/config/shares/Share1.cfg) should of had the entry of shareUseCache="yes" inside of it.  As I check to see if the share has a cache.

 

    for SHARECFG in /boot/config/shares/* ; do


        if grep -qs 'shareUseCache="yes"' "$SHARECFG" ; then
                echo "Share Config: $SHARECFG"
 

 

 

 

Link to post
8 hours ago, OsoPolar said:

HI Guys

 

Need a little guidance please (semi newbie)

I have the correct version of the plugin for 6.8.3.

But i am getting errors messages can you help me whey this is happening?

See image here

Laptop Screenshot- 65.png

Ignore for now.  If you have the old one from the archive raw link posted earlier.. you should be fine. (Which it looks like you do.)

Link to post
51 minutes ago, Mihle said:

With this plugin,

Is the only move files that is a spesifc amount of days old looking at when it was first written there or when it was modified/read from?

Can you make an spesific folder(s) in a share be moved to array while the rest dont?

We can't do read time because of the way unraid mounts the shares (Look back a  few pages as the details are discussed).  I go off of "mtime".  The majority of what this plugin does, is change the "find" string to send to the mover binary.  The stock mover script looks something like "find /mnt/usr/ | /usr/local/sbin/move"   and I will change it to look something like "find /mnt/usr/ -mtime +30 | /usr/local/sbin/move"

 

Yes you could use the "Ignore files listed inside of a text file:" setting, and put all the directories in that file you don't want moved off cache.  As it adds a 'grep' statement in between the find and the mover.  (<findstatement> | grep -vFf '$SKIPFILESLIST' | <mover binary>)

Link to post
41 minutes ago, hugenbdd said:

That's odd.

 

This file (/boot/config/shares/Share1.cfg) should of had the entry of shareUseCache="yes" inside of it.  As I check to see if the share has a cache.

 

    for SHARECFG in /boot/config/shares/* ; do


        if grep -qs 'shareUseCache="yes"' "$SHARECFG" ; then
                echo "Share Config: $SHARECFG"
 

 

 

 

Right, it worked up to that point, but looks like $CACHEPOOLNAME never gets set at line 138.

Tower root: Share Path: /mnt//Share1
PCTLIMIT=`cat /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg | grep "=" | rev | cut -c2-3 | rev`

Edit: I'm guessing shareCachePool option didn't exist when my share was initially created and since I only have ever used one cache drive it wasn't needed. Either way, thank you for this! 

Edited by classiccrime
Link to post
4 hours ago, hugenbdd said:

Ignore for now.  If you have the old one from the archive raw link posted earlier.. you should be fine. (Which it looks like you do.)

Will it still work correctly?

When you say ignore are we expecting a fix over this issue?

Link to post
1 hour ago, OsoPolar said:

Will it still work correctly?

When you say ignore are we expecting a fix over this issue?

Yes it will still work correctly.

since you are on the version from may and in in raid 6.8.3.  The warning is because the latest install log file has a minimum requirement of 6.9.0-rc2.  But because you are on the older version you are fine.

 

The “fix” will happen once 6.9 is out of beta.

Link to post
12 hours ago, hugenbdd said:

We can't do read time because of the way unraid mounts the shares (Look back a  few pages as the details are discussed).  I go off of "mtime".  The majority of what this plugin does, is change the "find" string to send to the mover binary.  The stock mover script looks something like "find /mnt/usr/ | /usr/local/sbin/move"   and I will change it to look something like "find /mnt/usr/ -mtime +30 | /usr/local/sbin/move"

🙁 Is mtime=modify time?

12 hours ago, hugenbdd said:

 

Yes you could use the "Ignore files listed inside of a text file:" setting, and put all the directories in that file you don't want moved off cache.  As it adds a 'grep' statement in between the find and the mover.  (<findstatement> | grep -vFf '$SKIPFILESLIST' | <mover binary>)

So if I want everything to stay on cache except one folder that move to array, I would have to list all the other folders in that text file? (Even if it's like 20 other folders or whatever that should stay in cache)

Link to post
8 hours ago, Mihle said:

🙁 Is mtime=modify time?

So if I want everything to stay on cache except one folder that move to array, I would have to list all the other folders in that text file? (Even if it's like 20 other folders or whatever that should stay in cache)

 

 

Mtime
https://www.unixtutorial.org/find-mtime-identify-recent-files-in-unix/#:~:text=as you probably know from,on when they were modified.

 

Yes, you would have to list all other directories you don't want to move.  Maybe put the directory you don't want on cache on it's own share and disable cache so it writes directly to the array.

Link to post
On 1/14/2021 at 4:18 PM, hugenbdd said:

 

 

Mtime
https://www.unixtutorial.org/find-mtime-identify-recent-files-in-unix/#:~:text=as you probably know from,on when they were modified.

 

Yes, you would have to list all other directories you don't want to move.  Maybe put the directory you don't want on cache on it's own share and disable cache so it writes directly to the array.

Ok, to be spesific I talk about my Nextcloud share. Thank for the info

Link to post
3 hours ago, b0mb said:

it would be nice if we could define the size of files bigger... biggest size is 1024 kb..... wouls be niche if this could be set as big as the user want ;)

It should be 1024M not kb.  if you have 1024 selected the find will add a -size +1024M

Link to post
On 1/13/2021 at 1:35 PM, classiccrime said:

Right, it worked up to that point, but looks like $CACHEPOOLNAME never gets set at line 138.


Tower root: Share Path: /mnt//Share1
PCTLIMIT=`cat /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg | grep "=" | rev | cut -c2-3 | rev`

Edit: I'm guessing shareCachePool option didn't exist when my share was initially created and since I only have ever used one cache drive it wasn't needed. Either way, thank you for this! 

 

For anyone else seeing this- I had these errors in my logs as well, and found that there were orphaned .cfg files in /boot/config/shares for shares that no longer existed. After deleting them, the errors went away.

Link to post
2 hours ago, b0mb said:

@hugenbdd

 

I know but it would be nice to set G also instead of only M ;)

How big do you need?

I fear not moving large files will make it very easy for the cache to fill up quickly and cause problems.

Link to post
On 4/8/2020 at 3:57 PM, hugenbdd said:

posted an update that

1.) Removes the "echo" statements from the age_mover file.  This should prevent the email to the admin.

2.) Checked the variable being passed in from the command line.  (this should fix your issue chizll).

 

Please let me know if you still run into issues.

 

Hi all,

 

Similar to user @chizll, I am seeing this message in my logs as well, every time mover runs:

 

Feb  1 09:10:42 Tower root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 171: [: no: integer expression expected
Feb  1 09:10:42 Tower root: Mover not Needed.
Feb  1 09:10:42 Tower root: Share Config: /boot/config/shares/Photos.cfg
Feb  1 09:10:42 Tower root: Pool Pct Used:   5 %
Feb  1 09:10:42 Tower root: Threshold Used: no
Feb  1 09:10:42 Tower root: "0
Feb  1 09:10:42 Tower root:  0
Feb  1 09:10:42 Tower root: 80
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: "1
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: "1
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: 10

 

This same block of log messages displays for every share in my cache.  If something IS moved from cache to array for that share, the "integer expression expected" message doesn't show up.  

 

The mover appears to be working correctly, so I just wanted to share in case this isn't expected.  I am using cache % used (80%) and file age (10 days) in my Mover settings, and I am on RC2.  Thanks!

Edited by Andiroo2
grammar
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.