[Plugin] Mover Tuning


Recommended Posts

16 minutes ago, hugenbdd said:

Can you post a pic of your schedule page or let me know what you have in the file ignore box so I can try and reproduce on my end.

Sure. Screen attached. The contents of moverignore.txt is (probably redundant):

/mnt/cache/Share1/.DOWNLOADS/
/mnt/cache/Share1/.DOWNLOADS/*

 

Capture.PNG

Link to comment

With this plugin,

Is the only move files that is a spesifc amount of days old looking at when it was first written there or when it was modified/read from?

Can you make an spesific folder(s) in a share be moved to array while the rest dont?

Link to comment
21 hours ago, hugenbdd said:

Can you post a pic of your schedule page or let me know what you have in the file ignore box so I can try and reproduce on my end.

Hey, so I figured out what my issue was.

 

My share config file didn't have the shareCachePool parameter at all. Setting 'use cache pool' in share settings in Unraid from yes to no, then back to yes added this entry and script now works. (I only have one cache drive)

Link to comment
6 minutes ago, classiccrime said:

Hey, so I figured out what my issue was.

 

My share config file didn't have the shareCachePool parameter at all. Setting 'use cache pool' in share settings in Unraid from yes to no, then back to yes added this entry and script now works. (I only have one cache drive)

That's odd.

 

This file (/boot/config/shares/Share1.cfg) should of had the entry of shareUseCache="yes" inside of it.  As I check to see if the share has a cache.

 

    for SHARECFG in /boot/config/shares/* ; do


        if grep -qs 'shareUseCache="yes"' "$SHARECFG" ; then
                echo "Share Config: $SHARECFG"
 

 

 

 

Link to comment
8 hours ago, OsoPolar said:

HI Guys

 

Need a little guidance please (semi newbie)

I have the correct version of the plugin for 6.8.3.

But i am getting errors messages can you help me whey this is happening?

See image here

Laptop Screenshot- 65.png

Ignore for now.  If you have the old one from the archive raw link posted earlier.. you should be fine. (Which it looks like you do.)

Link to comment
51 minutes ago, Mihle said:

With this plugin,

Is the only move files that is a spesifc amount of days old looking at when it was first written there or when it was modified/read from?

Can you make an spesific folder(s) in a share be moved to array while the rest dont?

We can't do read time because of the way unraid mounts the shares (Look back a  few pages as the details are discussed).  I go off of "mtime".  The majority of what this plugin does, is change the "find" string to send to the mover binary.  The stock mover script looks something like "find /mnt/usr/ | /usr/local/sbin/move"   and I will change it to look something like "find /mnt/usr/ -mtime +30 | /usr/local/sbin/move"

 

Yes you could use the "Ignore files listed inside of a text file:" setting, and put all the directories in that file you don't want moved off cache.  As it adds a 'grep' statement in between the find and the mover.  (<findstatement> | grep -vFf '$SKIPFILESLIST' | <mover binary>)

Link to comment
41 minutes ago, hugenbdd said:

That's odd.

 

This file (/boot/config/shares/Share1.cfg) should of had the entry of shareUseCache="yes" inside of it.  As I check to see if the share has a cache.

 

    for SHARECFG in /boot/config/shares/* ; do


        if grep -qs 'shareUseCache="yes"' "$SHARECFG" ; then
                echo "Share Config: $SHARECFG"
 

 

 

 

Right, it worked up to that point, but looks like $CACHEPOOLNAME never gets set at line 138.

Tower root: Share Path: /mnt//Share1
PCTLIMIT=`cat /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg | grep "=" | rev | cut -c2-3 | rev`

Edit: I'm guessing shareCachePool option didn't exist when my share was initially created and since I only have ever used one cache drive it wasn't needed. Either way, thank you for this! 

Edited by classiccrime
Link to comment
4 hours ago, hugenbdd said:

Ignore for now.  If you have the old one from the archive raw link posted earlier.. you should be fine. (Which it looks like you do.)

Will it still work correctly?

When you say ignore are we expecting a fix over this issue?

Link to comment
1 hour ago, OsoPolar said:

Will it still work correctly?

When you say ignore are we expecting a fix over this issue?

Yes it will still work correctly.

since you are on the version from may and in in raid 6.8.3.  The warning is because the latest install log file has a minimum requirement of 6.9.0-rc2.  But because you are on the older version you are fine.

 

The “fix” will happen once 6.9 is out of beta.

Link to comment
12 hours ago, hugenbdd said:

We can't do read time because of the way unraid mounts the shares (Look back a  few pages as the details are discussed).  I go off of "mtime".  The majority of what this plugin does, is change the "find" string to send to the mover binary.  The stock mover script looks something like "find /mnt/usr/ | /usr/local/sbin/move"   and I will change it to look something like "find /mnt/usr/ -mtime +30 | /usr/local/sbin/move"

🙁 Is mtime=modify time?

12 hours ago, hugenbdd said:

 

Yes you could use the "Ignore files listed inside of a text file:" setting, and put all the directories in that file you don't want moved off cache.  As it adds a 'grep' statement in between the find and the mover.  (<findstatement> | grep -vFf '$SKIPFILESLIST' | <mover binary>)

So if I want everything to stay on cache except one folder that move to array, I would have to list all the other folders in that text file? (Even if it's like 20 other folders or whatever that should stay in cache)

Link to comment
8 hours ago, Mihle said:

🙁 Is mtime=modify time?

So if I want everything to stay on cache except one folder that move to array, I would have to list all the other folders in that text file? (Even if it's like 20 other folders or whatever that should stay in cache)

 

 

Mtime
https://www.unixtutorial.org/find-mtime-identify-recent-files-in-unix/#:~:text=as you probably know from,on when they were modified.

 

Yes, you would have to list all other directories you don't want to move.  Maybe put the directory you don't want on cache on it's own share and disable cache so it writes directly to the array.

Link to comment
On 1/14/2021 at 4:18 PM, hugenbdd said:

 

 

Mtime
https://www.unixtutorial.org/find-mtime-identify-recent-files-in-unix/#:~:text=as you probably know from,on when they were modified.

 

Yes, you would have to list all other directories you don't want to move.  Maybe put the directory you don't want on cache on it's own share and disable cache so it writes directly to the array.

Ok, to be spesific I talk about my Nextcloud share. Thank for the info

Link to comment
3 hours ago, b0mb said:

it would be nice if we could define the size of files bigger... biggest size is 1024 kb..... wouls be niche if this could be set as big as the user want ;)

It should be 1024M not kb.  if you have 1024 selected the find will add a -size +1024M

Link to comment
On 1/13/2021 at 1:35 PM, classiccrime said:

Right, it worked up to that point, but looks like $CACHEPOOLNAME never gets set at line 138.


Tower root: Share Path: /mnt//Share1
PCTLIMIT=`cat /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg | grep "=" | rev | cut -c2-3 | rev`

Edit: I'm guessing shareCachePool option didn't exist when my share was initially created and since I only have ever used one cache drive it wasn't needed. Either way, thank you for this! 

 

For anyone else seeing this- I had these errors in my logs as well, and found that there were orphaned .cfg files in /boot/config/shares for shares that no longer existed. After deleting them, the errors went away.

Link to comment
On 4/8/2020 at 3:57 PM, hugenbdd said:

posted an update that

1.) Removes the "echo" statements from the age_mover file.  This should prevent the email to the admin.

2.) Checked the variable being passed in from the command line.  (this should fix your issue chizll).

 

Please let me know if you still run into issues.

 

Hi all,

 

Similar to user @chizll, I am seeing this message in my logs as well, every time mover runs:

 

Feb  1 09:10:42 Tower root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 171: [: no: integer expression expected
Feb  1 09:10:42 Tower root: Mover not Needed.
Feb  1 09:10:42 Tower root: Share Config: /boot/config/shares/Photos.cfg
Feb  1 09:10:42 Tower root: Pool Pct Used:   5 %
Feb  1 09:10:42 Tower root: Threshold Used: no
Feb  1 09:10:42 Tower root: "0
Feb  1 09:10:42 Tower root:  0
Feb  1 09:10:42 Tower root: 80
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: "1
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: "1
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: no
Feb  1 09:10:42 Tower root: ="
Feb  1 09:10:42 Tower root: es
Feb  1 09:10:42 Tower root: 10

 

This same block of log messages displays for every share in my cache.  If something IS moved from cache to array for that share, the "integer expression expected" message doesn't show up.  

 

The mover appears to be working correctly, so I just wanted to share in case this isn't expected.  I am using cache % used (80%) and file age (10 days) in my Mover settings, and I am on RC2.  Thanks!

Edited by Andiroo2
grammar
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.