[Plugin] Mover Tuning

Terebi · October 9, 2023

On 6/7/2023 at 8:32 AM, hugenbdd said:

If I wrote this, it would be an independent script and not part of the plug-in. But this could be done as a cron entry.

Any progress on this feature?

Kloudz · October 12, 2023

On 9/12/2023 at 8:53 AM, Kloudz said:

Its been a few days, seems to stop working today. The only thing I did today was stop then started my array. Needed to check some configs but didn't make any changes.

Anyway, I even added 3 additional files. Triggered Mover via Main screen and nothing. Then I followed your steps that you posted last week and went all the way to Step #6 and its now working again.

Something is definitely up here. Just to Recap, since I last posted that it was working. There hasn't been any issues since. But earlier this morning, I noticed theres still a file in my cache drive. Ran Mover like usual and didn't think of any till 5 mins later and noticed that the file was still in the cache. From here, I stopped the Array to check some configs, didn't change anything then started the Array. Ran Mover again and nothing. So I tried to perform all the troubleshooting steps again and went all the way to Step #6 and once I performed step #6. Mover started working again from the Main Screen. Between Steps 1-5, I didn't make any changes until Step #6

I just want to go back to this.

I rebooted my server today to change some bios settings. Mover stopped working. Performed Step #6 and it started working again.

Just an FYI not sure if the issue is fixed yet

methanoid · October 12, 2023

On 8/22/2023 at 3:20 PM, hugenbdd said:

New release from some files Swarles and I have made last month. Sorry for the delay, life got in the way....

2023.08.22

Fixed Cron Job entry

Modified ignore command to include folders (Swarles)

Updated mover cmdline functions (Hugenbdd)

is this as simple as adding folder path in the ignore list? It doesnt SEEM to be working for me if thats the process. And does the descriptive heading and help text reflect the revisioN?

Swarles · October 13, 2023

On 10/12/2023 at 3:59 PM, Kloudz said:

I just want to go back to this.

I rebooted my server today to change some bios settings. Mover stopped working. Performed Step #6 and it started working again.

Just an FYI not sure if the issue is fixed yet

Haven't had the update pushed yet so every time you restart the server it will revert unfortunately. I know it's not ideal but I think it's a bit of work to push an update so just have to wait until hugenbdd has some spare time.

Would you be able to confirm if you have "Move now button uses mover settings" enabled though?

Swarles · October 13, 2023

21 hours ago, methanoid said:

is this as simple as adding folder path in the ignore list? It doesnt SEEM to be working for me if thats the process. And does the descriptive heading and help text reflect the revisioN?

Yes this is correct, though I think the description could probably use an update because as you said it doesn't reflect this. It will probably take me another 5 weeks before I work on anything as I am in the process of finishing up my studies.
In the meantime, even the old code worked for skipping folders (just had a few edge cases). I suspect you may have given it a path on the "user" share, you will want to make sure any file or folder paths you give it are on your "cache" (or whatever yours is called) share/drive.

Eg.

/mnt/user/data/media/linux-isos   <--Bad
/mnt/cache/data/media/linux-isos  <--Good

This happens because mover tuning never goes through the "user" share and only deals with "cache" and "disk#" directly. I may create a fix for this in the future where it detects if it's on the user share and replaces it with the cache, but this could potentially be problematic on systems with multiple cache pools.

Hopefully that fixes, your issue

Edited October 13, 2023 by Swarles

xokia · October 16, 2023

This is probably an issue with my config. I downloaded mover tuning

Even if I click Move now nothing happens it also does not run daily. If I disable "disable mover running on a schedule" and click Move now it will run mover. So it appears the issue is with mover tuning. These are the default settings when you download mover tuning. I must not be understanding something in mover tuning. What am I missing?

Kilrah · October 16, 2023

3 hours ago, xokia said:

If I disable "disable mover running on a schedule" and click Move now it will run mover.

Well if you want to run mover you certainly don't want to disable it, do you?

xokia · October 16, 2023

9 hours ago, Kilrah said:

Well if you want to run mover you certainly don't want to disable it, do you?

Not sure you follow the issue with mover tuning enabled, mover does not run.

With "Disable mover running on a schedule" set to yes. I could get mover to run as expected.

Kilrah · October 16, 2023

5 hours ago, xokia said:

Not sure you follow the issue with mover tuning enabled, mover does not run.

With "Disable mover running on a schedule" set to yes. I could get mover to run as expected.

You said "disable "Disable mover running on a schedule"", which would have meant setting it to No, which would have been normal.

So something else then.

Edited October 16, 2023 by Kilrah

adammerkley · October 19, 2023

On 9/12/2023 at 4:32 AM, 2TC said:
When I boot up my server I get the following error in the output of the screen
mv: cannot stat '/usr/local/bin/mover': No such file or directory
it is not really an issue, I searched the web and I found this post talking about this and the addon "move tuning plugin", which I assume is "CA Mover Tuning". The post talks about deleting the plugin. Now I do not have an issue with the server not starting, just that the entry is there.

I looked at the '/usr/loval/bin/mover' which is a shell script
#!/bin/bash
PPPID=`ps h -o ppid= $PPID 2>/dev/null`
P_COMMAND=`ps h -o %c $PPPID 2>/dev/null`
/usr/local/emhttp/plugins/ca.mover.tuning/mover.php $P_COMMAND $1
I looked at the file mentioned, and the mover.php file is where it is supposed to be, seems everything is working as intended, except that I get the error when booting it, surely I could just ignore it. But I rather fix it than leave it there.

Did you ever get an answer to this? I see this message upon boot too. Doesn't seem to be harming anything, but just knowing it's there is annoying

xokia · October 19, 2023

On 10/16/2023 at 2:32 PM, Kilrah said:

You said "disable "Disable mover running on a schedule"", which would have meant setting it to No, which would have been normal.

So something else then.

No.....If I set it to yes then I can click move now and it moves the files. If its set to no and I click move now nothing happens.

Alternatively

With it set like I listed above which is to move daily (with disable mover tuning set to no) exactly like pictured above. Nothing gets moved files just sit on the SSD.

When I finish with my latest transcode I will circle back to this.

knights_of_pine · October 22, 2023

i have a folder inside of a share that i would like to keep on the cache. how can i do this. the share is set to move from cache to array on move but for this one folder i would like it to stay on cache. can someone help me out? thank you!

Swarles · October 23, 2023

On 10/20/2023 at 8:14 AM, xokia said:

No.....If I set it to yes then I can click move now and it moves the files. If its set to no and I click move now nothing happens.

Alternatively

With it set like I listed above which is to move daily (with disable mover tuning set to no) exactly like pictured above. Nothing gets moved files just sit on the SSD.

When I finish with my latest transcode I will circle back to this.

Hey, sorry for the delayed response, very very busy. The following link is a comment I created a little while ago to try to troubleshoot issues. You can ignore MOST of it. Just skip to steps 3 and 4. Try those and let me know if one of those was your issue

Swarles · October 23, 2023

On 10/23/2023 at 3:29 AM, knights_of_pine said:

i have a folder inside of a share that i would like to keep on the cache. how can i do this. the share is set to move from cache to array on move but for this one folder i would like it to stay on cache. can someone help me out? thank you!

In mover tuning settings, enable "Ignore files listed inside of a text file:" setting.
Below it, supply a path to a .txt file that will contain the paths of the files and folders you want the mover to ignore.

eg.

/mnt/cache/files/ignore_list.txt

And then inside the .txt file supply the path to the folder you want to ignore. If the folder is called "CatPics" it will be like this:

/mnt/cache/files/photos/CatPics

It is very important that you supply the path to the cache pool ("cache" in my example) and not on the "user" share. If you supply "/mnt/user/files/photos/CatPics" it won't skip the folder on the cache.

knights_of_pine · October 24, 2023

On 10/23/2023 at 10:06 AM, Swarles said:
In mover tuning settings, enable "Ignore files listed inside of a text file:" setting.
Below it, supply a path to a .txt file that will contain the paths of the files and folders you want the mover to ignore.

eg.
/mnt/cache/files/ignore_list.txt
And then inside the .txt file supply the path to the folder you want to ignore. If the folder is called "CatPics" it will be like this:
/mnt/cache/files/photos/CatPics
It is very important that you supply the path to the cache pool ("cache" in my example) and not on the "user" share. If you supply "/mnt/user/files/photos/CatPics" it won't skip the folder on the cache.

for the ignore list location can I use /user or must I use /cache on that as well.

for folder names that have spaces should I use \ to escape the space?

FayeInMay · October 25, 2023

On 9/10/2023 at 10:44 PM, sourCream said:

I followed Kloudz advice and I did seem to be missing the file as well. While I was able to run mover from the dashboard again I seem to be running into another issue and that is despite "Move Now button follows plug-in filters" being set to NO unless I change my "Only move at this threshold of used cache space" to something lower than what is currently being used the mover will not run when activated from the dashboard but will run when activated from the the shares page or the scheduler page.

I'm not missing the file and have the same problem with the Move Now button follows plug-in filters. Any ideas towards that?

Swarles · October 26, 2023

23 hours ago, FayeInMay said:

I'm not missing the file and have the same problem with the Move Now button follows plug-in filters. Any ideas towards that?

Which one? For step 3 did you check if the relevant line was inside the file? Step 4 only requires checking if the file exists.

Swarles · October 26, 2023

On 10/25/2023 at 4:35 AM, knights_of_pine said:

for the ignore list location can I use /user or must I use /cache on that as well.

for folder names that have spaces should I use \ to escape the space?

You should be able to put the ignore list .txt anywhere, I personally put the skip lists on my usb.

It has been a while since I did the implementation and testing, but I thought I made it so that it could handle spaces just fine. Best practice is to not use spaces in paths though.

dopeytree · October 30, 2023

Hi is this plugin broken on 6.12.4 ?

I have it set to move files over 90days old but when I trigger it starts then immediately stops?

There are lots of older TV shows & movies it should be moving from cache to array.

At the moment having to do it manually via Midnight Commander..

Maybe my config is broken?

moulin-rouge-diagnostics-20231030-1839.zip

Edited October 30, 2023 by dopeytree

Swarles · October 31, 2023

17 hours ago, dopeytree said:

Hi is this plugin broken on 6.12.4 ?

I have it set to move files over 90days old but when I trigger it starts then immediately stops?

There are lots of older TV shows & movies it should be moving from cache to array.

At the moment having to do it manually via Midnight Commander..

Maybe my config is broken?

moulin-rouge-diagnostics-20231030-1839.zip 203.31 kB · 0 downloads

I think you're the first to mention it, I have my files set to move over 30 days and it is working just fine on 6.12.4 with zfs.
Would you perhaps be able to share the ".log" file in "/tmp/Mover" which should give more information on what the mover is actually doing (can send in pm if you want, or review it yourself). The ".list" files also show lists of what files should be moved.

dopeytree · October 31, 2023

Sure.

it's probably something I've mis-set etc.

Noticed this:

No /tmp/Mover/Custom_Mover_Directory_2023-10-30T180820.list created as no empty directories

Files attached.

Mover_Tuning_2023-10-30T180820.log Mover_Tuning_2023-10-30T180825.log Mover_Tuning_2023-10-30T181114.log Mover_Tuning_2023-10-30T181129.log Mover_Tuning_2023-10-30T181154.log Mover_Tuning_2023-10-30T181235.log

akazu · November 2, 2023

Hello,

I would like to exclude from mover the subdirectory "pictures" from my media share which is located on the cache.

Can I do this by setting the option "Ignore files listed inside of a text file:" to "yes" and

under the option "File list path:" the path to the file which contains the path to the pictures directory?

If so, where do you save such "configuration files"?

If no, what do I have to do to exclude the complete folder?.

Thanks in advance

alturismo · November 2, 2023

57 minutes ago, akazu said:

If so, where do you save such "configuration files"?

If no, what do I have to do to exclude the complete folder?.

Thanks in advance

may just take a look at the recommended post on top of this topic

akazu · November 2, 2023

1 hour ago, alturismo said:

may just take a look at the recommended post on top of this topic

Thanks for your help. I have now done the following

Created a moverignore folder with the ignore file inside.

ls -la /mnt/cachessd/appdata/moverignore/
total 4
drwxrwxrwx 1 nobody users 30 Nov 2 07:23 ./
drwxr-xr-x 1 nobody users 76 Nov 2 07:21 ../
-rw-rw-rw- 1 root root 30 Nov 2 07:23 moverignore.txt

cat /mnt/cachessd/appdata/moverignore/moverignore.txt 
/mnt/cachessd/media/Pictures/

image.png.8fd4c24d3e557d97605a0acb720cafc7.png

Afterwards I executed the mover and now all shares on the unraid GUI are missing under shares and nothing was moved

image.png.e1ce51f7b4c5ae505bf7dbe104858bc9.png

akazu · November 2, 2023

okay after a reboot the shares are back, phew.

Now when I run the Mover it does not exlude the Pictures folder. Have I misunderstood something in the instructions?

[Plugin] Mover Tuning

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid

hugenbdd

hugenbdd

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation