[Plugin] Mover Tuning

jcato · November 20, 2020

Back in April I started getting emails from this plugin. hugenbdd replied with a fix that worked. Here's a link to my post, followed by his replies.

Well, the emails are back. They're address to root with a bcc to my gmail address. The subject line (cron for user root /usr/local/sbin/mover force 2>/dev/null) is the same, but the content is a little different:

Complete Mover Command: find "/mnt/cache/Backups/" -depth -mtime +5 | /usr/local/sbin/move -d 0

Complete Mover Command: find "/mnt/cache/Downloads/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/Emby/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/TV/" -depth -mtime +5 | /usr/local/sbin/move -d 0

It's been doing this for awhile. I'm a little surprised no one else has posted about it. I must have something setup different than others. Any ideas?

sonic6 · November 20, 2020

I knew that kind of mail. Got those once every few month, but i forgot to ask for a solution.

jungle · December 17, 2020

I can't find this under Apps anymore. I'm on beta 35.

itimpi · December 17, 2020

5 minutes ago, jungle said:

I can't find this under Apps anymore. I'm on beta 35.

I believe that is because the plugin is not compatible with the 6.9.0 releases.

Squid · December 17, 2020

6 hours ago, jungle said:

I can't find this under Apps anymore. I'm on beta 35.

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools). Hence it is listed as being incompatible. Far easier to do that than to list it as "sort of incompatible"

CS01-HS · December 18, 2020

I'm running the original (more basic) version of this plugin with RC1 and two cache pools. So far so good.

Janrar · December 19, 2020

On 12/17/2020 at 8:24 PM, Squid said:

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools). Hence it is listed as being incompatible. Far easier to do that than to list it as "sort of incompatible"

Is there anyway I can run it on beta35 when I'm not using the multiple cache pools?

and if yes how do I install it?

Squid · December 19, 2020

Apps, Settings - Display Incompatible Apps

Not recommended to do, but it's your choice.

Rav20 · December 22, 2020

just a heads up, I believe ca.mover.tuner caused instability in the parity write during a large extended move (mover was moving and files were being written to cache as fast as it could move, resulting in a "move" run of about 4 hours). I ignored the warning that it was incompatible. I am running 6.9.0-RC1 with multiple cache pools. HDD checked out fine on smart and is happily rebuilding for an hour so far. needless to say, i've uninstalled it until it is updated. I would not recommend running until its sorted.

Edited December 22, 2020 by Rav20

trurl · December 22, 2020

6 minutes ago, Rav20 said:

HDD checked out fine on smart and is happily re-writing to parity drive for an hour so far

I don't understand what you are saying here. Any writes to the array updates parity at the same time. What exactly do you mean by "re-writing to parity drive"?

Rav20 · December 22, 2020

1 minute ago, trurl said:

I don't understand what you are saying here. Any writes to the array updates parity at the same time. What exactly do you mean by "re-writing to parity drive"?

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it.

Edited December 22, 2020 by Rav20

YEAHHWAY · December 22, 2020

On 12/19/2020 at 6:34 AM, Squid said:

Apps, Settings - Display Incompatible Apps

Not recommended to do, but it's your choice.

Ugh, so this app won't work with rc2?

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

DZMM · December 22, 2020

46 minutes ago, YEAHHWAY said:

Ugh, so this app won't work with rc2?

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:

########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

YEAHHWAY · December 22, 2020

8 minutes ago, DZMM said:

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:


########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

Would I need to have that "Turbo write mode..." part of the code if I always have turbo write mode running?

DZMM · December 22, 2020

39 minutes ago, YEAHHWAY said:

Would I need to have that "Turbo write mode..." part of the code if I always have turbo write mode running?

No, I just had that in there when I used to have a parity disk to turn on Turbo mode when diskmv was running. You don't need it

trurl · December 22, 2020

10 hours ago, Rav20 said:

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it.

If the parity drive was disabled then it was most likely a hardware problem such as connection or controller so probably nothing at all to do with the plugin. If you had posted diagnostics we might have been able to see what was happening.

hugenbdd · December 23, 2020

Got my second box up and running with RC2 and had a chance to review the code.

There doesn't appear to be a lot of changes that I have to make. I'll be working on a release, but the holidays may slow me down a bit.

hugenbdd · December 24, 2020

If you see an update in 6.8, do not update. This is for 6.9 RC2 only...

I have a new version, but am working through some bugs. You can install if you want to test, but I foresee a few changes soon.
https://raw.githubusercontent.com/hugenbd/ca.mover.tuning/master/plugins/ca.mover.tuning.plg

Currently, I check all cache pools for their size, and if one is over the threshold it will initiate a move for all pools. I plan to change this to check each pool individually. But wanted to get an initial release out right now.

Squid - Can you point me to any changes for the logger and how to check if cache exists?
Currently, the mover.tuning.page file throws a warning that no cache exists... I would like to keep the check instead of just deleting it.

if (empty($disks['cache']) || $disks['cache']['status']=='DISK_NP') {
echo "<p class='notice'>No Cache disk present!</p>";

Squid · December 24, 2020

Anything in $disks where the type is "Cache"

jedimstr · December 24, 2020

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path? Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin? What about those users who just click Update All and don't check the release notes?

Edited December 24, 2020 by jedimstr

Squid · December 24, 2020

3 hours ago, jedimstr said:

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path? Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin? What about those users who just click Update All and don't check the release notes?

Yup. Better to have set the min attribute as 6.9.0-rc2 in which case 6.8 while it would show an update available wouldn't allow you to actually install it.

hugenbdd · December 24, 2020

4 minutes ago, Squid said:

Yup. Better to have set the min attribute as 6.9.0-rc2 in which case 6.8 while it would show an update available wouldn't allow you to actually install it.

Where can I set this attribute? I don't see it in the plg file.

hugenbdd · December 24, 2020

something like this? (Saw it in your Community Apps plg..)

$version = parse_ini_file("/etc/unraid-version");

if ( version_compare($version['version'],"6.9.0", "<") )

{

echo "********************************************************************\n";

echo "\n";

echo "unRaid version 6.4.0 or greater to run\n";

echo "\n";

echo "********************************************************************\n";

exit(1);

}

Squid · December 24, 2020

Something akin to this

<PLUGIN name="&name;"
		author="&author;"
		version="&version;"
		pluginURL="&pluginURL;"
		support="&supportURL;"
		icon="gear"
		min="6.9.0-rc2">

Once that's there then I'll remove the moderation that's preventing it from appearing in CA

hugenbdd · December 25, 2020

15 hours ago, Squid said:
Something akin to this
<PLUGIN name="&name;"
		author="&author;"
		version="&version;"
		pluginURL="&pluginURL;"
		support="&supportURL;"
		icon="gear"
		min="6.9.0-rc2">
Once that's there then I'll remove the moderation that's preventing it from appearing in CA

Added, hope that fixes the "update" issue on 6.8.

Sign In

[Plugin] Mover Tuning

312 posts in this topic Last Reply 23 hours ago

Recommended Posts

jcato 1

Link to post

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Posted Images

sonic6 8

Link to post

jungle 1

Link to post

itimpi 818

Link to post

Squid 2884

Link to post

CS01-HS 27

Link to post

Janrar 0

Link to post

Squid 2884

Link to post

Rav20 0

Link to post

trurl 1510

Link to post

Rav20 0

Link to post

YEAHHWAY 0

Link to post

DZMM 236

Link to post

YEAHHWAY 0

Link to post

DZMM 236

Link to post

trurl 1510

Link to post

hugenbdd 11

Link to post

hugenbdd 11

Link to post

Squid 2884

Link to post

jedimstr 8

Link to post

Squid 2884

Link to post

hugenbdd 11

Link to post

hugenbdd 11

Link to post

Squid 2884

Link to post

hugenbdd 11

Link to post

Join the conversation