[Plugin] Mover Tuning


312 posts in this topic Last Reply

Recommended Posts

Back in April I started getting emails from this plugin. hugenbdd replied with a fix that worked. Here's a link to my post, followed by his replies.

 

Well, the emails are back. They're address to root with a bcc to my gmail address. The subject line (cron for user root /usr/local/sbin/mover force 2>/dev/null) is the same, but the content is a little different:

 

Complete Mover Command: find "/mnt/cache/Backups/" -depth -mtime +5 | /usr/local/sbin/move -d 0


Complete Mover Command: find "/mnt/cache/Downloads/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/Emby/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/TV/" -depth -mtime +5 | /usr/local/sbin/move -d 0

 

It's been doing this for awhile. I'm a little surprised no one else has posted about it. I must have something setup different than others. Any ideas?

 

Link to post
  • Replies 311
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

CA Mover Tuning       A simple little plugin (mainly created for my own purposes) that will let you fine tune the operation of the mover script On scheduled runs of mover, o

I would suggest removing the plug in, then moving everything.   I'm working on getting a Dev box up and running so I can continue working on this for the new Beta.

Easiest way to come close to what you want is to change mover to "hourly" and then set the "threshold of used cache" to your desired % of when to actually move files.  Mover will not move anything if

Posted Images

  • 4 weeks later...
6 hours ago, jungle said:

I can't find this under Apps anymore. I'm on beta 35.

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools).  Hence it is listed as being incompatible.  Far easier to do that than to list it as "sort of incompatible"

Link to post
On 12/17/2020 at 8:24 PM, Squid said:

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools).  Hence it is listed as being incompatible.  Far easier to do that than to list it as "sort of incompatible"

Is there anyway I can run it on beta35 when I'm not using the multiple cache pools?

 

and if yes how do I install it?

Link to post

just a heads up, I believe ca.mover.tuner caused instability in the parity write during a large extended move (mover was moving and files were being written to cache as fast as it could move, resulting in a "move" run of about 4 hours). I ignored the warning that it was incompatible. I am running 6.9.0-RC1 with multiple cache pools. HDD checked out fine on smart and is happily rebuilding for an hour so far. needless to say, i've uninstalled it until it is updated. I would not recommend running until its sorted. 

Edited by Rav20
Link to post
6 minutes ago, Rav20 said:

HDD checked out fine on smart and is happily re-writing to parity drive for an hour so far

I don't understand what you are saying here. Any writes to the array updates parity at the same time. What exactly do you mean by "re-writing to parity drive"?

Link to post
1 minute ago, trurl said:

I don't understand what you are saying here. Any writes to the array updates parity at the same time. What exactly do you mean by "re-writing to parity drive"?

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it. 

Edited by Rav20
Link to post

 

On 12/19/2020 at 6:34 AM, Squid said:

Apps, Settings - Display Incompatible Apps

 

Not recommended to do, but it's your choice.

Ugh, so this app won't work with rc2? 

 

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

Link to post
46 minutes ago, YEAHHWAY said:

 

Ugh, so this app won't work with rc2? 

 

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:

 

########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

 

Link to post
8 minutes ago, DZMM said:

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:

 


########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

 

Would I need to have that "Turbo write mode..." part of the code if I always have turbo write mode running?

Link to post
39 minutes ago, YEAHHWAY said:

Would I need to have that "Turbo write mode..." part of the code if I always have turbo write mode running?

No, I just had that in there when I used to have a parity disk to turn on Turbo mode when diskmv was running.  You don't need it

Link to post
10 hours ago, Rav20 said:

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it. 

If the parity drive was disabled then it was most likely a hardware problem such as connection or controller so probably nothing at all to do with the plugin. If you had posted diagnostics we might have been able to see what was happening.

Link to post

If you see an update in 6.8, do not update.  This is for 6.9 RC2 only...

I have a new version, but am working through some bugs.  You can install if you want to test, but I foresee a few changes soon.
https://raw.githubusercontent.com/hugenbd/ca.mover.tuning/master/plugins/ca.mover.tuning.plg

 

Currently, I check all cache pools for their size, and if one is over the threshold it will initiate a move for all pools.  I plan to change this to check each pool individually.  But wanted to get an initial release out right now.

 

 

Squid - Can you point me to any changes for the logger and how to check if cache exists?
Currently, the mover.tuning.page file throws a warning that no cache exists... I would like to keep the check instead of just deleting it.

 

if (empty($disks['cache']) || $disks['cache']['status']=='DISK_NP') {
  echo "<p class='notice'>No Cache disk present!</p>";
 

Link to post

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path?  Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin?  What about those users who just click Update All and don't check the release notes?

Edited by jedimstr
Link to post
3 hours ago, jedimstr said:

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path?  Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin?  What about those users who just click Update All and don't check the release notes?

Yup.  Better to have set the min attribute as 6.9.0-rc2 in which case 6.8 while it would show an update available wouldn't allow you to actually install it.

Link to post
4 minutes ago, Squid said:

Yup.  Better to have set the min attribute as 6.9.0-rc2 in which case 6.8 while it would show an update available wouldn't allow you to actually install it.

Where can I set this attribute?  I don't see it in the plg file.

Link to post

something like this? (Saw it in your Community Apps plg..)

 

$version = parse_ini_file("/etc/unraid-version");

if ( version_compare($version['version'],"6.9.0", "<") )

{

echo "********************************************************************\n";

echo "\n";

echo "unRaid version 6.4.0 or greater to run\n";

echo "\n";

echo "********************************************************************\n";

exit(1);

}

Link to post

Something akin to this

<PLUGIN name="&name;"
		author="&author;"
		version="&version;"
		pluginURL="&pluginURL;"
		support="&supportURL;"
		icon="gear"
		min="6.9.0-rc2">

Once that's there then I'll remove the moderation that's preventing it from appearing in CA

Link to post
15 hours ago, Squid said:

Something akin to this


<PLUGIN name="&name;"
		author="&author;"
		version="&version;"
		pluginURL="&pluginURL;"
		support="&supportURL;"
		icon="gear"
		min="6.9.0-rc2">

Once that's there then I'll remove the moderation that's preventing it from appearing in CA

Added, hope that fixes the "update" issue on 6.8.

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.