[Plugin] Mover Tuning

Squid

By Squid,
in Plugin Support

Recommended Posts

Recommended

Posted by hugenbdd,

You Can.   From the Plugin's tab, then "install plugin"  put this URL into the textbox   https://raw.githubusercontent.com/hugenbd/ca.mover.tuning/6c146ad3ad63d162488d7e965c011d48d3e47462/plugins/ca.mover.tuning.plg   Then hit install.
Recommended by Squid

0 reactions

Go to this post
jcato Rookie

jcato

Posted
Posted

Back in April I started getting emails from this plugin. hugenbdd replied with a fix that worked. Here's a link to my post, followed by his replies.

 

Well, the emails are back. They're address to root with a bcc to my gmail address. The subject line (cron for user root /usr/local/sbin/mover force 2>/dev/null) is the same, but the content is a little different:

 

Complete Mover Command: find "/mnt/cache/Backups/" -depth -mtime +5 | /usr/local/sbin/move -d 0


Complete Mover Command: find "/mnt/cache/Downloads/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/Emby/" -depth -mtime +5 | /usr/local/sbin/move -d 0
Complete Mover Command: find "/mnt/cache/TV/" -depth -mtime +5 | /usr/local/sbin/move -d 0

 

It's been doing this for awhile. I'm a little surprised no one else has posted about it. I must have something setup different than others. Any ideas?

 

Link to comment
  • Replies 633
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid
Squid

CA Mover Tuning       A simple little plugin (mainly created for my own purposes) that will let you fine tune the operation of the mover script On scheduled runs of mover, o

hugenbdd
hugenbdd

I would suggest removing the plug in, then moving everything.   I'm working on getting a Dev box up and running so I can continue working on this for the new Beta.

hugenbdd
hugenbdd

New Release ###2021.04.15 - Changed text for field description of "Force move of files on a schedule:" to "Force move of all files on a schedule:" - No log entry for this, only an email

Posted Images

  • 4 weeks later...
Squid Grand Master

Squid

Posted
  • Author
Posted
6 hours ago, jungle said:

I can't find this under Apps anymore. I'm on beta 35.

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools).  Hence it is listed as being incompatible.  Far easier to do that than to list it as "sort of incompatible"

Link to comment
Janrar Newbie

Janrar

Posted
Posted
On 12/17/2020 at 8:24 PM, Squid said:

The plugin rewrites the mover command, but in such a way that while it still works under 6.9 you will lose a key feature of 6.9 (multiple cache pools).  Hence it is listed as being incompatible.  Far easier to do that than to list it as "sort of incompatible"

Is there anyway I can run it on beta35 when I'm not using the multiple cache pools?

 

and if yes how do I install it?

Link to comment
Rav20 Newbie

Rav20

Posted
Posted (edited)

just a heads up, I believe ca.mover.tuner caused instability in the parity write during a large extended move (mover was moving and files were being written to cache as fast as it could move, resulting in a "move" run of about 4 hours). I ignored the warning that it was incompatible. I am running 6.9.0-RC1 with multiple cache pools. HDD checked out fine on smart and is happily rebuilding for an hour so far. needless to say, i've uninstalled it until it is updated. I would not recommend running until its sorted. 

Edited by Rav20
Link to comment
Rav20 Newbie

Rav20

Posted
Posted (edited)
1 minute ago, trurl said:

I don't understand what you are saying here. Any writes to the array updates parity at the same time. What exactly do you mean by "re-writing to parity drive"?

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it. 

Edited by Rav20
Link to comment
YEAHHWAY Newbie

YEAHHWAY

Posted
Posted

 

On 12/19/2020 at 6:34 AM, Squid said:

Apps, Settings - Display Incompatible Apps

 

Not recommended to do, but it's your choice.

Ugh, so this app won't work with rc2? 

 

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

Link to comment
DZMM Proficient

DZMM

Posted
Posted
46 minutes ago, YEAHHWAY said:

 

Ugh, so this app won't work with rc2? 

 

I've been tyring to find something that would help my mover with moving files faster, instead of taking 3 days to move 3 TB of data off the cache drive.

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:

  

########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

 

Link to comment
YEAHHWAY Newbie

YEAHHWAY

Posted
Posted
8 minutes ago, DZMM said:

Have a look at the diskmv script - I use this to create my own little mover that I run on a schedule with user scripts, to move the files that I don't need 'fast' access to:

  


########################################
#######  Move Cache to Array  ##########
########################################

# check if mover running
if [ -f /var/run/mover.pid ]; then
	if ps h `cat /var/run/mover.pid` | grep mover ; then
		echo "$(date "+%d.%m.%Y %T") INFO: mover already running. Not moving files."
	fi
else
# move files
		ReqSpace=200000000
		AvailSpace=$(df /mnt/cache | awk 'NR==2 { print $4 }')

		if [[ "$AvailSpace" -ge "$ReqSpace" ]];then
			echo "$(date "+%d.%m.%Y %T") INFO: Space ok - exiting"
		else
			echo "$(date "+%d.%m.%Y %T") INFO: Cache space low.  Moving Files."

#	/usr/local/sbin/mdcmd set md_write_method 1
#	echo "Turbo write mode now enabled"
			echo "$(date "+%d.%m.%Y %T") INFO: moving backup."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/backup" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving local."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/complete" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/local/tdrive_vfs/downloads/seeds" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving media."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/books" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/calibre" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/magazines" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/photos" cache disk2
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/media/other_media/videos" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving mount_rclone cache."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/mount_rclone/cache" cache disk2
			echo "$(date "+%d.%m.%Y %T") INFO: moving software."
			bash /boot/config/plugins/user.scripts/scripts/unraid-diskmv/script -f -v "/mnt/user/software" cache disk2
#	/usr/local/sbin/mdcmd set md_write_method 0
#	echo "Turbo write mode now disabled"
		fi
fi

 

Would I need to have that "Turbo write mode..." part of the code if I always have turbo write mode running?

Link to comment
trurl Grand Master

trurl

Posted
Posted
10 hours ago, Rav20 said:

parity drive had write errors. the drive was removed and readded to the array. rebuilding would have been a better way to say it. 

If the parity drive was disabled then it was most likely a hardware problem such as connection or controller so probably nothing at all to do with the plugin. If you had posted diagnostics we might have been able to see what was happening.

Link to comment
hugenbdd Contributor

hugenbdd

Posted
Posted

If you see an update in 6.8, do not update.  This is for 6.9 RC2 only...

I have a new version, but am working through some bugs.  You can install if you want to test, but I foresee a few changes soon.
https://raw.githubusercontent.com/hugenbd/ca.mover.tuning/master/plugins/ca.mover.tuning.plg

 

Currently, I check all cache pools for their size, and if one is over the threshold it will initiate a move for all pools.  I plan to change this to check each pool individually.  But wanted to get an initial release out right now.

 

 

Squid - Can you point me to any changes for the logger and how to check if cache exists?
Currently, the mover.tuning.page file throws a warning that no cache exists... I would like to keep the check instead of just deleting it.

 

if (empty($disks['cache']) || $disks['cache']['status']=='DISK_NP') {
  echo "<p class='notice'>No Cache disk present!</p>";
 

  • Like 1
Link to comment
jedimstr Apprentice

jedimstr

Posted
Posted (edited)

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path?  Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin?  What about those users who just click Update All and don't check the release notes?

Edited by jedimstr
Link to comment
Squid Grand Master

Squid

Posted
  • Author
Posted
3 hours ago, jedimstr said:

Am I the only one who thinks its a bad idea to put out a version update with "DO NOT INSTALL ON 6.8" in the default update path?  Wouldn't it have been better to put out a separate plugin beta for manual install for 6.9 RC2 and keep the plugin that's compatible on the current stable versions on the regular plugin?  What about those users who just click Update All and don't check the release notes?

Yup.  Better to have set the min attribute as 6.9.0-rc2 in which case 6.8 while it would show an update available wouldn't allow you to actually install it.

Link to comment
hugenbdd Contributor

hugenbdd

Posted
Posted

something like this? (Saw it in your Community Apps plg..)

 

$version = parse_ini_file("/etc/unraid-version");

if ( version_compare($version['version'],"6.9.0", "<") )

{

echo "********************************************************************\n";

echo "\n";

echo "unRaid version 6.4.0 or greater to run\n";

echo "\n";

echo "********************************************************************\n";

exit(1);

}

Link to comment
hugenbdd Contributor

hugenbdd

Posted
Posted
15 hours ago, Squid said:

Something akin to this 


<PLUGIN name="&name;"
		author="&author;"
		version="&version;"
		pluginURL="&pluginURL;"
		support="&supportURL;"
		icon="gear"
		min="6.9.0-rc2">

Once that's there then I'll remove the moderation that's preventing it from appearing in CA

Added, hope that fixes the "update" issue on 6.8.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing