Jump to content
Squid

[Plugin] CA Auto Turbo Write Mode

98 posts in this topic Last Reply

Recommended Posts

I think short spindown times are bad for drive health. (Mine is 5 hours in attempt to only have one per day). It is very hard to prove what is bad and how bad. But my research is that it is better to run a drive too hot than to let it go from cool to warm frequently.

 

If Tom were to implement this, I think that he could have a "convenience turbo write mode" that would not count disk accesses for turbo write purposes against the spindown timer. And that unRAID would take the the system out of turbo write when a drive spun down. This would enable turbo writes when all disks were spun up for other purposes, without stopping them from spinning down. The cutover from turbo to non-turbo write if writes are in process would be a little tricky - but Tom is a smart guy and would figure it out. There would be no reason not to use this convenience mode. Not saying he would do this, but would be an option.

Share this post


Link to post

installed and maxing out my gigabit connection now without using cache drive!

Share this post


Link to post

Looks like your image hosting is mad for the first post

Share this post


Link to post
4 hours ago, wreave said:

Looks like your image hosting is mad for the first post

I guess that its not enough that I have to look at a million ads with Photobucket, but now they're blocking forums. 

 

Quote

On June 28, 2017, Photobucket changed its Terms of Use regarding free accounts and third party hosting (hosting on forums, eBay, etc). Only the most expensive plan, at $399.99 per year, now permits third party hosting and linking to forums, etc. Which effectively alienating a very large percentage of their users, whom only used Photobucket for third-party image hosting. Photobucket made this change with no input or consultation with their current users, or the public. Adding this to photobuckets use of excessive pop-up ads, scams, clickbait advertisements, and in some cases viruses embedded in ads, the number of active account quickly decreased. [11]

 

5929770772_687efd74d4_m.jpg

 

 

 

( or more to the point, here's a link that I'll probably get asked to remove: https://www.youtube.com/watch?v=Sa5HkcKkCfA )

 

Edited by Squid
  • Upvote 1

Share this post


Link to post

Nice plugin, however it always get's the number of spun down disks wrong. Even with polling under 10 seconds. Ie with all disks spun up, invoke setting of 2  and poll of 5 seconds it reported 6 disks spun down and disabled turbo, then 2 disks next poll, then 8, then 1 and enabled turbo, then disabled and reported 4 etc etc etc, all while every disk was spun up and active. (15 disk array btw)

 

Wonder if it's something to do the plugin not polling from the sas hba properly?

 

Anyway, look forward to when it's integrated into unraid or fixed. Keep up the good work

Share this post


Link to post

The plugin utilizes hdparm to check the status of the drives which will be more accurate than what's reported with unRaid's GUI.

 

You can always enable the additional debug logging, and then post your diagnostics here.  But don't leave it enabled as it will basically spam the log with the output from the various commands as it runs.

 

  • Upvote 1

Share this post


Link to post
5 hours ago, Squid said:

The plugin utilizes hdparm to check the status of the drives which will be more accurate than what's reported with unRaid's GUI.

 

You can always enable the additional debug logging, and then post your diagnostics here.  But don't leave it enabled as it will basically spam the log with the output from the various commands as it runs.

 

Ok, doing an unbalance op atmo. when that's done in a day or so ill do some testing. Btw with my previous comment, I had spun up all drives and disabled spin down delay (was 30mins before) and had manually enabled turbo write before enabling the plugin, so even if the gui was reporting wrongly, the drives 'should' have been spun up.

Share this post


Link to post

great script squid.  Is there anyway I can combine your custom mover script with your turbo scripts, so that when the mover script kicks in turbo mode is activated and then turned off when mover finishes?

 

Thanks

 

 

  • Upvote 1

Share this post


Link to post

Run this as a script before mover starts

/usr/local/sbin/mdcmd set md_write_method 1

 

After mover is done and drives start spinnning down, the auto turbo script will detect it and disable it.

  • Upvote 1

Share this post


Link to post
On 19/10/2017 at 1:01 PM, Squid said:

Run this as a script before mover starts

 


/usr/local/sbin/mdcmd set md_write_method 1

 

 

After mover is done and drives start spinnning down, the auto turbo script will detect it and disable it.

Thanks - is this correct?  I'm not sure about the '1' at the end.

#!/usr/bin/php
<?PHP
$moveAt = 85;                 # Adjust this value to suit (% cache drive full to move at)
$runDuringCheck = true;     # change to true to run mover during a parity check / rebuild

$diskTotal = disk_total_space("/mnt/cache");
$diskFree = disk_free_space("/mnt/cache");
$percent = ($diskTotal - $diskFree) / $diskTotal * 100;

if ( $percent > $moveAt ) {
  if ( ! $runDuringCheck ) {
    $vars = parse_ini_file("/var/local/emhttp/var.ini");
    if ( $vars['mdResync'] ) {
      echo "Parity Check / Rebuild Running - Not executing mover\n";
      exec("logger Parity Check / Rebuild Running - Not executing mover");
    } else {
	  exec("/usr/local/sbin/mdcmd set md_write_method 1");
      exec("/usr/local/sbin/mover");
    }
  } else {
    exec("/usr/local/sbin/mdcmd set md_write_method 1");
    exec("/usr/local/sbin/mover");
  }
}
?>

I'm going to do this for my weekly overnight vm backup jobs as well to ensure they finish quickly.

Edited by DZMM
  • Upvote 1

Share this post


Link to post

The 1 is correct.  The script should work.  Not tested though.

Share this post


Link to post

It's working, but when mover is triggered I need to add turn TW off after mover has finished.  I think:

 

exec("/usr/local/sbin/mdcmd set md_write_method");

turns TW off, but how do I put a delay in to (i) not do this until mover has finished or (ii) hardcode a 30 min delay.  Have I used sleep correctly?

 

#!/usr/bin/php
<?PHP
$moveAt = 85;                 # Adjust this value to suit (% cache drive full to move at)
$runDuringCheck = true;     # change to true to run mover during a parity check / rebuild

$diskTotal = disk_total_space("/mnt/cache");
$diskFree = disk_free_space("/mnt/cache");
$percent = ($diskTotal - $diskFree) / $diskTotal * 100;

if ( $percent > $moveAt ) {
  if ( ! $runDuringCheck ) {
    $vars = parse_ini_file("/var/local/emhttp/var.ini");
    if ( $vars['mdResync'] ) {
      echo "Parity Check / Rebuild Running - Not executing mover\n";
      exec("logger Parity Check / Rebuild Running - Not executing mover");
    } else {
      exec("/usr/local/sbin/mdcmd set md_write_method 1");
      exec("/usr/local/sbin/mover");
	  sleep 1800;									# trying to put a 30 min pause in
	  exec("/usr/local/sbin/mdcmd set md_write_method");
    }
  } else {
    exec("/usr/local/sbin/mdcmd set md_write_method 1");
    exec("/usr/local/sbin/mover");
	sleep 1800;										# trying to put a 30 min pause in
	exec("/usr/local/sbin/mdcmd set md_write_method");
  }
}
?>

 

Share this post


Link to post
On 3/13/2017 at 4:49 PM, Squid said:

 

 

As an aside, the default polling time is 300 seconds.  I've set it to be 1 sec during development, and no stuttering or anything happened on my system while playing back.

 

 

 

The only way I seem to be able to get my drives to switch back to normal mode is to set my polling to 1 second.  Even 10 seconds doesn't switching back to normal if I manually try and force them to spin down- they just spin right back up, even when I know there's no activity on most of the drives.

 

Is 1 second still safe to use?  If so, I'm loving this plugin as even though most of my writes occur in the background it's still nice knowing they are completing as fast as possible when appropriate, as well as my drives spinning down afterwards for maximum noise reduction.

Share this post


Link to post
3 hours ago, DZMM said:

The only way I seem to be able to get my drives to switch back to normal mode is to set my polling to 1 second.  Even 10 seconds doesn't switching back to normal if I manually try and force them to spin down- they just spin right back up, even when I know there's no activity on most of the drives.

 

Is 1 second still safe to use?  If so, I'm loving this plugin as even though most of my writes occur in the background it's still nice knowing they are completing as fast as possible when appropriate, as well as my drives spinning down afterwards for maximum noise reduction.

You would have to enable debug mode in the plugin, and then see what's happening via the syslog

  • Like 1

Share this post


Link to post
20 minutes ago, Squid said:

You would have to enable debug mode in the plugin, and then see what's happening via the syslog

I've cut my syslog down to show where I had to hit 'spin down' twice to turn off turbo mode.  As you can see I hit spin down twice within 16 seconds, so there's no way 4 drives suddenly became inactive and I know for certain only disk 3 was active at the time as that's how I've got my shares setup to try and only have this drive spinning during the day (other disks contain old downloads/movies/tv shows and new stuff goes to cache/disk 3 before being added to libraries)

 

BTW is there something wrong with my cron job:

 

0 0 2 ? * MON *

as the logs are showing this error:

 

Oct 29 12:10:01 Highlander crond[2957]: failed parsing crontab for user root: ? * MON * /usr/local/emhttp/plugins/ca.turbo/scripts/turboSchedule.php enable 90 > /dev/null 2>&1

I created it at cronmaker.com to run every Monday at 2am (when my vm backups run)

edited syslog.txt

Share this post


Link to post
10 minutes ago, DZMM said:

BTW is there something wrong with my cron job:

A cron should only have 5 entries.  You've got 7  http://corntab.com/

 

You want 0 2 * * 1 to run Mondays @ 2am

 

I'll look at the log later in the day.

Edited by Squid

Share this post


Link to post
2 minutes ago, Squid said:

A cron should only have 5 entries.  You've got 7  http://corntab.com/

 

You want 0 2 * * 1 to run Mondays @ 2am

 

I'll look at the log later in the day.

Thanks

 

I'll stick to crontab-generator.org as that's the format it created.

Share this post


Link to post

This is what I see

 

You got it set up so that one data disk is allowed to be spun down and turbo mode is still enabled

 

Everything looks like it works fine.  When 2 or more data disks are spun down turbo mode gets disabled

 

The only thing that I'm seeing is that when you manually spin down all drives, I'm not picking up a second later most of the drives even being installed.  This is probably because every scan the first thing I do is check to see what drives are even available via an unRaid variable.  That variable during a mass spin down is probably not complete with its reporting, so its possible that if I only see say 2 drives being available at that time, and only one of them is spun down that I will switch to turbo mode and spin everything back up when the next write happens.  But, on the next scan if no write happened, then the drives are still spun down and I would wind up switching to normal.

 

IE:  I don't think anything is wrong here.  But, your log does highlight a flaw where I can't 100% tell what's going on if unRaid is manipulating the spindowns at the same time that I'm checking.  But, on the next scan everything recovers itself and carries on.

 

 

Ultimately something like this plugin needs to be incorporated at the driver level by Limetech themselves (and make the turbo mode setting a true "auto")

 

 

Don't forget to disable debug mode in the plugin.  Your syslog will fill up very quickly with it enabled.

  • Like 1

Share this post


Link to post

I'm not sure what's happening.  At first I thought maybe polling every second isn't giving unRAID enough time to spin down the disks i.e. the disks spinning down takes >1 second, so while the disks are spinning down, your plugin is sending a 'spin up'/'turbo' command every second.  So, I tried extending the polling to 15 seconds, but it still doesn't behave properly i.e. there are periods I know where only disk 3 is being accessed, yet the disks are all spun up.

 

What I've done for now is just schedule turbo mode overnight when I'm typically not at the PC working - that way I can have silent running during the day and faster writes if necessary overnight, when I tend to remotely access the server so noise isn't important.

Share this post


Link to post
1 hour ago, DZMM said:

your plugin is sending a 'spin up'/'turbo' command every second.

It doesn't.  It only sends the command to switch when the # of drives threshold is reached one direction or the other.

 

I'll look at it again, but unRaid does not log any spinups caused by drive access.  Only spindowns due to inactivity.

Share this post


Link to post
On 10/29/2017 at 7:10 PM, Squid said:

Only spindowns due to inactivity.

But, if turbo mode is on the chances of there being no activity is low i.e. the drives never spin down e.g. a kodi client decides to check if there's new files, radarr/sonarr/headphones etc moves a file, nzbget/deluge downloading etc etc. 

 

Maybe it's my setup and level of server activity, but I just can't get this plugin to spin down my drives once turbo is on.

Share this post


Link to post
1 hour ago, DZMM said:

But, if turbo mode is on the chances of there being no activity is low i.e. the drives never spin down e.g. a kodi client decides to check if there's new files, radarr/sonarr/headphones etc moves a file, nzbget/deluge downloading etc etc. 

 

Maybe it's my setup and level of server activity, but I just can't get this plugin to spin down my drives once turbo is on.

Plugin has nothing to do with spin ups / downs.  If the drives for whatever reason do not spin down, then turbo mode never gets disabled.

Share this post


Link to post

The "Done" button is missing from the dialog on the 2017.11.23 update to the plugin.

Share this post


Link to post
3 minutes ago, StanC said:

The "Done" button is missing from the dialog on the 2017.11.23 update to the plugin.

You're talking about when its performing the update?  Don't worry about it.  Its because I'm running some PHP code in the installation.  Happens to a few plugins out there.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now