Jump to content
nsta

Disk Spin Down Delay.....whats a good starting point?.,.

86 posts in this topic Last Reply

Recommended Posts

Proof of concept only

 

Good idea.

I think for some scripts (or projects) which users will always want, google code is the way.

For lil scriptlets the pastebin will work very well.

Might be something we use for syslogs too.

At what point does pastebin delete old files?

 

Share this post


Link to post

The best thing about pastebin is no registration required to post stuff so its instantly open with no skill required.

 

They has 3 options for retention, a day, a month or never.

 

One thing we need to test is can you edit the file without changing the id which is a potential security risk. I dont think you can but needs checked.

 

You will also notice that it has syntax highlighting for dozens of languages. And to top all that it has an option to see diffs

Share this post


Link to post

They has 3 options for retention, a day, a month or never.

 

That is a day, a month and "forever" (not never  :D )

Share this post


Link to post

My biggest concern with using pastebin (or anything) is that 1 year, 2 years, 5 years from now, when someone is searching the forums, and they come across an interesting post, that the referenced information is no longer available.  I have it happen to me all the time.  Dead links.  Deleted content.  The more self contained we stay the better from that perspective.

 

If we're talking about syslogs which are bulky and low value from a long-term perspective, I'm fine with that.  But if it is the slickest 30 line script ever created, I'd like to have it in the body of the message!  Perhaps the best method is what Joe L. usually does, and that is to list the code inline, and then include a a separate means to download it.

 

Just my $0.02.

Share this post


Link to post

Two points:

 

most big forums lose their data in my experience. someones going to say this one hasnt or that one hasnt but forums are not good places to keep info and istory has shown me many times over forums get killed and restarted

 

More importantly were not talking years. Natural evolution and we will have a complete addons site within that timescale.

 

IMO its a better solution than manual copy and paste and copy to unRAID. Its a complete chore to grab and upload scripts to unRAID and updating them is equally a chore. A one liner that users can copy and paste direct is far superior.

 

Its a stop gab but better than we have.

Share this post


Link to post

Does this script look right, if i were to add disk 1,2,and 3 to the script...and remove user???

 

#!/bin/bash

 

if [ ${DEBUG:=0} -gt 0 ]

  then set -x -v

fi

 

P=${0##*/}              # basename of program

R=${0%%$P}              # dirname of program

P=${P%.*}              # strip off after last . character

 

 

cache_loop()

{

 

    echo "$$" > /var/run/${P}.pid

    trap "rm -f /var/run/${P}.pid" EXIT HUP INT QUIT TERM

 

    logger -is -t${P} "Starting"

 

    while [ -f /var/run/${P}.pid ]

    do ls -R /mnt/disk1  >/dev/null 2>&1

    do ls -R /mnt/disk2  >/dev/null 2>&1

    do ls -R /mnt/disk3  >/dev/null 2>&1

      sleep 10

    done

 

    logger -is -t${P} "Terminating"

 

}

 

if [ -f /var/run/${P}.pid ]

  then echo "$0: already running? pidfile: /var/run/${P}.pid"

        ps -fp $(</var/run/cache_user.pid)

        exit

fi

 

cache_loop > /var/log/${P}.log 2>&1 &

JPID=$!

 

logger -is -t${P} "Spawned (Pid=$JPID)"

# ps -fp "$JPID"

disown "$JPID"

 

 

Share this post


Link to post

Does this script look right, if i were to add disk 1,2,and 3 to the script...and remove user???

 

#!/bin/bash

 

if [ ${DEBUG:=0} -gt 0 ]

   then set -x -v

fi

 

P=${0##*/}              # basename of program

R=${0%%$P}              # dirname of program

P=${P%.*}               # strip off after last . character

 

 

cache_loop()

{

 

    echo "$$" > /var/run/${P}.pid

    trap "rm -f /var/run/${P}.pid" EXIT HUP INT QUIT TERM

 

    logger -is -t${P} "Starting"

 

    while [ -f /var/run/${P}.pid ]

    do ls -R /mnt/disk1  >/dev/null 2>&1

    do ls -R /mnt/disk2  >/dev/null 2>&1

    do ls -R /mnt/disk3  >/dev/null 2>&1

       sleep 10

    done

 

    logger -is -t${P} "Terminating"

 

}

 

if [ -f /var/run/${P}.pid ]

   then echo "$0: already running? pidfile: /var/run/${P}.pid"

        ps -fp $(</var/run/cache_user.pid)

        exit

fi

 

cache_loop > /var/log/${P}.log 2>&1 &

JPID=$!

 

logger -is -t${P} "Spawned (Pid=$JPID)"

# ps -fp "$JPID"

disown "$JPID"

 

 

Too many "do" key-words.

Only need one.

Put it on its own line for clarity.

while [ -f /var/run/${P}.pid ]

    do

      ls -R /mnt/disk1  >/dev/null 2>&1

      ls -R /mnt/disk2  >/dev/null 2>&1

      ls -R /mnt/disk3  >/dev/null 2>&1

      sleep 10

    done

Share this post


Link to post

Awesome - Thanks!, will give it a go today.....is there any harm in having use both /user and disk settings, if the disks are on a user share anyway.....NAS advised for him it was slightly faster having it set to disk, so i thought why not....

 

Regards

Share this post


Link to post

you could parse mount or the like and get it to automatically do it for each drive rather than manually specify them

Share this post


Link to post

Only need one.

Put it on its own line for clarity.

 

while [ -f /var/run/${P}.pid ]

    do

      ls -R /mnt/disk1  >/dev/null 2>&1

      ls -R /mnt/disk2  >/dev/null 2>&1

      ls -R /mnt/disk3  >/dev/null 2>&1

      sleep 10

    done

 

better to do it like this

 

while [ -f /var/run/${P}.pid ]

    do

      ls -R /mnt/disk1 

      ls -R /mnt/disk2 

      ls -R /mnt/disk3 

      sleep 10

    done >/dev/null 2>&1

 

It's less operations/system calls  in the long run.

 

Share this post


Link to post

 

It looks like this will run every 10sec. and I assume this will not spin up disks to run the ls-r right, I guess that would defeat the purpose??

 

The assumption is incorrect. the purpose is to "try" to keep directory entries in the dentry cache by making them referenced as often as possible.

If the system is starved for ram, it still may free unreferenced dentry blocks, which later may cause a drive to spin up.

It all depends on how much ram you have and how often you do operations that require large amounts of ram.

 

I understand having this run on startup since all of the disks will be spun up by default and it will be able to cache all of the entries, but how does it work once the disks are spun down and then say one disk is spun up while watching a movie, how does it refresh the directory entries when the other disks are spun down.

 

If the dentry blocks are already in ram. then the dentry cache supplies the data rather then reading from disk and populating the dentry_cache.

Again, by referencing the dentry blocks often, it diminishes the chance they will be freed/removed from the cache.

 

 

Thanks for taking the time to answer WeeboTech. I just ordered 4gb of ram from NewEgg yesterday. I will install that when it comes in and setup your script....thanks!!!!

 

Ok I got the memory (Gskill 4gb) and installed it and setup the script. I rebooted and when it came up I saw drives 2-7 doing some reading. I spun down the drives and called a movies via it's share name and all disks (2-7) spun up getting read one by one ???

I then stopped the movie and spun the drives down and tried a different movie and it only spun the one disk but I thought it did that before before the memory.

 

Still testing but have a couple questions:

 

1) How can I verify how much memory is being seen....is there a command I can run from a telnet session?

2) I see the initial ls-R run at bootup but after 10sec I don't see anymore activity, by activity I mean the read count is not increasing on the web page. Is that the script knows not to run because the directory entries are all in the dentry cache, just more curious about this than anything else  ;D

 

Thanks!!!

Share this post


Link to post

1) How can I verify how much memory is being seen....is there a command I can run from a telnet session?

 

cat /proc/meminfo

or

top

 

root@Atlas ~ #grep dentry /proc/slabinfo

dentry            544736 545152    128  32    1 : tunables    0    0    0 : slabdata  17036  17036      0

 

2) I see the initial ls-R run at bootup but after 10sec I don't see anymore activity, by activity I mean the read count is not increasing on the web page. Is that the script knows not to run because the directory entries are all in the dentry cache, just more curious about this than anything else  ;D

 

Thanks!!!

 

If you do not see any new drive activity then the ls -R is being cached.

I'm not sure if the ls -R is enough. (but for now leave it).

 

 

Please add this line to the script somewhere in the first few lines.

It should have been in there already.

It only needs to be run once.

 

sysctl vm.vfs_cache_pressure=0

 

Share this post


Link to post

Thanks WeeboTech. I will try it out. I think I have that line but I don't think it's near the top.

Share this post


Link to post

For those who copied my script in line.

 

I see in allot of places the code block was quoted or copied.

 

Please add the following lines to the top after the variable definitions.

 

 



# Controls the tendency of the kernel to reclaim the memory which is used for
# caching of directory and inode objects.
# At the default value of vfs_cache_pressure=100 the kernel will attempt to
# reclaim dentries and inodes at a "fair" rate with respect to pagecache and
# swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer
# to retain dentry and inode caches.  Increasing vfs_cache_pressure beyond 100
# causes the kernel to prefer to reclaim dentries and inodes.
# This was 100, I changed to 0.

sysctl vm.vfs_cache_pressure=0

 

Share this post


Link to post

On a similar ilk... anyone who used the nohup example ive noticed that sometimes this causes the unformatted bug meqning you cant shut down via the web gui and one wrong button press could wipe your whole array. That bug REALLY needs squashed.

Share this post


Link to post

On a similar ilk... anyone who used the nohup example ive noticed that sometimes this causes the unformatted bug meqning you cant shut down via the web gui and one wrong button press could wipe your whole array. That bug REALLY needs squashed.

 

That is going to happen, nohup or disown.

If the shell triggers off at just the right moment keeping the filesystems busy, the unRAID array is not going to stop gracefully.

Share this post


Link to post

Then the obvious way is to either have unraid directly stop the script when the array is not up or have the script itself check the array status. The former is obviously the better solution.

 

thoughts?

 

I am quite concerned about this as there are a myriad of posts re: the ls -R trick and if this causes one user to mistakenly format their whole array its one too many.

Share this post


Link to post

Then the obvious way is to either have unraid directly stop the script when the array is not up or have the script itself check the array status. The former is obviously the better solution.

 

thoughts?

 

Good Idea.

I'll add some code in my version to check the status of /proc/mdcmd.

 

I am quite concerned about this as there are a myriad of posts re: the ls -R trick and if this causes one user to mistakenly format their whole array its one too many.

 

I agree. I consider this to be a bug rather then something that should just be fixed.

Anytime the array is stopped and a drive is not unmounted gracefully, it looks as if it is unformatted. This is incorrect.

It's so easy to make the mistake of formatting and erasing your data in this situation. It really does need to be addressed.

Share this post


Link to post

 

I agree. I consider this to be a bug rather then something that should just be fixed.

Anytime the array is stopped and a drive is not unmounted gracefully, it looks as if it is unformatted. This is incorrect.

It's so easy to make the mistake of formatting and erasing your data in this situation. It really does need to be addressed.

 

But the user would have to hit the format button right?? So to be safe don't hit the format button, right :)

Share this post


Link to post

easy to say for us guys who read this forum all the time... I'm sure many users out there never even visit the forum once ! :o

 

Share this post


Link to post

.....is spinning drives up and down ok for the HDD?...i mean doing it too often wont cause it to start deteriorating no???

 

Didn't really see a good answer to this ... so I'll respond.   Modern drives have a spinup cycle rating of at least 50,000 cycles -- some are double or triple that.   But using 50,000 as a baseline, that means if you spun the drives up every 15 minutes it would take about 1.5 years before the drives would, on average, begin to have problems spinning up.   That's not very long ... HOWEVER => a spindown setting of 15 minutes simply means the drive spins down after 15 minutes.   It would only then spin back up if you accessed it -- so unless you accessed your drives every 15 minutes plus 1 second (i.e. immediately after it spun down) repeatedly for 24 hours a day for a year-and-a-half, you wouldn't come close to that level.

 

More realistically, if you spin down after 15 minutes of inactivity, it probably means you're not using the system.   Since many of us use UnRAID for video servers; it's likely that a drive is either being accessed repeatedly (while streaming a movie) or ... if not accessed for 15 minutes, you're probably not using the ssytem at all.   If you assume a drive spins up ten times a day, then it would take nearly 14 years to hit the 50,000 spinup point -- this is probably far more realistic, even with a 15 minute spindown setting.

 

Bottom line:  For any reasonable usage pattern, you can set the spindown timing to any value greater than 10 minutes and you'll be fine.

 

 

Share this post


Link to post
Bottom line:  For any reasonable usage pattern, you can set the spindown timing to any value greater than 10 minutes and you'll be fine.

 

Thanks for an excellent contribution.  I have added it to the FAQ (link back only), here.  Feel free to edit.

Share this post


Link to post

Bottom line:  For any reasonable usage pattern, you can set the spindown timing to any value greater than 10 minutes and you'll be fine.

 

I'd like to, with respect, off another perspective.

 

Although looking at drive failure based solely on exceeding a certain number of spinups / spindown may justify short spindown cycles, I believe that the thermal impact of repeated heating up and cooling down would be damaging to the drive.  If all I did with my array was watch movies, the chances of accessing a drive 15 minutes after a movie was over might be very low.  But I use the array for all sorts of things - and if I start using it on a Saturday afternoon it would be quite easy for 15-45 minutes to go by without my accessing it.  It could go through 5-6 thermal cycles (heat up and cool downs) in one afternoon.  I'd much prefer that the thing just ran for 5-6 hours until I was done with it for the day.  I have my timeouts set to 5 hours, but I do manually spindown my drives before bed (usually).  Even with that long a timeout, most of my drives stay spundown easily 95% of the time.  Setting a lower spindown interval is not going to change that very much IMO.

 

This is the same argument about leaving a computer running versus powering it down all the time.  Mine tend to get turned on and off once a day.

Share this post


Link to post

Spinning down a drive doesn't power off the electronics -- nor does it cool the drive completely.  What it DOES do is let the drive cool down a few degrees and save a bit of power.  The drives don't undergo a full thermal cycle, as the electronics are still generating heat -- turning the system OFF will drop the temps a LOT more.  There's a difference between a power on/off cycle; and a spinup/spindown cycle -- I agree you would not want to have an excessive number of full power cycles every day.    Personally, I NEVER power off the electronics -- my computers are never turned off (nor do I use S3).

 

Whether you use 15 minutes or 5 hours for your setting, you're not likely to have more than a few spinup/spindown cycles per day in real world usage.  If your drives are spun down 95% of the time (i.e. they're only powered up for an average of 72 minutes per day, that means you're only accessing each drive on average once every 4.2 days (since once you access it, it spins for 5 hours).  In that case, a 15 minute spindown would make little difference in the number of spinup/spindown cycles -- but would save 4 hrs and 45 minutes of unnecessary spinup time.  If you frequently have 30-45 minutes of inactivity during "active" use times, I'd set the spindown to 1 hour.

 

There's a reason most commercial servers use LOUD, high-volume fans -- to keep the drive temps in the low 30's, as 40+ temps are generally undesirable (although the drives are typically rated to 50 or better (enterprise drives typically to 55).  But most UnRAID setups use much lower-volume (and quieter) fans, and will run in the 40's when spinning.  While this isn't necessarily "dangerous", it's a good idea to let the temps settle back into the 30's when possible ... and spinning the drives down achieves this easily.  My drives typically run 38-40 when highly active; and in the low 30's when spun down.

 

I agree, however, that as long as your thermals are good, it doesn't really matter if you spin the drives down.

 

 

 

Share this post


Link to post

.... What it DOES do is let the drive cool down a few degrees and save a bit of power. ...

 

this totally dependent on a number of factors. A few degrees can be a few or 20 depending on your case operating thermal characterisics.

 

...

Whether you use 15 minutes or 5 hours for your setting, you're not likely to have more than a few spinup/spindown cycles per day in real world usage. ...

 

Again this is too generic. It totally depends on your usage parameters.

 

.... But most UnRAID setups use much lower-volume (and quieter) fans, and will run in the 40's when spinning.  While this isn't necessarily "dangerous", it's a good idea to let the temps settle back into the 30's when possible ... ...

 

Sorry but this is way to generic as well. Who knows what temperatures most unRAID users run at. The majprity of users arent even members of this forum.

 

I agree thats 30s are better but low 40s are also fine. There is real impirical evidence via the massive google survey that low 40s might be better than low 30s although I totally admit this is almost a religious debate and were far from having enough data to draw conclusion for residential users.

 

Dont get me wrong I agree with the stuff your saying its just important we dont state assumptions as facts when giving advice to real people.

 

 

Persaonlly i thing 45 is a good number. This is based on my collection which is mostly 44minute videos.

 

We really should be logging spin ups and down per user because with this data we could start to come up with some math to recommend a schedule based on real world per setup usage.

 

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.