unRAID Server release 4.5-beta12 available


limetech

106 posts in this topic Last Reply

Recommended Posts

Download

 

This release addresses three separate disk spinup issues:

 

1. Hard drives that are controlled by a common host controller.  In this case, if I/O is occurring on a spunup drive, and an I/O request is issued to a spundown drive on the same host controller, then I/O will freeze on the first drive until the spundown drive gets spunup.  This is because the common host controller is busy spinning up a drive and can not process I/O requests for other drives it controls.

 

The most obvious example of multi-drive host controllers are IDE Master/Slave controllers: Two IDE drives connected via common ribbon cable.  If I/O is occuring on say the Slave, and the Master is spundown, an I/O reqeust on the Master will cause I/O on the Slave to freeze.  Other examples exist with SATA controllers which internally implement the same Master/Slave design.

 

The solution (or rather workaround) to this problem is a new feature called 'Spinup Groups'.  This feature permits you to group hard drives so that if an I/O request is received on any drive in the group, the unRAID driver will spinup any/all spundown hard drives also in the group.  For example, if the two drives (Master and Slave) on an IDE host controller are put in the same spinup group, then an acces to say the Slave drive will cause both it and the Master drive to be spunup.  Then later, if access to the Master occurs, I/O to the Slave will not be blocked.

 

Admittedly this is not an ideal solution because you might have drives spunup which never actually get accessed.  Depending on how you have storage assigned to disks you may not want to use Spinup Groups, or seletively use Spinup Groups.

 

Spinup Groups are assigned by clicking on the individual 'parity', 'disk1', ..., 'cache' links on the webGui Main page.  There is a new configurable field there called "Spinup group(s)".  Here you can enter one or more identifiers separated by spaces, e.g:

 

Spinup group(s): host1

 

When the array is started, the spinup group identifers are examined for each drive, and drives with common identifiers are grouped together for spinup purposes.

 

When there are multiple drives in the same group, they are all spunup/spundown together, and the "Spindown delay" for the group is taken to be the longest of any member in the group.

 

Important: when you start -beta12 for the first time, you will find that the spinup groups are filled in with identifiers.  These identifiers are extracted from the linux /sys/block entries: in the case of IDE drives we group according to IDE host controllers; in the case of SATA/SAS drives, according to 'host' controllers.  To my knowledge this is sufficient to identify hard drives on the same host controllers.  Also, whenever a device is 'assigned' via the Devices page, we also initialize it's Spinup Group with the host controller identifier.  If you do not want to use Spinup Groups, then you must go to each 'disk' page and erase the Spinup group(s) field.  (sorry).

 

Finally, Spinup Groups can be used to force spinup of drives with similar I/O likelyhood.  For example, suppose you have a Music share which spans 2 or more disks.  Further suppose you have playlists that randomly access your music.  If you want to ensure no interruption in music waiting for disks to spinup, you can assign each of the Music share disks to the same group, eg., 'music'.  That way once you start playing music all the music disks on the server will spin up together.  Later when the music stops and the last disk has been idle for "spindown delay" time, all disks will be spundown together.

 

Another Note: Since the Cache drive is not at present controlled by the unRAID driver, it's Spinup Group is ignored.

 

2. User Share access to a spundown disk causes I/O to freeze on a stream from another User Share disk.  This was caused by a bug which is now fixed.

 

3. An I/O stream via Windows PC freezes when same Windows PC is used to access data on a spundown disk.  This is a Windows problem and can not be fixed by unRAID OS.  From examining SMB connection traces, what I have determined is this: When a Windows PC (client) connects to a Server, it opens a single "SMB Connection".  Over this connection it sends it's requests: lookups, read, writes, etc, no matter which Share is being accessed (that is, all shares on the same server are accessed over a single SMB connection).  Hence, say for example, video is being read from the server and rendered on a Windows PC; if from the same Windows PC you now try to access another share with a spundown drive, Windows will use the same connection, resulting in "freeze" of the video while the I/O reqeust to the spundown drive is being serviced.

 

I have spent a couple of hours looking into this issue on the Windows side and have not found any way to tell Windows to open multiple SMB connections to multiple shares on the same server.  If anyone finds a solution to this, please send to tomm@lime-technology.com.

 

One more change in this release that may affect some users: There is a mode in Samba (linux-side Windows Networking), called "asynchronous I/O" (aio).  When enabled, this causes Samba to internally spawn a separate thread to process each type of SMB request vs. processing them one-by-one.  Prior to 4.5 aio was disabled & it was only enabled to see what effect it had on system loading.  Well, the results are indeterminate, and I think most users benefit by the default behavior (aio off), so that is restored in this release.  To re-enable aio, please create the file "config/smb-extra.conf" with these lines:

aio read size = 1
aio write size = 1

 

Please reply to this thread to report any bugs or ask questions related to this release only.  Thanks!

 

unRAID Server 4.5-beta12 Release Notes
======================================

Changes from 4.5-beta11 to 4.5-beta12
-------------------------------------

New features:
- Added 'Spinup groups' feature.

Bug fixes:
- Fix user share I/O freeze problem when spinning up disks.

Other changes:
- Disable Samba async I/O by default.  Can be re-enabled by adding these lines in 'config/smb-extra.conf':
      aio read size = 1
      aio write size = 1


Changes from 4.5-beta10 to 4.5-beta11
-------------------------------------

Bug fixes:
- Fix webGui crash caused by uninitialized md_num_stripes.
- Restore Samba default "unix extensions = No"


Changes from 4.5-beta9 to 4.5-beta10
------------------------------------

Bug fixes:
- Fix user security mode password handling.


Changes from 4.5-beta8 to 4.5-beta9
-----------------------------------

Bug fixes:
- Fix race condition on startup where smbd was not always started.
- Fixed enabling Samba asynchronous I/O, which wasn't really enabled in -beta8.
- Fixed symlink creation and handling in user share file system.

Other changes:
- Upgrade samba to 3.4.3
- Upgrade linux kernel to 2.6.31.6
- Make unraid driver tunables (md_num_stripes, md_write_limit, md_sync_window) configurable via webGui Settings page.
- Upgrade pkgtools in order to support slackware packages which have .txz extension.
- Change e100 and Broadcom GigE drivers from module to built-in so that kernel includes the firmware.
- Enable name and attribute caching in user share file system to speed up SMB large directory operations.


Changes from 4.5-beta7 to 4.5-beta8
-----------------------------------

New features:
- Enable 'Cache Drive' feature with 'Plus' registration key.

Bug fixes:
- Fixed path length limitation in user share file system.
- Fixed incorrect handling of file/directory mode and extended attributes in user share file system.
- Fixed intermittant kernel crash with more than 19 hard drives in array.

Other changes:
- Increased write peformance and added three user-tunables:
        md_num_stripes to set the stripe cache size
md_write_limit to set a ceiling on most number of active write stripes
md_sync_window to set the parity sync/check window size in stripes
- The 'mover' script now uses 'rsync' to move files from the Cache drive to the array.
- Upgrade linux kernel to 2.6.31.5
- Changed linux kernel preemption model to "Voluntary Kernel Preemption".
- Enable Samba asynchronous I/O.
- Changed 'ide_core' from module to built-in.


Changes from 4.5-beta6 to 4.5-beta7
-----------------------------------

Bug fixes:
- If 'config/style.css' is not present, then don't tell browser to load it.
- Fixed kernel crash when number of drives > 17, now correctly supports up to 20 drives ('Pro' issue only).

Other changes:
- Prevent disks appearing 'unformatted' as a result of incomplete Stop Array operation.
- Add driver for Adaptec 2410SA PCI RAID card (untested).
- Upgrade linux kernel to 2.6.30.8
- Upgrade samba to 3.3.7
- Upgrade fuse to 2.8.1


Changes from 4.5-beta5 to 4.5-beta6
-----------------------------------

Bug fixes:
- Fix ntp startup problem.
- Fix bogus temperature values being displayed when disks are spun down (bug introduced in -beta5).

Other changes:
- Permit spaces in Active Directory account login username.  Any valid AD character should now be possible EXCEPT for the percent '%' symbol.  This is because the '%' symbol is used by linux 'net' command as a separator between username and password, as in:
net ads join -U username%password


Changes from 4.5-beta4 to 4.5-beta5
-----------------------------------

New features:
- Added new share allocation method called "Fill-up".
- Added "Min free space" setting for each share.
- Special handling of split level explictly set to "0".
- The 'mover' will now delete empty directories on the cache disk.
- After looking in the 'config' directory on the Flash for the license key file (for backward compatibility), if not found, unRAID Server OS will look in the root of the Flash.  This allows easier backup/restore of the 'config' directory contents.

Bug fixes:
- Fix wrong 'memtest' in zip file - oops.
- Fix powerdown script.
- Fixed problem where drives would sometimes spin down immediately following spin up.

Other changes:
- Don't store AD adminstrator password, and don't display personal information in the system log.
- Can now use any printable characters in password strings.
- When disabling NCQ don't try to set queue_depth to 1 if it's already set to 0.
- Upgrade ntp from version to 4.2.4p0 to 4.2.4p6.  Also save/restore ntp drift file to/from Flash.
- Upgrade linux kernel to 2.6.29.1
- Support Marvell SAS driver.


Changes from 4.5-beta3 to 4.5-beta4
-----------------------------------

New features:
- Increased maximum number of array devices from 16 to 20 (Pro only).
- Pressing Power button gracefully shuts down the server.
- Disable NCQ on all disk devices that support NCQ.  This typically results in much better write throughput.  A setting called "Force NCQ disabled [yes/no]" is also available in the Disk section of the Settings page of the System Management Utility to override this new behavior.  That is, if this setting is 'yes', then we force NCQ off; if setting is 'no', we leave NCQ queue_depth as-is, ie, whatever linux driver sets it to.

Bug fixes:
- Fixed syslog rotation problem - syslog was rotated, but then syslogd was not restarted.

Other changes:
- Support SAS (Serially-Attached SCSI).
- Support Initio 162x SATA chipset.
- Support the motherboard speaker (beeping).
- Added 'lm_sensors' package.
- Upgrade Samba to version 3.3.3.
- Upgrade memtest to version 2.11 in release zip file.


Changes from 4.5-beta2 to 4.5-beta3
-----------------------------------

Known issues:
- The 'reiserfs' file system is built with kernel-option to enable extended attribute support.  This is necessary for Active Directory.  Even if file system is not mounted with extended attributes, reiserfs still seems to create a hidden file called '.reiserfs_priv' in the volume root. This file is harmless and does not appear in any share.

Bug fixes:
- Fixed problem where all Flash files appeared to have Hidden/System/Archive all set.
- Fixed upgrade problem where 'simple security' was not being initialized properly.
- After formatting a new disk, the 'File system type' was not being updated.
- If cache disk present, should allow 'use cache disk' option in creating new share.
- Fixed problem with mover not moving.

Other changes:
- Changed name of samba include file introduced in -beta2 from 'boot/config/smb.extra' to 'boot/config/smb-extra.conf'.
- Upgrade Samba to (patched) version 3.3.2.  The patch is a bug fix that prevented windows client from removing Read-only attribute (previous versions of samba 3.3.x fail with 'Access denied').
- If security model is not Active Directory don't mount the data disks with acl & extended attributes enabled.
- Prevent recording (ie, writing) 'last access time' when directories and files are read on the Flash.
- Added 'lsof' command.
- Include 'Hardware Monitoring' in the linux kernel build along with this device support:
  AMD Athlon64/FX or Opteron temperature sensor
  Intel Core (2) Duo/Solo temperature sensor
  ITE IT87xx and compatibles
  Winbond W83627HF, W83627THF, W83637HF, W83687THF, W83697HF
  Winbond W83627EHF/DHG


Changes from 4.4.2 to 4.5-beta2
-------------------------------

New features:
- Support Active Directory Service (ADS).  This lets an unRAID server join an Active Directory (AD) domain.
- System Management Utility will now use a CSS style sheet file on the Flash (config/style.css) if present.
- May now read syslog directly via browser by referencing 'http://tower/log/syslog' (substitute 'tower' with your server name).  Actually any 'file' in the /var/log directory can be read via 'http://tower/log/file'.
- May now read arbitrary files on disk and user shares via http protocol by referencing URL 'http://tower/share/<diskN>/...' or 'http://tower/share/user/<sharename>/...' (substitute 'tower' with your server name).
- Samba configuration now 'includes' the file on the Flash 'config/smb.extra' if present.  This is included at the end of the 'global' section just before the share definitions.  This may be used to customize the Samba configuration.
- Added control to enable/disable 'mover' logging.

Bug fixes:
- With user mode security enabled, would not accept 'root' share login until password was set at least once.
- Fixed problem in 'mover' script where mover would attempt to move objects in a top-level directory staring with a '.' character.  These would all fail and cause excessive syslog messages.
- Fixed bug in 'logrotate' which would prevent syslog from rotating.

Other changes:
- Part of adding AD support: Removed "User security [enabled/disabled]" control from Shares page, and added "Share security [simple/User Level/Active Directory]" control to Settings page:
  unRAID 'Basic' (free version) supports only 'Simple' share security model;
  unRAID 'Plus' supports 'Simple' and 'User Level' share security models only;
  unRAID 'Pro' supports all share security models ('Simple', 'User Level', and 'Active Directory').
- Removed System Management Utility control for setting SMB ports; this can be done via 'config/smb.extra' if desired.
- Change spin-down logic to account for external programs spinning drives up/down.
- Per user request, added '/usr/lib/libstdc++.so.6.0.9'
- Upgraded to linux kernel 2.6.28.4.
- Upgraded to samba 3.3.0.


Upgrade Instructions (Please Read Carefully)
============================================

If you are currently running unRAID Server 4.2-beta1 or higher (including 4.2.x 'final'), please copy the following files from the new release to the root of your Flash device:
    bzimage
    bzroot

If you are currently running unRAID server 4.0 or 4.1, please copy the following files from the new release to the root of your Flash device:
    bzimage
    bzroot
    syslinux.cfg
    menu.c32
    memtest

This can be done either by plugging the Flash into your PC or, by copying the files to the 'flash' share on your running server.  The server must then be rebooted.

If you are currently running unRAID Server 3.0-beta1 or higher, please follow these steps to upgrade:

1. Referring to the System Management Utility 'Main' page, make a note of each disks's model/serial number; you will need this information later.

2. Shut down your server, remove the Flash and plug it into your PC.

3. Right-click your Flash device listed under My Computer and select Properties.  Make sure the volume label is set to "UNRAID" (without the quotes) and click OK.  You do NOT need to format the Flash.

4. Copy the files from the new release to the root of your Flash device.

5. Right-click your Flash device listed under My Computer and select Eject.  Remove the Flash, install in your server and power-up.

6. After your server has booted up, the System Management Utility 'Main' page will probably show no devices; this is OK, navigate to the 'Devices' page. Using the model/serial number information gathered in step 1, assign each of your hard drives to the correct disk slot.

7. Go back to the 'Main' page and your devices should appear correctly.  You may now Start the array.


If you are installing this release to a new Flash, please refer to instructions on our website at:

http://www.lime-technology.com/joomla/unraid-os

Link to post
  • Replies 105
  • Created
  • Last Reply

Top Posters In This Topic

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

 

Yes, that is expected & perhaps I should have mentioned.  In order to keep track of spinup groups and drive spinup status, this state info is maintained in the unRAID driver (because the driver needs to spin up drives based on incoming I/O requests) and new driver commands were added to have the driver do the spinup/down.  So any command to change the spinup/down state done outside the array, will result in inconsistencies, i.e., it should not be done.  unMenu will have to be changed (by Joe?) to use these new driver commands:

 

mdcmd spindown <slot>
mdcmd spinup <slot>

 

Note these commands do not apply to the Cache disk.

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

What happens when you simply press "refresh" on the unRAID main screen.  (The actual unRAID screen, not the one in the unMENU menu's window.)
Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

 

The main unRAID webGUI does not have any knowledge that you are spinning up or down drives via another GUI.  If you hit the refresh button on the main unRAID webGUI you will probably see that the drives are now reported as spinning.  The webGUI does not auto-refresh, you have to do it manually.

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

What happens when you simply press "refresh" on the unRAID main screen.  (The actual unRAID screen, not the one in the unMENU menu's window.)

 

Nothing. I was [Refreshing]  as well as reloading the browser (Chrome) at each step and there was no change.

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

 

Yes, that is expected & perhaps I should have mentioned.  In order to keep track of spinup groups and drive spinup status, this state info is maintained in the unRAID driver (because the driver needs to spin up drives based on incoming I/O requests) and new driver commands were added to have the driver do the spinup/down.  So any command to change the spinup/down state done outside the array, will result in inconsistencies, i.e., it should not be done.  unMenu will have to be changed (by Joe?) to use these new driver commands:

 

mdcmd spindown <slot>
mdcmd spinup <slot>

 

Note these commands do not apply to the Cache disk.

You are correct, currently to spin up a disk, unMENU reads a random block of data from /dev/hda, sda, etc.  This is not seen by the "md" driver and the fact that the drive is spinning is unknown.    Currently, unMENU uses /usr/sbin/hdparm -y /dev/hda, dsa, etc to spin down devices.

 

I'll change it to use your new "set" commands.  

 

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

 

If maintenance commands such as smartctl are needed, can we use hdparm to learn the current sleeping/spinning status, and then invoke one of the other "set" commands to get the unRAID driver in sync with what is really happening?

 

Joe L.

Link to post

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

Joe L.

 

I think this is going to be needed otherwise any outside scripts which change the status is going to cause inconsistencies in the known state.

How does emhttp do it?

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

 

Yes, that is expected & perhaps I should have mentioned.  In order to keep track of spinup groups and drive spinup status, this state info is maintained in the unRAID driver (because the driver needs to spin up drives based on incoming I/O requests) and new driver commands were added to have the driver do the spinup/down.  So any command to change the spinup/down state done outside the array, will result in inconsistencies, i.e., it should not be done.  unMenu will have to be changed (by Joe?) to use these new driver commands:

 

mdcmd spindown <slot>
mdcmd spinup <slot>

 

Note these commands do not apply to the Cache disk.

You are correct, currently to spin up a disk, unMENU reads a random block of data from /dev/hda, sda, etc.  This is not seen by the "md" driver and the fact that the drive is spinning is unknown.    Currently, unMENU uses /usr/sbin/hdparm -y /dev/hda, dsa, etc to spin down devices.

 

I'll change it to use your new "set" commands.  

 

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

 

If maintenance commands such as smartctl are needed, can we use hdparm to learn the current sleeping/spinning status, and then invoke one of the other "set" commands to get the unRAID driver in sync with what is really happening?

 

Joe L.

 

The driver issues "STANDBY IMMEDIATE" ATA command to put drive in standby mode (ie, spin down).  I believe this is the same as the "hdparm -y" option.  To spin up, driver issues "IDLE" ATA command to put drive in idle mode (ie, spun-up).  If unMenu issues a random read to spin-up, that should actually work but will result in spinning up all drives of the group.

 

Reason driver needs to keep track of this is that on each I/O request, we need to see if there is a spun down disk in the target disk's group.  This has to be fast so that performance is not affected, so the driver maintains "state" of disk spin-up or spun-down status.

Link to post

Just playing with disk spin-up/down.  If I spin-up one disk using UnMenu it does not show active in UnRAID Main.  If I click the [spin Down] button nothing happens.  Clicking [spin Up]  and then [spin Down] does work though.  The response is the same spining-up all the drives one by one using UnMenu. None of it registers on the UnRAID Main menu until I click [spin Up].

 

 

Yes, that is expected & perhaps I should have mentioned.  In order to keep track of spinup groups and drive spinup status, this state info is maintained in the unRAID driver (because the driver needs to spin up drives based on incoming I/O requests) and new driver commands were added to have the driver do the spinup/down.  So any command to change the spinup/down state done outside the array, will result in inconsistencies, i.e., it should not be done.  unMenu will have to be changed (by Joe?) to use these new driver commands:

 

mdcmd spindown <slot>
mdcmd spinup <slot>

 

Note these commands do not apply to the Cache disk.

You are correct, currently to spin up a disk, unMENU reads a random block of data from /dev/hda, sda, etc.  This is not seen by the "md" driver and the fact that the drive is spinning is unknown.    Currently, unMENU uses /usr/sbin/hdparm -y /dev/hda, dsa, etc to spin down devices.

 

I'll change it to use your new "set" commands.  

 

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

 

If maintenance commands such as smartctl are needed, can we use hdparm to learn the current sleeping/spinning status, and then invoke one of the other "set" commands to get the unRAID driver in sync with what is really happening?

 

Joe L.

 

The driver issues "STANDBY IMMEDIATE" ATA command to put drive in standby mode (ie, spin down).  I believe this is the same as the "hdparm -y" option.  To spin up, driver issues "IDLE" ATA command to put drive in idle mode (ie, spun-up).  If unMenu issues a random read to spin-up, that should actually work but will result in spinning up all drives of the group.

If we issue an hdparm -y command to a /dev/sdb disk, the "md" driver will not know we spun the disk down.   In the same way, to spin up a drive, currently unMENU is not reading the random block from the "/dev/md" device, but it is spinning up by reading from the "device" returned by "mdcmd status" in the various rdevName fields.  These are the hda, hdb, sda, sdb, devices... again the "md" device will not have a clue the drive is spinning up.

Reason driver needs to keep track of this is that on each I/O request, we need to see if there is a spun down disk in the target disk's group.  This has to be fast so that performance is not affected, so the driver maintains "state" of disk spin-up or spun-down status.

I completely understand the need for the tracking of the status in the "md" driver...    I think however a way to sync the tracked state and the physical state exists.  They will get out of sync... I can guarantee it.

 

Joe L.

 

 

Link to post

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

Joe L.

 

I think this is going to be needed otherwise any outside scripts which change the status is going to cause inconsistencies in the known state.

How does emhttp do it?

 

emhttp does it by issuing 'spinup' commands wihen it starts - thus establishing the spinup state for each disk.  I need to think about how to detect outside spinup/spindown but here is really the crux of it: a spun-down drive is spun up "on demand" as a result of receiving a read request (and others), therefore the driver has to have the information available, at the time an I/O request is received, to "know" whether other drives not immediately involved in I/O need to be spun up.  So if the driver thinks a drive is already spun up, but outside code spins it down, the driver will not spin it back up when ordinarily it should.  This is because we can't have the driver issue actual status command to every disk upon every I/O request - would kill performance.

Link to post

Important: when you start -beta12 for the first time, you will find that the spinup groups are filled in with identifiers.  These identifiers are extracted from the linux /sys/block entries: in the case of IDE drives we group according to IDE host controllers; in the case of SATA/SAS drives, according to 'host' controllers.  To my knowledge this is sufficient to identify hard drives on the same host controllers.  Also, whenever a device is 'assigned' via the Devices page, we also initialize it's Spinup Group with the host controller identifier.  If you do not want to use Spinup Groups, then you must go to each 'disk' page and erase the Spinup group(s) field.  (sorry).

 

Messy.

 

(I am currently still with 4.5-beta11)

In the unRAID Settings page, in the Disk settings section, my "Default spin down delay" is set to "Never".

I would like that to mean that I don't want unRAID to get involved in any spinning Up/Down of any disks.

 

Purko

 

 

Link to post
...So if the driver thinks a drive is already spun up, but outside code spins it down, the driver will not spin it back up...

And that's a good thing!  Otherwise you'd be spinning up the drive now, and potentially creating a freeze now, just so you could prevent eventual future freeze. Would make no sense. So, if it's down, it's down -- you leave it that way.

Link to post

Important: when you start -beta12 for the first time, you will find that the spinup groups are filled in with identifiers.  These identifiers are extracted from the linux /sys/block entries: in the case of IDE drives we group according to IDE host controllers; in the case of SATA/SAS drives, according to 'host' controllers.  To my knowledge this is sufficient to identify hard drives on the same host controllers.  Also, whenever a device is 'assigned' via the Devices page, we also initialize it's Spinup Group with the host controller identifier.  If you do not want to use Spinup Groups, then you must go to each 'disk' page and erase the Spinup group(s) field.  (sorry).

 

Messy.

 

(I am currently still with 4.5-beta11)

In the unRAID Settings page, in the Disk settings section, my "Default spin down delay" is set to "Never".

I would like that to mean that I don't want unRAID to get involved in any spinning Up/Down of any disks.

 

Purko

 

Yes, if you set delay to "Never", unRAID will never try to spin a drive down; but, a drive can get spun-up under these circumstances:

1. as a result of I/O (of course),

2. as a result of reading temperature.  If unRAID thinks drive is spun-up, when actually outside code spun it down, when unRAID tries to get the disk temperature via SMART, virtually all hard drives will get spun-up as a result. [probably could fix this if I have to]

3. as a result of 'Stopping' the array - in this case the drives will spin up in order to un-mount them.

Link to post

Important: when you start -beta12 for the first time, you will find that the spinup groups are filled in with identifiers.  These identifiers are extracted from the linux /sys/block entries: in the case of IDE drives we group according to IDE host controllers; in the case of SATA/SAS drives, according to 'host' controllers.  To my knowledge this is sufficient to identify hard drives on the same host controllers.  Also, whenever a device is 'assigned' via the Devices page, we also initialize it's Spinup Group with the host controller identifier.  If you do not want to use Spinup Groups, then you must go to each 'disk' page and erase the Spinup group(s) field.  (sorry).

 

Messy.

 

(I am currently still with 4.5-beta11)

In the unRAID Settings page, in the Disk settings section, my "Default spin down delay" is set to "Never".

I would like that to mean that I don't want unRAID to get involved in any spinning Up/Down of any disks.

 

Purko

 

Yes, if you set delay to "Never", unRAID will never try to spin a drive down; but, a drive can get spun-up under these circumstances:

1. as a result of I/O (of course),

2. as a result of reading temperature.  If unRAID thinks drive is spun-up, when actually outside code spun it down, when unRAID tries to get the disk temperature via SMART, virtually all hard drives will get spun-up as a result. [probably could fix this if I have to]

3. as a result of 'Stopping' the array - in this case the drives will spin up in order to un-mount them.

 

I understand that.

Still, it should mean that unRAID will not be doing any proactive spinning Up or Down of disks.

 

Link to post

Important: when you start -beta12 for the first time, you will find that the spinup groups are filled in with identifiers.  These identifiers are extracted from the linux /sys/block entries: in the case of IDE drives we group according to IDE host controllers; in the case of SATA/SAS drives, according to 'host' controllers.  To my knowledge this is sufficient to identify hard drives on the same host controllers.  Also, whenever a device is 'assigned' via the Devices page, we also initialize it's Spinup Group with the host controller identifier.  If you do not want to use Spinup Groups, then you must go to each 'disk' page and erase the Spinup group(s) field.  (sorry).

 

Messy.

 

(I am currently still with 4.5-beta11)

In the unRAID Settings page, in the Disk settings section, my "Default spin down delay" is set to "Never".

I would like that to mean that I don't want unRAID to get involved in any spinning Up/Down of any disks.

 

Purko

 

Yes, if you set delay to "Never", unRAID will never try to spin a drive down; but, a drive can get spun-up under these circumstances:

1. as a result of I/O (of course),

2. as a result of reading temperature.  If unRAID thinks drive is spun-up, when actually outside code spun it down, when unRAID tries to get the disk temperature via SMART, virtually all hard drives will get spun-up as a result. [probably could fix this if I have to]

3. as a result of 'Stopping' the array - in this case the drives will spin up in order to un-mount them.

 

I understand that.

Still, it should mean that unRAID will not be doing any proactive spinning Up or Down of disks.

 

 

Isn't that what I just said it does mean?  Anyway, what is your concern about this?  Are you going to be spinning drives up/down via some other mechanism?

Link to post
Are you going to be spinning drives up/down via some other mechanism?

 

Well, yes. I've been doing it all along.

 

My reason is, in my server I have a whole bunch of disks which are outside the protected arrary.  As unRAID doesn't care about spinning them up/down, I have to take care of that myself.  So, while I'm at it, I may as well take care of all the disks the same way.  In my 'go" script I have a "hdparm -S" for each "/dev/[hs]d?" in this machine, thus saving the unRAID driver the trouble.  My thinking is, less overhead for the driver should in theory make it run more efficently, which is important for a low-spec server.  Same goes for spinning UP stuff, for which I don't really care.

 

Purko

 

 

Link to post

Are you going to be spinning drives up/down via some other mechanism?

 

Well, yes. I've been doing it all along.

 

My reason is, in my server I have a whole bunch of disks which are outside the protected arrary.  As unRAID doesn't care about spinning them up/down, I have to take care of that myself.  So, while I'm at it, I may as well take care of all the disks the same way.  In my 'go" script I have a "hdparm -S" for each "/dev/[hs]d?" in this machine, thus saving the unRAID driver the trouble.  My thinking is, less overhead for the driver should in theory make it run more efficently, which is important for a low-spec server.  Same goes for spinning UP stuff, for which I don't really care.

 

Purko

 

 

 

Ok, fair enough.  The problem you will see with this release is that your spun-down array drives will spin back up as soon as you refresh the webGui screen because it will try and get the temperatures.  BTW, originally unRAID also used the method of setting a spin-down timer in the drive, but quickly found out that many drives didn't support this properly.  Anyway, use of Spinup Groups precludes this option now.

Link to post

There is a potential for other commands to spin up a drive...  smartctl for example...   Does the new driver have a command to "get" in sync with the current spinning status of a drive (as opposed to specifically spinning it up or down)?

Joe L.

 

I think this is going to be needed otherwise any outside scripts which change the status is going to cause inconsistencies in the known state.

How does emhttp do it?

 

emhttp does it by issuing 'spinup' commands wihen it starts - thus establishing the spinup state for each disk.   I need to think about how to detect outside spinup/spindown but here is really the crux of it: a spun-down drive is spun up "on demand" as a result of receiving a read request (and others), therefore the driver has to have the information available, at the time an I/O request is received, to "know" whether other drives not immediately involved in I/O need to be spun up.  So if the driver thinks a drive is already spun up, but outside code spins it down, the driver will not spin it back up when ordinarily it should.  This is because we can't have the driver issue actual status command to every disk upon every I/O request - would kill performance.

 

I was not implying the state should be synced on every I/O request. Perhaps a command for the md driver which goes through the drive table and syncs state (only at that moment).

Perhaps a parameter to the status command whereby the state could be queried and updated.

 

This allows other programs tell mdcmd something may have changed externally, and/or get 100% accurate states.

 

I asked how emhttp did it because if I issue an hdparm command, and refresh emhttp the green drive status will show it's new status.

Just some ideas.

Link to post

I was not implying the state should be synced on every I/O request. Perhaps a command for the md driver which goes through the drive table and syncs state (only at that moment).

Perhaps a parameter to the status command whereby the state could be queried and updated.

 

This allows other programs tell mdcmd something may have changed externally, and/or get 100% accurate states.

 

I asked how emhttp did it because if I issue an hdparm command, and refresh emhttp the green drive status will show it's new status.

Just some ideas.

 

Understood.  Let me think about this.  In meantime, one of the variables returned in 'mdcmd status" is "rdevLastIO.x" (one for each disk) which records the timestamp of the last I/O operation on that disk (through the md driver).  This gets set with the system time on each I/O (this is a very fast operation).  When md driver spins up a disk, current time is also recorded here.  When md driver spins down a disk, the value gets set to 0.  This is the thing checked on each I/O for other disks in the spinup group.  So any spinup/spindown or I/O outside the md driver to a hard disk within the array can cause confusion.

 

NOTE: for those following this thread who just use 'stock' unRAID OS, this discussion has no effect on normal use - spinup/down all works as it should (barring any bugs).

Link to post
The problem you will see with this release is that your spun-down array drives will spin back up as soon as you refresh the webGui screen because it will try and get the temperatures.

 

Hmmm.... That's not cool.  It shouldn't be too hard to check the drive status with "hdparm -C" or something before taking any temperatures.  And, the refresh from the webGui is a good time to sync the driver's idea about the drive status with real life.

 

Purko

 

 

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.