Jump to content
jonp

Do you use spin up groups?


107 posts in this topic Last Reply

Recommended Posts

I would also be curious how many of these use cases could be better handled with data disk balancing and some upstream bug reports to software vendors

 

One of the nicest features of UnRAID is the user shares, which allow multiple disks to be seamlessly combined into a large storage unit without user involvement.    While I understand that some like to "balance" their data, this is NOT necessary and somewhat defeats the "seamless-ness" of the user shares.  It does make sense to use split level settings that maintain all components of a particular media element together (i.e. if you're storing DVDs in their original .VOB's, you want all VOB's for a particular DVD together) ... but even that isn't necessary except to eliminate a spin-up requirement when switching between VOB's.

 

Further, even if the data is "balanced", spin-up groups are still useful => the issue isn't that they prevent the need to spin up another disk associated with the currently streaming bit of media;  it's that there's an interaction between disks on the same controller channel (or even the same controller) such that when you're spinning up a disk on that controller, all activity on other disks on the same unit is suspended during the spin-up.    If you have disks that exhibit that interaction, you can use a spin-up group so that when any disk in that group spins up, all the others do as well, so any subsequent use of one of the other disks won't impact your current media steam/copy operation/write activity/etc.

 

 

Share this post


Link to post

I dont really care either way but I have to say all the plus cases for it so far feel to me like work arounds for client problems and spinning up wide arrays feels like one of things i hated about old fashioned RAID.

 

I hold out judgement because I dont believe we have any info why it is being removed other than it is perceived its not being used. That is never going to be the case as in any large userbase there will always be people using every feature in some way.

Share this post


Link to post

The spin-up groups doesn't just solve an IDE drive issue. It also eliminates playback pauses if the same Windows PC is playing back a file and then tries to access a file on a spun-down disk at the same time.

 

I dont really care either way but I have to say all the plus cases for it so far feel to me like work arounds for client problems....

 

It's a work around for Windows.

 

 

Share this post


Link to post

That sounds fixable in unRAID to me. There should be no real reason why unRAID should stall like this (conceptually at least).

Share this post


Link to post

That sounds fixable in unRAID to me. There should be no real reason why unRAID should stall like this (conceptually at least).

 

unRAID doesn't cause the stall but the bandage fix on the unRAID side is to use spinup groups. It's a Windows problem. Windows basically creates one connection to the server so when a second program tries to access a spun-down disk the connection hangs until it gets a response.

Share this post


Link to post

I believe this feature was needed if people had configured their user shares poorly so that files for a single movie got split across multiple physical disk in the share. This was happening with regularity in the days of multiple files needed to store a single movie (e.g., DVD rips). When one file finished playing, there could be a noticeable lag before the next one started if it was on a spun down disk. Setting the user share settings properly from the beginning prevented this issue, and today most movies are converted to mkv's, which is a single file and not subject to the issue described.

 

Some people may have a considerably large set of DVD rips from the old days that have this issue and they may have spin up groups configured to avoid the stuttering described. I don't think IDE vs SATA matters, although I guess if the feature only works on IDE drives you may be right. I never used it.

Share this post


Link to post

I believe this feature was needed if people had configured their user shares poorly so that files for a single movie got split across multiple physical disk in the share.

 

No ... at least not in the instances I saw (both mine and others).    The issue is that some controllers -- whether IDE or SATA -- will "hang" activity on one attached when other attached disks on the same controller channel or group are being spun up.    Has nothing to do with whether or not the other disk is on the same share - or even if the access is from the same client.

 

Say, for example, that the wife/kids are streaming a movie from Disk 3, and you decide to copy a file (pictures, a document, a spreadsheet, or whatever) from Disk 5, which isn't currently spun up.    On many controllers, spinning up Disk 5 won't have any impact on Disk 3 ... but on some it will cause Disk 3 to hang while Disk 5 is spinning up.    If you have this kind of interaction between disks, you can eliminate the problem by putting them in a spin-up group, so whenever Disk 3 is spun up, Disk 5 is also, so that any access to Disk 5 isn't going to cause an issue with anything currently being streamed/copied from Disk 3.    Note that this isn't a Windows issue, Mac issue, or whatever ... what OS the client is using is irrelevant ... it's caused by the controller.

 

You could, of course, if you know you have media that spans multiple disks, use spin-up groups to do as you noted -- prevent a delay while the next disk is spun up.    But that issue is easily avoided by using the correct split level for your shares.    And this issue has nothing to do with the controller -- it's simply due to the need to spin up another disk.

 

 

 

Share this post


Link to post

Don't use it like it is now, because I need to assign the specific drives to the group.

When using another method than fill-up it easily happens that the data is spread on more disks.

Then it is necessary to permanently adjust the drives in that group.

 

If the "collection" is big enough, even fill-up won't save you.

With cache-dirs working everything is fine but if cache-dirs can not be run (e.g. too many files)

then share-based spinup would be the thing I want to use.

 

Share this post


Link to post

You could, of course, if you know you have media that spans multiple disks, use spin-up groups to do as you noted -- prevent a delay while the next disk is spun up.    But that issue is easily avoided by using the correct split level for your shares.    And this issue has nothing to do with the controller -- it's simply due to the need to spin up another disk.

That works if unRaid is used purely as a NAS attached to a single computer.

 

But, in this day and age a significant number of people have multiple computers streaming from unRaid.  And more to the point with LT's emphasis on dockers and VMs, you are going to be running other applications on the system which may or may not cause spin ups at any particular point in time.  Things like Plex, kodi, Openelec as they scan the media.

 

To go ahead and tell everyone that it's your hardware, or you've arranged your media incorrectly on the system is simplistic at best.

 

As has been previously been mentioned we have not as of yet been informed as to why the feature is being considered to be removed.  Until that happens this debate is rather moot.  The net result is that while the majority of people responding do not use spin up groups, they do under certain circumstances solve some problems.

 

Perhaps instead of debating allocation methods and split levels we should debate tossing all HDDs from the array and replacing them with SSDs which should be immune to the problem.

Share this post


Link to post

That sounds fixable in unRAID to me. There should be no real reason why unRAID should stall like this (conceptually at least).

 

unRAID doesn't cause the stall but the bandage fix on the unRAID side is to use spinup groups. It's a Windows problem. Windows basically creates one connection to the server so when a second program tries to access a spun-down disk the connection hangs until it gets a response.

 

Right!!

 

The reason I commissioned the poll is this.  The code that implements spinup groups is somewhat complex.  This is because in order to implement, the test for whether "other" disks of group should be spun-up as a result of I/O to any particular disk is done right in the driver I/O path.  Because of this we cannot simply execute something like 'hdparm' to interrogate the drive because it would occur on every I/O making it run really sloooooooooowwwww.  Instead we must maintain spinning state in the driver, meaning a simple integer flag that is tested.  But in order to do this, the driver has to maintain this flag.  Anyway, longtime unRaid users will recall that spinning status has been "problematic" over the years, and it's easy to break - latest occuring in 6.1-rc2.

 

I'm thinking the time to eliminate all this code is going to be with the introduction of P+Q support, and all my code documentation that refers to spinup groups talks about reason being to support master/slave on IDE channel.  (What Gary talks about here: http://lime-technology.com/forum/index.php?topic=41877.msg398674#msg398674).

 

But in back of my mind I seemed to recall there was another case that was solved by spinup groups and that is the Windows single SMB connection case.  So looks like I need to keep the code in place.

Share this post


Link to post
But in back of my mind I seemed to recall there was another case that was solved by spinup groups and that is the Windows single SMB connection case.  So looks like I need to keep the code in place.

Yes and no.  Looks like for most people it isn't needed and while it helps me out I can live without it.  Once SageTV releases it's open source version I can change the code for me to eliminate it as much as possible.  Plus I will be able to switch to a linux version and can further eliminate what appears to be a windows problem.  In other words I'm OK if it disappears because it doesn't completely eliminate my problems anyway.

Share this post


Link to post

Agree with Bob => and like him I use spin-up groups, but can live without it.  Simple fact is an occasional pause in a movie stream is simply no big deal.

 

Share this post


Link to post

Given the complexity, how about moving forward without it for the extensive closed alpha of P+Q and even for the beta and regather feedback during that time? Perhaps a simplier solution can be discovered after a NFO (nuke from orbit) approach or perhaps the latest version of Windows with newer Samba improves the situation enough?

Share this post


Link to post

... or perhaps the latest version of Windows with newer Samba improves the situation enough?

 

As I noted several times, the issue is interaction within the controller -- NOT the Windows connection.  You get this behavior even if you're accessing the array from multiple clients (each with their own SMB connection) ... and independent of what the client is [Windows, Linux, Apple].

 

If it's too complex to retain, so be it ... but it DOES resolve an issue that can be a bit of a nuisance (but not, in the great scheme of things, a big deal).    One very simple way to bypass the issue entirely is to simply hit the "Spin Up" button whenever you're going to have multiple folks using the server ... and perhaps set the spin-down timer to a moderately long time (2-3 hours).

 

Share this post


Link to post

... or perhaps the latest version of Windows with newer Samba improves the situation enough?

 

As I noted several times, the issue is interaction within the controller -- NOT the Windows connection.  You get this behavior even if you're accessing the array from multiple clients (each with their own SMB connection) ... and independent of what the client is [Windows, Linux, Apple].

 

If it's too complex to retain, so be it ... but it DOES resolve an issue that can be a bit of a nuisance (but not, in the great scheme of things, a big deal).    One very simple way to bypass the issue entirely is to simply hit the "Spin Up" button whenever you're going to have multiple folks using the server ... and perhaps set the spin-down timer to a moderately long time (2-3 hours).

 

Actually it was both, which is why some of the spinup issues in the past were so baffling - any time there are two or more issues happening at the same time it becomes very difficult to figure out what's going on.  Specifically:

a) there was an issue with IDE style controllers, which were really 'channel' interfaces, that only allowed command/response to a single device at a time.  And this includes early SATA controllers which were bolted on to IDE/PATA controller designs.

b) single-connection nature of windows.  Not sure if this is still the case, especially now with SMB3 protocol.

Share this post


Link to post

I think even later SATA controllers have this issue -- not sure what the exact relationship is between the various channels, but they can definitely still exhibit this behavior.

 

I can see both the SMB issue and the controller-related issue by starting a large copy from one drive; then starting another copy on the same PC from another drive that's not spun up ... the first copy will "hang" for a few seconds until the 2nd one starts.  [sMB issue]      If the 2nd copy is done from another PC, then whether or not the first one hangs depends on WHICH disk I use as the source for the first copy.

 

I haven't built an exhaustive "interaction spreadsheet" to know which drives have these interactions on my two newer servers (2.5 yr and 1 yr old) ... but did build one for my older CS2EE based server, and that's where I use the spin-up groups.

 

Share this post


Link to post

I have the issue of a short delay in watching media while a disk is spun up on my array, and it is annoying. But I see the spin up groups as a solution worse than the problem. Not interested in ALWAYS spinning up a bunch of disks as a preventative measure for a rare 3-4 second pause watching a movie.

 

Not advocating for or against the feature being removed, just explaining why I don't use it.

Share this post


Link to post

How about replacing it with a new feature related to shares rather than hardware ports? IOW, if you check this setting on a share, then as soon as a file in the share is accessed, all drives that participate in that share are immediately spun up as well? That setting combined with spin down timers could cover almost any scenario I can think of.

 

I'm about to vote NO, because I don't use them, but I like this idea because it solves one of my very recent conundrums. I've found myself doing what others have said, forcing shares to specific a specific drive.

 

The problem is, I hate that. It makes me feel limited in a way that I specifically choose unRaid to avoid. CacheDir solves some of the problems spin-up groups solves, but not all, so I was forced to put some of my shares (like acronis backup which would time out) onto the same drive. But those backups are my biggest growth consumers of data and I'd like to avoid being stuck on a single drive.

 

So, while I am voting NO right now, I have actually considered implementing spin-up groups just this very weekend.

Share this post


Link to post

I use them.  Without them, you get pauses when starting media that is on a spun down drive and sometimes a pause or stutter on playing media if another user causes a drive to spin up.

 

The users don't care that the hard drive controller isn't configured optimally or that Windoze sucks, all they know is that they are sitting there watching a spinning icon when the media should be playing.

 

An option to spin up all drives together would be fine, but totally eliminating the feature would be a major degradation of the user experience IMO.

Share this post


Link to post

As others have said, I voted no, because I didn't realise it would solve the issue I have with Plex pausing at change of media when the new media is on a spun down drive.  The spinup groups would solve this issue and improve WAF.

 

Share this post


Link to post

I missed the voting because I don't come to the forum too often since unRAID never gives me problem.  However, I have used Spin Up Groups since my very first unRAID server.  Spin up groups elminate lag and timeouts from application when the pause for disks to spin up takes too long.  If it wasn't for Spin Up Groups, I would have never gone beyond the trial license since the delay was a deal breaker and having the disks constantly spinning was also a deal breaker.

Share this post


Link to post

sorry for being late (maybe), but then, i'm not an unRAID user yet, but have ordered all necessary parts to built that new machine for unRAID and other purposes.

 

DON'T KILL A FEATURE because of an hardware usage change, but rather CHANGE THIS FEATURE into something useful for todays usage patterns.

 

as one user stated: change this feature to be useable on shares (default: disabled). this opens up a solution everybody can use, but has not to. probably most do use shares to access the data on drives, i'll definitely do so. and while it's beneficial to me as feature in this case, i don't try to figure out every-bodies else's usage scenario. and then, who get's hurt? it's optional.

 

just my 2 cts here.

Share this post


Link to post

Hi,

 

When I add a disk to the array, I set the spin up group appropriately.

 

I like to think that what it does is make sure that when a file is requested on that disk, _ONLY_ that disk is spun up.

 

Probably, with the current codebase, it would behave like this without the need of specifying a spin up group.

 

If, in my use case, not specifying a spin group gets the same result, then remove the feature.

 

On the other hand, I do share some of s.Oliver's feelings.

 

If you have already written the code .... :)

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.