Jump to content
jonp

unRAID Project Roadmap Announcements

414 posts in this topic Last Reply

Recommended Posts

The issue occurs when you are already streaming media and concurrently access a spun down drive.  When you do, the stream will stop for up to 4-5 seconds.  I've posted this video on here before and Tom has even seen it, but here it is again.  This video shows me playing some music off my server through J. River Media Center while trying to scrape information for new movies (also on the server) using Ember Media Manager.  To do so it scans each disk for new movies.  When it does, the stream stops. 

 

https://www.youtube.com/watch?v=9Lj-ISONmBo

 

This also happens whenever SickBeard or Couch Potato copy files to my server.  Also if I simply browse the directory via Windows Explorer or copy files to a spun down drive.  If all the drives are spun up there is no problem.

 

This is a windows issue, not an unRAID issue.  Basically, windows only opens a single "channel" to the unRaid server, regardless of how many connections are required.  If one of those connections has to wait for a drive to spin up, then all connections are paused because they are all running through the same channel.

 

The way that I solved it on my system (probably similar to yours - the HTPC is also running Sickbeard / Couch Potato / SabNZBD), was to create a VM using VirtualBox (dirt easy), and install Windows on it and then SB/CP).  Then there is no pauses, since the VM Windows is using its own channel, and not sharing the host's (even though they are going through the same ethernet port)

Share this post


Link to post
Just for clarification, how/when does this happen? I've had a delay when starting a video on a spun down disk, but that is expected. Once the drive is up I've never seen any issues. If you are playing a video how are you also accessing a spun down disk?

 

I suspect that there is a little lack of clarity here.

 

Or is this a second client connecting to a spun down disk, which is causing you issues on video playback you've already started?

 

I think that is the correct understanding - that what is being reported is this scenario: Data is being read from drive A in a steady stream for one application.  If another application then requires to read data from drive B, which is currently spun down, drive B will spin up.  During the time that drive B is spinniing up, until it becomes ready, the data streaming from drive A pauses.

Share this post


Link to post

The way that I solved it on my system (probably similar to yours - the HTPC is also running Sickbeard / Couch Potato / SabNZBD), was to create a VM using VirtualBox (dirt easy), and install Windows on it and then SB/CP).  Then there is no pauses, since the VM Windows is using its own channel, and not sharing the host's (even though they are going through the same ethernet port)

 

Any chance that having SAB & SB in their own dockers will alleviate this issue also?  If not, what about putting SAB & SB into the windows VM?  I also suffer from this issue from time to time, and agree that it's pretty irritating.

Share this post


Link to post

The issue occurs when you are already streaming media and concurrently access a spun down drive.  When you do, the stream will stop for up to 4-5 seconds.  I've posted this video on here before and Tom has even seen it, but here it is again.  This video shows me playing some music off my server through J. River Media Center while trying to scrape information for new movies (also on the server) using Ember Media Manager.  To do so it scans each disk for new movies.  When it does, the stream stops. 

 

This also happens whenever SickBeard or Couch Potato copy files to my server.  Also if I simply browse the directory via Windows Explorer or copy files to a spun down drive.  If all the drives are spun up there is no problem.

RockDawg,

 

Thank you for taking the time to record and post a video of this.  I just watched it and agree, that is annoying.  I will add this to our task list to investigate further.  I haven't experienced this problem myself but to be fair, I don't think I've tested this specific scenario in depth myself before.  Can you send me a PM with a link to the thread you had opened on this issue before?

Share this post


Link to post

... four core pillars ...

 

uh oh! Sounds like someone sat through a PHB meeting ;-)

Lol!  Not sure what you meant by PHB but Google didn't help me much with such a vague input unless the urban dictionary was the one you meant ;-)

Share this post


Link to post

What kind of hardware limitation?  I've run across this in 5 completely different systems (ranging from a Pentium 4 based Celeron with 1GB RAM to my current Xeon E3-1230 w/32GB RAM) and across many different versions of unRAID,  I can appreciate that you and others my not have experienced this issue, but others of us have and it sucks.

 

It's just the way that Linux interacts with ALL PC hardware and hard drives that can spin-down and take a while to spin-up. As I said before, if you don't want to be bothered by this issue, do not spin down your drives.

 

Tom himself said this should not be happening:

 

Should not glitch during spinups unless you are reading media file from IDE drive and then try to spin up the other IDE drive on the same channel. Or, similar scenario, if using SATA drives attached to fairly old SATA controllers.

 

I no longer spin down my drives, but that is hardly a solution.

Share this post


Link to post

The issue occurs when you are already streaming media and concurrently access a spun down drive.  When you do, the stream will stop for up to 4-5 seconds.  I've posted this video on here before and Tom has even seen it, but here it is again.  This video shows me playing some music off my server through J. River Media Center while trying to scrape information for new movies (also on the server) using Ember Media Manager.  To do so it scans each disk for new movies.  When it does, the stream stops. 

 

https://www.youtube.com/watch?v=9Lj-ISONmBo

 

This also happens whenever SickBeard or Couch Potato copy files to my server.  Also if I simply browse the directory via Windows Explorer or copy files to a spun down drive.  If all the drives are spun up there is no problem.

 

This is a windows issue, not an unRAID issue.  Basically, windows only opens a single "channel" to the unRaid server, regardless of how many connections are required.  If one of those connections has to wait for a drive to spin up, then all connections are paused because they are all running through the same channel.

 

The way that I solved it on my system (probably similar to yours - the HTPC is also running Sickbeard / Couch Potato / SabNZBD), was to create a VM using VirtualBox (dirt easy), and install Windows on it and then SB/CP).  Then there is no pauses, since the VM Windows is using its own channel, and not sharing the host's (even though they are going through the same ethernet port)

 

The scenario in my video is only one of the ways the problem rears it's head.  I run ESXi and have a Windows VM that runs Sickbeard and SabNZBD.  Whenever it begins moving the downloaded episode to a spun down drive on my unRAID server it will pause the video streaming to XBMC (Linux).  I've had it happen without Windows involved at all while streaming video with XBMC and browsing a spun down drive in Ubuntu.

Share this post


Link to post

Or is this a second client connecting to a spun down disk, which is causing you issues on video playback you've already started?

 

I think that is the correct understanding - that what is being reported is this scenario: Data is being read from drive A in a steady stream for one application.  If another application then requires to read data from drive B, which is currently spun down, drive B will spin up.  During the time that drive B is spinniing up, until it becomes ready, the data streaming from drive A pauses.

 

You are correct PeterB.  That is the scenario.

Share this post


Link to post

The scenario in my video is only one of the ways the problem rears it's head.  I run ESXi and have a Windows VM that runs Sickbeard and SabNZBD.  Whenever it begins moving the downloaded episode to a spun down drive on my unRAID server it will pause the video streaming to XBMC (Linux).  I've had it happen without Windows involved at all while streaming video with XBMC and browsing a spun down drive in Ubuntu.

 

Are you not running a cache drive? I would assume this would resolve any issues with SAB/SB. I realize there are other scenarios where this might present itself, but I would think this would address these instances at least.

 

Share this post


Link to post

What kind of hardware limitation?  I've run across this in 5 completely different systems (ranging from a Pentium 4 based Celeron with 1GB RAM to my current Xeon E3-1230 w/32GB RAM) and across many different versions of unRAID,  I can appreciate that you and others my not have experienced this issue, but others of us have and it sucks.

 

It's just the way that Linux interacts with ALL PC hardware and hard drives that can spin-down and take a while to spin-up. As I said before, if you don't want to be bothered by this issue, do not spin down your drives.

 

Tom himself said this should not be happening:

 

Should not glitch during spinups unless you are reading media file from IDE drive and then try to spin up the other IDE drive on the same channel. Or, similar scenario, if using SATA drives attached to fairly old SATA controllers.

 

I no longer spin down my drives, but that is hardly a solution.

 

Yup, it sucks and it's not a solution, just a hack-around.

 

Unfortunately what Tom indicates doesn't seem to be the case from my recent experience even on older and newer hardware and using either motherboard SATA or add-in SATA cards with two or three different client systems.  :(

Share this post


Link to post

Probably best to fork this discussion to keep the roadmap discussion as lean as possible

Share this post


Link to post

We were promised the latest AFP to come to v6, its been stalled over an over. Would be appreciated to know when this will be done already, especially since the goal stated is for v6 to hit final by Q3. This stall in AFP is a large pain point for those of us who use and rely on AFP with unRAID. It was stated it wasn't addressed in v5 because the time was going to be taken to get it going in v6. Six beta's not a single AFP update... when there is an up roar, promises are made and request we be patient, so then its swept under the rugs, again we have to complain many months later.

 

 

The stall issue reported affect myself as well. This is not because what is being accessed all comes from one device on the network. If say device #1 HTPC is playing a movie, and another device, #2 is playing music, and device #3 wants to access say photos, but the disk containing the photos is spun down, access by this third device causes the other two devices content to pause while the disk is spun up. Almost want to say even though (in my case) I have multiple controllers, all access is halted across all controllers while a disk is spinning up.

 

 

Please keep in mind that apcupsd supports network management cards (not just usb, direct connections), myself and a few others use apcupsd in this manner, with custom configuration script as none of the plugin authors addressed this ability. IF you require a sample let me know. (Its not a change to apcupsd, but additional parameter/settings to enable communications with network ups's)

Share this post


Link to post

So still no duel parity support or something similar?  :'(

 

Sorry if I missed it

 

I hope this will be part of the next round of features in 6.x or 7.x

Share this post


Link to post

So still no duel parity support or something similar?  :'(

 

Sorry if I missed it

 

I hope this will be part of the next round of features in 6.x or 7.x

 

Personally, I am glad this is not in the current release. I would much rather they get a stable 6.0 RTM release out the door with reasonable goals, and then move onto new features for 6.1 or whatever it will be. Far better to tackle all these options in bite size chunks vs. trying to re-boil the ocean and taking another 4 years to get 6.0 RTM out.

 

At least if they get 6.0  done with ups, email notifications, etc. then they've addressed a number of long standing concerns. Plus 6.0 becomes the current release, so they don't need to deal with all the 5.0 people who were concerned with moving to 6.0. We get everyone (just about anyways) on a common, stable platform and they can focus exclusively on 6.1 feature sets (of which dual parity would be nice to see).

 

Share this post


Link to post

 

 

Please keep in mind that apcupsd supports network management cards (not just usb, direct connections), myself and a few others use apcupsd in this manner, with custom configuration script as none of the plugin authors addressed this ability. IF you require a sample let me know. (Its not a change to apcupsd, but additional parameter/settings to enable communications with network ups's)

The version of apcupsd I'm using, which is a compilation of revisions from the version 6 plugin thread has a network option. Not sure if it's what you need.  It's in my github.

 

 

Share this post


Link to post

 

So still no duel parity support or something similar?  :'(

 

Sorry if I missed it

 

I hope this will be part of the next round of features in 6.x or 7.x

 

Personally, I am glad this is not in the current release. I would much rather they get a stable 6.0 RTM release out the door with reasonable goals, and then move onto new features for 6.1 or whatever it will be. Far better to tackle all these options in bite size chunks vs. trying to re-boil the ocean and taking another 4 years to get 6.0 RTM out.

 

At least if they get 6.0  done with ups, email notifications, etc. then they've addressed a number of long standing concerns. Plus 6.0 becomes the current release, so they don't need to deal with all the 5.0 people who were concerned with moving to 6.0. We get everyone (just about anyways) on a common, stable platform and they can focus exclusively on 6.1 feature sets (of which dual parity would be nice to see).

 

I agree.

Share this post


Link to post

Thanks for this Jon, although saying Q3 does make you a hostage to fortune and Murphy !

 

I can understand the decision to focus on Docker for compartmentalising apps - it's only one thing to get working, rather than two. Hopefully that Docker solution is going to be totally user friendly (on installation, setup, spinning down, crash resolution etc.) ?

 

I do personally want/need a working real virtualisation at some point though - Team Viewer into a Windows session etc. Also useful for apps with their own linux distros. Don't back off too far.

 

Hopefully the email reporting of problems and UPS control will keep some people happy (or at least less annoying).

 

Three points/thoughts from a Roadmap perspective:

[*]It would be sensible to think about smartphone/tablet control and monitoring of the NAS. That probably in turn means consideration of how the web GUI is designed. Done with an eye to a smartphone-sized screen it gives you a route to creating a Cordova based app. It's a small step then to notifications and widgets for system events/monitoring that's immediate. At present the webGUI is fairly desktop focused.

[*]On a related tack, it's probably worth giving some attention to app solutions like 'cloud' for providing access to smartphones/tablets on the move. If HTPC is one use case, sensible cloud storage and serving for people's on-the-go devices is another major area for people. A no-config, idiot proof solution would help - so closer integration.

[*]As we know, RAID shouldn't be considered backup. At present, to actually backup the data, I connect a USB drive, have the NTFS drivers loaded, mount the drive on the CLI and conduct the actual backup copying via Midnight Commander (so the speed isn't totally slow). A better, more user friendly route to backup, at speed, would be at least as important as email notifications or extra parity drives.

Oh, and still need a ReiserFS > BTRFS converter  ;)

Share this post


Link to post
In addition, all existing paid customers of unRAID 5 or earlier will continue to see this benefit as they upgrade for free to unRAID 6!

 

So Unraid 7 will be a paid upgrade? Maybe I'm reading too much into that.but I'm cool with that, God knows that tom has been great with upgrades thus far. With the additional staff and increased development it was only a matter of time anyways.

Share this post


Link to post

Thanks for this Jon, although saying Q3 does make you a hostage to fortune and Murphy !

 

I can understand the decision to focus on Docker for compartmentalising apps - it's only one thing to get working, rather than two. Hopefully that Docker solution is going to be totally user friendly (on installation, setup, spinning down, crash resolution etc.) ?

 

I do personally want/need a working real virtualisation at some point though - Team Viewer into a Windows session etc. Also useful for apps with their own linux distros. Don't back off too far.

 

Hopefully the email reporting of problems and UPS control will keep some people happy (or at least less annoying).

 

Three points/thoughts from a Roadmap perspective:

[*]It would be sensible to think about smartphone/tablet control and monitoring of the NAS. That probably in turn means consideration of how the web GUI is designed. Done with an eye to a smartphone-sized screen it gives you a route to creating a Cordova based app. It's a small step then to notifications and widgets for system events/monitoring that's immediate. At present the webGUI is fairly desktop focused.

[*]On a related tack, it's probably worth giving some attention to app solutions like 'cloud' for providing access to smartphones/tablets on the move. If HTPC is one use case, sensible cloud storage and serving for people's on-the-go devices is another major area for people. A no-config, idiot proof solution would help - so closer integration.

[*]As we know, RAID shouldn't be considered backup. At present, to actually backup the data, I connect a USB drive, have the NTFS drivers loaded, mount the drive on the CLI and conduct the actual backup copying via Midnight Commander (so the speed isn't totally slow). A better, more user friendly route to backup, at speed, would be at least as important as email notifications or extra parity drives.

Oh, and still need a ReiserFS > BTRFS converter  ;)

 

I did mention in the post that machine-based virtualization is still VERY important to us.  This should be evident by the amount of effort that went into upgrading it for this release.  We wouldn't throw away that much work so lightly ;-).  Stay tuned for more information on where we are headed with that...

Share this post


Link to post

In addition, all existing paid customers of unRAID 5 or earlier will continue to see this benefit as they upgrade for free to unRAID 6!

 

So Unraid 7 will be a paid upgrade? Maybe I'm reading too much into that.but I'm cool with that, God knows that tom has been great with upgrades thus far. With the additional staff and increased development it was only a matter of time anyways.

 

?  Never said that.

Share this post


Link to post

In addition, all existing paid customers of unRAID 5 or earlier will continue to see this benefit as they upgrade for free to unRAID 6!

 

So Unraid 7 will be a paid upgrade? Maybe I'm reading too much into that.but I'm cool with that, God knows that tom has been great with upgrades thus far. With the additional staff and increased development it was only a matter of time anyways.

 

?  Never said that.

 

It should be. How else do you get paid.

Share this post


Link to post

Double edged sword.

 

I certainly wouldn't be out out for paying $20 for an upgrade for every license when it's a major version change.

 

But

 

LT would want to make sure the new release is production ready with basics like documentation etc available from Day 1.

 

As it is, the upgrades are free so we can't whine too much....

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.