Buffering Issues during data rebuild

[JimPhreak]

I'm in the middle of a data rebuild (33% after 8 hours) and I'm having buffering issues trying to stream a single media file in the process.  I realize that it's not ideal to do anything on the system while doing a parity check/data rebuild but given that it takes 24 hours for me to complete these tasks it's not exactly realistic.

 

I found this thread and am wondering if it still applies or if there is anything that has been done (or that I could do manually) to limit the effect of the data rebuild on my ability to stream simultaneously?

[kizer]

I can stream normally 720 content while rebuilding via XBMC on a Wired connection on the Simulated drive.

 

What exactly are you attempting to stream?

[JimPhreak]

I can stream normally 720 content while rebuilding via XBMC on a Wired connection on the Simulated drive.

 

What exactly are you attempting to stream?

 

Multiple 720 streams (remote users) some of which are on the simulated drive and some are not.  I'm assuming streams of videos from the simulate drive would have the most performance drop?

[Squid]

I can stream normally 720 content while rebuilding via XBMC on a Wired connection on the Simulated drive.

 

What exactly are you attempting to stream?

 

Multiple 720 streams (remote users) some of which are on the simulated drive and some are not.  I'm assuming streams of videos from the simulate drive would have the most performance drop?

Of course...  To read to simulated files, all the drives have to be read.  Added on that other drives are streaming other content.  Added on that you're trying to rebuild the drive at the same time.  Buffering is to be expected when you're running the drives like that

 

[kizer]

Multiple streams I'd probably say your pushing the system a bit.

[JimPhreak]

I figured as much I just wanted to confirm there were no tweaks I could make.  I decided to bring Plex up on my backup server in the meantime.  It's only a C2558 but it manages to handle two simultaneous 720p trancodes without issue.

[garycase]

I would simply not do the stream => especially since you have a backup server ... just do any streaming you want to do during the rebuild from the backup server 

[garycase]

... Just read the rest of the thread and noticed that you had already one exactly what I just suggested 

 

FWIW, for those who do not have backups, I'd consider your viewing/streaming needs BEFORE doing a rebuild ... and either start the rebuild AFTER you're done for the day; or just copy the media you want to watch to another PC before starting a rebuild.

 

It's also possible to tweak the disk tunables to improve the streaming performance during a parity check or disk rebuild.    If you set md_num_stripes well above the md_sync_window value the system should be able to buffer enough ahead that the impact will be significantly reduced ... hopefully to the point where a single stream will work just fine.    Note that the num_stripes value defines how many 4K I/O requests can be queued at any given time, while the sync_window defines the most number of stripes that will be used for a parity check or sync operation [i THINK this also applies to rebuilds ... these aren't mentioned in the details I found on these, but hopefully this is the case ... otherwise changing the parameter won't help]    So reducing the md_sync_window value will slightly degrade parity check/disk rebuild performance, but will improve the responsiveness of the array during these operations providing the md_num_stripes is large enough.

 

 

[SSD]

... or sync operation [i THINK this also applies to rebuilds ... these aren't mentioned in the details I found on these, but hopefully this is the case ...

 

The parity sync and rebuild are handled the same, just with a different target disk. All it is doing is reading all disks in the array but one, and creating parity on the other. In the disk rebuild, the "parity" it creates turns out to be the desired disk contents.

[danioj]

If you have a complete backup why not just amend your DNS to point your clients to the Backup Server while you sort out the issues with the Main Server. Easy. I've done it multiple times when my Main Server has gone down. Issues are then transparent to the users and content is accessable as normal.

 

I do make everything Read Only while the recovery operations are underway but that just means clients have to suspend writes while I recover the situation but content is (as I mention) still accessable.

 

I think when you have a rebuild / issue its better to recover as quickly as possible. Using the Array while doing this will just increase the time it takes to recover AFAIK. Plus reducing writes to the compromised Array and restricting access to that Server while things are being recovered I see as only a good thing. Also but restricting (or redirecting access as I do) gives you the ability to reset and tinker with the impacted server as required without impacting users access to content (just saves).

 

JM2C.

[JimPhreak]

If you have a complete backup why not just amend your DNS to point your clients to the Backup Server while you sort out the issues with the Main Server. Easy. I've done it multiple times when my Main Server has gone down. Issues are then transparent to the users and content is accessable as normal

 

I do make everything Read Only while the recovery operations are underway but that's just means clients have to suspend writes while I recover the situation but content is (as I mention) still accessable.

 

I think when you have a rebuild / issue its better to recover as quickly as possible. Using the Array while doing this will just increase the time it takes to recover AFAIK. Plus reducing writes to the compromised Array and restricting access to that Server while things are being recovered I see as only a good thing. Also but restricting (or redirecting access as I do) gives you the ability to reset and tinker with the impacted server as required without impacting users access to content (just saves).

 

JM2C.

 

I don't believe this would work with Plex and remote access unless I mirrored my exact Plex config to my backup server which I've read is not recommended because if by some chance both servers are online at the same time the database can't become screwed up.