HD streams have artifacts using unRAID

[guitarlp]

During playing of a ripped blu-ray movie (complete movie in ISO format... about 20 Gigs) I see artifacts and pixelation at certain points in the movie. If I stop and re-start the artifacts/pixelation occurs at the exact same part of the movie again. My network can read at 80+ MB/sec from the unRAID server and I've gone over my entire network/cable setup... it's performing optimally.

 

Update:

 

It appears to be hardware related and not related to the unRAID OS at all. I still haven't narrowed it down exactly but something is my setup is corrupting my HD files and it is most likely hardware related.

[WeeboTech]

What is your hardware?

 

IE. Motherboard, nic, switch media player

Have you tried a new cable temporarily?

 

[guitarlp]

My current setup is:

 

Gigabyte GA-EP35-DS3R Motherboard

Intel E6600 CPU

640W Corsair PSU (rated 80+ or whatever)

2 x 1gig Gskill 5-5-5-15 RAM

Intel Gigabit NIC

All drives are almost all new SATA

 

My last build (for the past week) was all of the above except with an AM2 5000+ AMD CPU with a GA-MA69GM-S2H MOBO.

 

Other users in the NFS thread mentioned issues with HD watching as well. I think it's not a big issue because most people aren't streaming full size HD movies across their network to their media center PC. Most people just do data, DVD's and Music. But so far HD has not been possible on my unRAID server. Works fine though on my windows home server which is not as robust as my unRAID server is.

[limetech]

From http://hometheater.about.com/od/dvdbasics/a/bluhddvdinfo.htm (referring to Blu-ray):

 

Data Transfer Rate: 36 to 48 MBPS (Megabits per Second) average - capable of up to 54 MPS - This exceeds the 19.3 Mbps transfer rate approved for HDTV broadcasts.

 

Note these values are Megabits per Second.  The data rate of current blu-ray is only 6 MB/sec.

 

So it's not a bandwidth issue.  There are a couple things for you to try:

 

1. If reading from a User Share, try reading directly from the disk share instead.

2. Try write/read to a different disk.

 

What player are you using?

[guitarlp]

From http://hometheater.about.com/od/dvdbasics/a/bluhddvdinfo.htm (referring to Blu-ray):

 

Data Transfer Rate: 36 to 48 MBPS (Megabits per Second) average - capable of up to 54 MPS - This exceeds the 19.3 Mbps transfer rate approved for HDTV broadcasts.

 

Note these values are Megabits per Second.  The data rate of current blu-ray is only 6 MB/sec.

 

So it's not a bandwidth issue.  There are a couple things for you to try:

 

1. If reading from a User Share, try reading directly from the disk share instead.

2. Try write/read to a different disk.

 

What player are you using?

 

I'll check as soon as my parity disk is done rebuilding. Found out today that my parity drive shrank by 1064 bytes or something... so when I added a brand new drive (exactly the same as my parity drive) it said the parity drive was not the smallest. So I had to take out the current drive and am rebuilding with the new drive (I will then add the old parity drive as a data drive when this is done). A bit of a pain... but it looks like it's the only work around

 

I have tried writing and reading from different disks (different brands in fact). I will try reading from the disk share instead and see if that fixes anything (although I'm pretty sure I tried this in the past couple of days as well).

 

I'm using Total Media Player on my Windows Vista Home Premium. The only other thing installed is AnyDVD HD (to get past BD+) and Daemon Tools (to mount my ISO image as a virtual CD rom drive). It's on a fresh install of Vista.

[WeeboTech]

I thought I remembered reading that vista needs to be slightly tuned or modified to play well with SAMBA (or is it the other way around)

Now that I realize it's a VISTA PC, I'm more inline with thinking it's a tuning issue rather then an NFS will fix it issue.

[guitarlp]

Do you have any links to anything regarding how to "tune" my Vista PC to work better with Samba?

[jimwhite]

Have you started by installing Service Pack 1 ? 

 

 

[limetech]

guitarlp- what unRAID release are you using?

[guitarlp]

Have you started by installing Service Pack 1 ? 

 

 

 

Of course. I actually install the system and before I even install anything such as drivers I always make sure to download all 60+ updates (including SP1) from MS

[guitarlp]

guitarlp- what unRAID release are you using?

 

4.3b6

 

I was using the 4.2 before and had the exact same issues. It made no difference in my HD streams with either version.

[NLS]

try reading directly from disk share as Tom said

 

also please note to check if the artifacts happen exactly the same time every time or if they have a steady pattern (like every two seconds)

 

install a nice bandwidth monitor (like DU Meter) - your bandwidth may be ok but you might get non continous flow (this has to do with the buffering the player does too)

 

try different players and note if anything changes (and if it does, how)

 

 

 

 

[guitarlp]

try reading directly from disk share as Tom said

 

also please note to check if the artifacts happen exactly the same time every time or if they have a steady pattern (like every two seconds)

 

install a nice bandwidth monitor (like DU Meter) - your bandwidth may be ok but you might get non continous flow (this has to do with the buffering the player does too)

 

try different players and note if anything changes (and if it does, how)

 

 

 

 

 

Ok... I read directly from the disk and it made no difference.

 

The artifacts happen at certain points in the movie. It's not at set times like every 10 seconds. But once they happen, you can pause, go back, and play it again and it will have the same pixelation or what not in the exact same place it did before.

 

I installed DU Meter and left it running. It appears to have continuous flow. The speed always showed between 20-30 Mbts while playing a movie. I compared both a movie played from the unRAID server and one played from my windows home server. Both graphs looked the same and I saw no differences.

 

I downloaded PowerDVD 8 which is the other big BD player on the market but it wouldn't work with my ISO. So I'm going to find and download PowerDVD 7.3 which should work.

 

Hopefully this information is useful though. After testing with PowerDVD we should know if it's player/unraid related. If it isn't, then it's either unRAID itself or the way vista communicates with unRAID.

[jasper9890]

you can play in powerdvd by mount the iso with a tool like daemontools

[NLS]

now if the artifacts come up at the same place all the time, first thought is that the file is trashed at that point

 

but if you say you copy that same (supposedly messed up) file back to your computer and replay it and the artifacts are gone... complicates things

 

then again, it might have to do with caching

try this: after seeing artifacts at a specific point, let the movie play for a couple of minutes, skip waaaay far from the artifact point (back or forth - I prefer forth), then play a few min. and then go back to the point that you saw the artifacts, to see if they are still at the same place (this way you'll refill the player's cache some)

 

 

[guitarlp]

now if the artifacts come up at the same place all the time, first thought is that the file is trashed at that point

 

but if you say you copy that same (supposedly messed up) file back to your computer and replay it and the artifacts are gone... complicates things

 

then again, it might have to do with caching

try this: after seeing artifacts at a specific point, let the movie play for a couple of minutes, skip waaaay far from the artifact point (back or forth - I prefer forth), then play a few min. and then go back to the point that you saw the artifacts, to see if they are still at the same place (this way you'll refill the player's cache some)

 

 

 

I thought the file was trashed too. That's why I've used different BD burners on different computers running different ripping software. I've tried all different sorts of combinations to rip these ISO's and the glitches happen at the same points no matter how the BD is ripped.

 

I've tried letting it play for about 30 minutes to see if anything changed... it still had glitches further into the movie. I then went back to the begining to see if it still glitched and yep... it did.

[NLS]

So come again: If you copy back THAT SAME ISO that had the glitches, FROM unRAID TO your local disk... do they happen again?

 

If they do, the files are indeed trashed (so if you tried different ripping mechanisms, probably they just don't work as advertised).

 

If they don't, something in the chain doesn't like specific bit combinations in the data stream (!)...

 

If the ISO works (locally), when goes to unRAID doesn't and if you get it back locally is still doesn't, then something messes the file during copy (prob. hardware and more specifically RAM).

 

 

[limetech]

guitarlp-

I'm very interested in figuring this out.  What player are you using exactly?  "TotalMedia Theater" from Acrsoft, or "Total Video Player" from EffectMatrix, or something else (can't find "Total Media Player").

 

It seems like you've done a good job in isolating h/w, so my theory is this: that's it's a latency issue.  The HD data stream is actually a variable bit-rate, and perhaps what's happening is that particular point in the movie demands a very high bit rate, so when the player issues a read request to proceed beyond that point, the round-trip latency is too high & the player gets momentarily starved of data resulting in artifacts.

 

In addition, Vista implements a new version of the SMB protocol called SMB2.  Among other things, this protocol permits larger block size requests.  Perhaps WHS is using SMB2 as well & this explains why everything works when streaming from WHS.  Samba 4 has "experimental" support for SMB2 & I can look into creating a special unRAID release for you to test.

 

But this is all conjecture.  Would be interesting to see if you get the same result using a different player.

[guitarlp]

So come again: If you copy back THAT SAME ISO that had the glitches, FROM unRAID TO your local disk... do they happen again?

 

If they do, the files are indeed trashed (so if you tried different ripping mechanisms, probably they just don't work as advertised).

 

If they don't, something in the chain doesn't like specific bit combinations in the data stream (!)...

 

If the ISO works (locally), when goes to unRAID doesn't and if you get it back locally is still doesn't, then something messes the file during copy (prob. hardware and more specifically RAM).

 

 

 

I think you may have just solved it... bad memory.

 

I tried copying the bad movie from my unRAID server to one of the PC's that plays it fine. It played with glitches in the same spot.

 

I replaced the memory in the unRAID server and copied one of my good playing blu-ray rips to it. Once it was finished it played fine and didn't have glitches at all. I'm copying some more movies I've had trouble with to verify... but I think I may have just had a bad stick of ram.

 

I replaced everything in the unRAID server... except the memory. I figured if the new sticks were bad they wouldn't post... but I should have just ran memtest anyways to make sure they were fine. I also didn't think that memory could leave to corruption on a hard drive.

 

If the memory was in fact the problem it looks like I'll need to wipe all the disks clean and start from scratch. If I go this route, is thee some command I can use to clean all the disks (format them)?

[WeeboTech]

I think you may have just solved it... bad memory.

 

I tried copying the bad movie from my unRAID server to one of the PC's that plays it fine. It played with glitches in the same spot.

 

DOH!!!! I did not think it was memory because of the followiing quotes from a priorr thread

 

I've gone over everything... it's not my network... it's not my hardware. The only thing left hardware wise I thought it could have been was the MOBO... but I replaced that today and it made no difference with my HD movies.

 

I can copy the movie on my unRAID server to my Windows Home Server and it plays over my network with no issues. I can copy that same movie to one of my PC's and stream across my network... again... no issues.

 

It would be interesting for you to put the old questionable ram back and do a memtest.

I think that should probably be the first recommended test when some of these things occur.

 

Probably some basic rules. too.

Pixelation = corruption.

Stutter - Network starvation.

 

It is possible that bad memory can corrupt a file or a filesystem.

Keep in mind that when files are written to a server, they are cached into a buffer, then when aged or full the buffer is flushed out to disk.  That's the reason they make ECC ram and servers usually require it.

 

 

 

[guitarlp]

I think you may have just solved it... bad memory.

 

I tried copying the bad movie from my unRAID server to one of the PC's that plays it fine. It played with glitches in the same spot.

 

DOH!!!! I did not think it was memory because of the followiing quotes from a priorr thread

 

I've gone over everything... it's not my network... it's not my hardware. The only thing left hardware wise I thought it could have been was the MOBO... but I replaced that today and it made no difference with my HD movies.

 

I can copy the movie on my unRAID server to my Windows Home Server and it plays over my network with no issues. I can copy that same movie to one of my PC's and stream across my network... again... no issues.

 

It would be interesting for you to put the old questionable ram back and do a memtest.

I think that should probably be the first recommended test when some of these things occur.

 

Probably some basic rules. too.

Pixelation = corruption.

Stutter - Network starvation.

 

It is possible that bad memory can corrupt a file or a filesystem.

Keep in mind that when files are written to a server, they are cached into a buffer, then when aged or full the buffer is flushed out to disk.  That's the reason they make ECC ram and servers usually require it.

 

 

 

 

My mistake. When I said I checked all my hardware I did... except the RAM (I didn't see how this could be the problem). I replaced MOBO's, CPU's SATA cards, NIC's, Router, Switches, Cables, ect...). I should have ran the memtest though.

 

I will try to stick the ram back in and running memtest to see if/when it fails.

 

Question though... should I re-format all the hard drives? The filesystem could be corrupt on them right? If I need to reformat them, how would I do that?

[jimwhite]

It's amazing, but many times we zealously defend our hardware and say "yes, I checked that and it's good" when really we only "assumed" it was good and really couldn't be bothered with really checking it ! 

 

Your assertions that a copy of the file from unraid to another computer left you with a good file really threw us off... bad ram should ALSO have affected this copy... oh well, glad you found thew problem.... 

 

 

[guitarlp]

It's amazing, but many times we zealously defend our hardware and say "yes, I checked that and it's good" when really we only "assumed" it was good and really couldn't be bothered with really checking it ! 

 

Your assertions that a copy of the file from unraid to another computer left you with a good file really threw us off... bad ram should ALSO have affected this copy... oh well, glad you found thew problem.... 

 

 

 

I know I know... I'm sorry 

[Joe L.]

You can run the reiserfsck program on each of your drives in turn. No need to re-format.

 

Joe L.

[guitarlp]

You can run the reiserfsck program on each of your drives in turn. No need to re-format.

 

Joe L.

 

Thank you for the reply Joe.

 

I ran:

 

reiserfsck --check /dev/sdx1

 

(replaced x with all my drive letters)

 

I was able to run it on all the drives except for 3 of them (I have 6 disks total).

 

3 of the checks returned saying no corruptions found (which I assume means the disk is completely fine).

 

My 3 other disks return the following:

 

reiserfs_open: the reiserfs superbloc cannot be found on /dvd/sdb1.

Failed to open the filesystem.

 

Does that mean the drive is corrupt? What should I do on those drives?

 

Edit: I tried running --rebuild-sb /dev/sda1

 

I answered some questions and let it do it's thing. Now, if I re-run that same command on /dev/sda1 it says the file system is not clean and some corruption exists. If I run --check on that disk it teks me:

 

Partition /dev/sda1 is mounted with write permissions, cannot check it

 

Any ideas?

 

I haven't played with linux since I was in college a long time ago... so please give me the dumbed down version