May 3, 200818 yr run reiserfsck on the /dev/md1, /dev/md2, etc partitions. the way you did it, on the /dev/sd partitions would get the parity drive out of sync. So... a manual parity check/rebuild is in order soon for you. Follow the directions as described here in the wiki: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems Joe L.
May 3, 200818 yr Author run reiserfsck on the /dev/md1, /dev/md2, etc partitions. the way you did it, on the /dev/sd partitions would get the parity drive out of sync. So... a manual parity check/rebuild is in order soon for you. Follow the directions as described here in the wiki: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems Joe L. Thank you again for the advice. I only have 4 disks with ls /dev/md* I can see that those are my 4 data disks. I did exactly what the wiki stated and the drives are back online (and all 4 had no errors). But I would like to check the file system on both the parity drive and cache drive. How would I go about doing that?
May 3, 200818 yr Glad you came out of the labyrinth. You know, when everything else fails, start from the beginning.
May 3, 200818 yr I can see that those are my 4 data disks. I did exactly what the wiki stated and the drives are back online (and all 4 had no errors). But I would like to check the file system on both the parity drive and cache drive. How would I go about doing that? Do NOT use reiserfsck on the parity drive, as it does not have a file system to check. It is simply 100% parity bits. If reiserfsck were run on the parity drive, and tried to make any changes, it would corrupt parity information. You should be able to run it on the Cache drive, but not as an md_ drive, because it is not a part of the parity protected array. Run it using the sd_1 label you were using before. You can get the specific device symbol for it from the Devices tab. I have to say though, that getting only 3 drives to return a good result makes me very suspicious of the whole array. You may be safer starting over from scratch, on all of them. The bad result from testing the parity drive is expected, but no superblock on 2 of the 5 data drives sounds really bad to me.
May 3, 200818 yr Author I can see that those are my 4 data disks. I did exactly what the wiki stated and the drives are back online (and all 4 had no errors). But I would like to check the file system on both the parity drive and cache drive. How would I go about doing that? Do NOT use reiserfsck on the parity drive, as it does not have a file system to check. It is simply 100% parity bits. If reiserfsck were run on the parity drive, and tried to make any changes, it would corrupt parity information. You should be able to run it on the Cache drive, but not as an md_ drive, because it is not a part of the parity protected array. Run it using the sd_1 label you were using before. You can get the specific device symbol for it from the Devices tab. I have to say though, that getting only 3 drives to return a good result makes me very suspicious of the whole array. You may be safer starting over from scratch, on all of them. The bad result from testing the parity drive is expected, but no superblock on 2 of the 5 data drives sounds really bad to me. The only way to do that is to leave 3 drives in the machine and start from a fresh copy of unRAID right? I then have to have unRAID clear the disks again and format right? Just a side note... I put the old ram back in the unRAID server and ran memtest. It's been running error free for the last 8 hours. Maybe I was wrong and the memory wasn't bad... Maybe I had a corrupt hard drive? Is there a utility/tool within the linux command prompt that I can run similar to the disk check in windows? reiserfsck seems to check the format of the hard drive... but I don't think it checks for bad sectors does it?
May 3, 200818 yr is it one stick or two? if two, did you put them the same way in? sometimes mechanical or electronic tolerances are "as much as needed" for specific specific combinations to not perform as they should, where otherwise you don't notice anything maybe even removing the sticks and putting them back was enough when I first saw computers with MS software on them, I introduced "maybe" in my IT vocabulary, which had only yes/no, white/black, 1/0 up to that time... now this goes even for non-MS software and for hardware
May 3, 200818 yr Author is it one stick or two? if two, did you put them the same way in? sometimes mechanical or electronic tolerances are "as much as needed" for specific specific combinations to not perform as they should, where otherwise you don't notice anything maybe even removing the sticks and putting them back was enough when I first saw computers with MS software on them, I introduced "maybe" in my IT vocabulary, which had only yes/no, white/black, 1/0 up to that time... now this goes even for non-MS software and for hardware It's two 1gig sticks... running dual channel. I put them back in the same way as before... not sure if I switched around their positions though. Interesting too... when I put the new ram in I set the memory timings to 4-4-4-12. When I put the old ram (the sticks I thought were bad) back in I forgot to change the timings in the BIOS and it still passed memtest (the old ram timings should be 5-5-5-15). Voltage has been on auto the whole time too.
May 3, 200818 yr guitarlp- Hey if you finally decide that this issue is indeed h/w related and not necessarily unRAID OS, could I ask a favor? Please update your thread over on AVS forum with your final result
May 3, 200818 yr Author guitarlp- Hey if you finally decide that this issue is indeed h/w related and not necessarily unRAID OS, could I ask a favor? Please update your thread over on AVS forum with your final result My thread about using hardware raid on a WHS environment? If that's the case, I already removed all references to unRAID. I never mentioned specifics in the thread... but there's no point to me even mentioning unRAID since it has nothing to do with my question on AVS. I deleted all the data on my flash drive and copied the latest 4.3b files to it along with my pro key. I'm currently doing a parity check and when that's done in a few hours I'm going to copy one movie to all 4 data drives and see how they play on each drive. If that works fine then I'll try to fill up those drives with some data and then try a new movie on each drive and see how they perform. I guess that's the next logical step for now. So far i LOVE unRAID for how it works... I just need to work through this one issue which I now know is file corruption on the unRAID server. How it's getting corrupted I'm not 100% sure... but once I know I'll be one happy camper. I've thought about converting this server into a Windows Home Server but I don't want to lose the storage and flexibility unRAID offers. So I'm planing (if I can finally get this working) to run this along side with my small windows home server box (HP Media Smart Server). That way I can have all the features of Windows Home Server with the great storage/redundancy unRAID provides. At this point I'm pretty sure it's not the unRAID OS that's causing the issue. In fact... I'll change my original post now so any new users that visit your forum don't get turned off on unRAID for HD media streaming.
May 4, 200818 yr after you load your files, have your drives mapped, open a cmd shell from the start menu. click start select run type in cmd press enter. there is a program called comp it will do a binary comparison of the source and destination file. Let's make sure you are working with apples and apples before trying to make sauce. C:\Documents and Settings\rcotrone>comp /? Compares the contents of two files or sets of files. COMP [data1] [data2] [/D] [/A] [/L] [/N=number] [/C] [/OFF[LINE]] data1 Specifies location and name(s) of first file(s) to compare. data2 Specifies location and name(s) of second files to compare. /D Displays differences in decimal format. /A Displays differences in ASCII characters. /L Displays line numbers for differences. /N=number Compares only the first specified number of lines in each file. /C Disregards case of ASCII letters when comparing files. /OFF[LINE] Do not skip files with offline attribute set. To compare sets of files, use wildcards in data1 and data2 parameters. C:\tmp>dir Volume in drive C is XP_C Volume Serial Number is 90A5-8047 Directory of C:\tmp 03/04/2007 08:14 AM <DIR> . 03/04/2007 08:14 AM <DIR> .. 04/05/2008 03:03 PM 2,337,790 test.tgz 2 File(s) 2,337,790 bytes 3 Dir(s) 3,720,876,032 bytes free C:\tmp>mkdir x:\tmp C:\tmp>copy test.tgz x:\tmp 1 file(s) copied. C:\tmp>comp test.tgz x:\tmp\test.tgz Comparing test.tgz and X:\tmp\test.tgz... Files compare OK Compare more files (Y/N) ? n
May 4, 200818 yr Author after you load your files, have your drives mapped, open a cmd shell from the start menu. click start select run type in cmd press enter. there is a program called comp it will do a binary comparison of the source and destination file. Let's make sure you are working with apples and apples before trying to make sauce. C:\Documents and Settings\rcotrone>comp /? Compares the contents of two files or sets of files. COMP [data1] [data2] [/D] [/A] [/L] [/N=number] [/C] [/OFF[LINE]] data1 Specifies location and name(s) of first file(s) to compare. data2 Specifies location and name(s) of second files to compare. /D Displays differences in decimal format. /A Displays differences in ASCII characters. /L Displays line numbers for differences. /N=number Compares only the first specified number of lines in each file. /C Disregards case of ASCII letters when comparing files. /OFF[LINE] Do not skip files with offline attribute set. To compare sets of files, use wildcards in data1 and data2 parameters. C:\tmp>dir Volume in drive C is XP_C Volume Serial Number is 90A5-8047 Directory of C:\tmp 03/04/2007 08:14 AM <DIR> . 03/04/2007 08:14 AM <DIR> .. 04/05/2008 03:03 PM 2,337,790 test.tgz 2 File(s) 2,337,790 bytes 3 Dir(s) 3,720,876,032 bytes free C:\tmp>mkdir x:\tmp C:\tmp>copy test.tgz x:\tmp 1 file(s) copied. C:\tmp>comp test.tgz x:\tmp\test.tgz Comparing test.tgz and X:\tmp\test.tgz... Files compare OK Compare more files (Y/N) ? n I just copied the movie from my local disk to disk1 on unRAID and ran the comp command. It returned the following: Compare Error at Offset 84CDB560 File1 = 4A File2 = 48 I ran the command comparing from my local storage to the network storage (\\Tower\disk1\movie_name.iso). I then copied the working file from my local drive to unRAID's disk2 (different size and brand HD). I ran the compare command and got the following error: Compare Error at OFFSET 425A3560 File1 = 52 File2 = 50 Is my error above a problem? Could it possibly be because I'm comparing accross my network? Or is that file now corrupt because it's not exactly the same as the file on my local drive? Since I'm getting the error on 2 different drives it's not a hard drive issue. Memory passed memtest for over 8 hours so that shouldn't be the problem.
May 4, 200818 yr This reveals that it's not the network player starved for data. It reveals a bigger issue in that you cannot reliably transport data from your source to unraid or vica versa. I suppose the next test I would do is test each disk with comp. If there are errors elsewhere, I would try and download a very large zip or tar.gz file (unix kernel perhaps) using wget. Then test the tar.gz with gunzip -tv You can get wget here http://packages.slackware.it/package.php?q=current/wget-1.11.1-i486-1 you can download the kernel by telnet'ing into the machine. Installing the wget package with installpkg wget-1.11.1-i486-1.tar.gz then cd to a drive and directory and issue a wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.25.tar.gz then gunzip -tv linux-2.6.25.tar.gz This will eliminate the windows workstation as a possible source of trouible. If there is anything overclocked in your machine whatsoever, I would suggest taming the settings until the issue is resolved. Just some idea's, this has me a bit stumped and I'm going to have to do some testing myself.
May 4, 200818 yr Author This reveals that it's not the network player starved for data. It reveals a bigger issue in that you cannot reliably transport data from your source to unraid or vica versa. I suppose the next test I would do is test each disk with comp. If there are errors elsewhere, I would try and download a very large zip or tar.gz file (unix kernel perhaps) using wget. Then test the tar.gz with gunzip -tv You can get wget here http://packages.slackware.it/package.php?q=current/wget-1.11.1-i486-1 you can download the kernel by telnet'ing into the machine. Installing the wget package with installpkg wget-1.11.1-i486-1.tar.gz then cd to a drive and directory and issue a wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.25.tar.gz then gunzip -tv linux-2.6.25.tar.gz This will eliminate the windows workstation as a possible source of trouible. If there is anything overclocked in your machine whatsoever, I would suggest taming the settings until the issue is resolved. Just some idea's, this has me a bit stumped and I'm going to have to do some testing myself. I'll try everything you mentioned in a bit and report back with my findings. Just for testing I tried running a comp on disk1 again and I got an error at the exact same place I did the first time I ran a comp on disk1. So it's not a network issue on reading the file from the server. The movie must become corrupt when being written to the unRAID server. The fact that the coruption happens at different places when put onto different drives shows that it's a writing issue. When comparing the 2 files the corruption shows up at the same place during each test (which shows it's not a reading issue). I'm completely lost on this one. Hardware checks out (different CPU's, MOBO's, DISKS, RAM passes memtest), network checks out, transfer speeds are fine. Something on the unRAID server is causing corruption though.
May 4, 200818 yr Author This reveals that it's not the network player starved for data. It reveals a bigger issue in that you cannot reliably transport data from your source to unraid or vica versa. I suppose the next test I would do is test each disk with comp. If there are errors elsewhere, I would try and download a very large zip or tar.gz file (unix kernel perhaps) using wget. Then test the tar.gz with gunzip -tv You can get wget here http://packages.slackware.it/package.php?q=current/wget-1.11.1-i486-1 you can download the kernel by telnet'ing into the machine. Installing the wget package with installpkg wget-1.11.1-i486-1.tar.gz then cd to a drive and directory and issue a wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.25.tar.gz then gunzip -tv linux-2.6.25.tar.gz This will eliminate the windows workstation as a possible source of trouible. If there is anything overclocked in your machine whatsoever, I would suggest taming the settings until the issue is resolved. Just some idea's, this has me a bit stumped and I'm going to have to do some testing myself. I tried copying wget-1.11.1-i486-1.tgz to my flash drive and once logged in via telnet running: installpkg wget-1.11.1-i486-1.tgz But it's saying: Cannot install wget-1.11.1-i486-1.tgz: package does not end in tgz Like I said before... I don't know my way around linux very well so dumb it down for me
May 4, 200818 yr Author Also, once I get wget installed... how would I open disk1 to download into? I can run the following: cd /dev But I can't seem to do: cd /md1 after I'm in the dev directory. What's the correct way to enter disk 1's root directory?
May 4, 200818 yr forget the slash in your second cd (if you keep it, you try to find md1 in your ROOT folder again)
May 4, 200818 yr Author forget the slash in your second cd (if you keep it, you try to find md1 in your ROOT folder again) I tried just: cd md1 and it returned: -bash: cd: md1: Not a directory. I also tried: cd /dev/md1 but that got me the same "not a directory" error above.
May 4, 200818 yr Author Also... just for testing I uploaded my movie to my Windows Home Server and ran the comp command... it returned no errors and the file was perfect.
May 4, 200818 yr Author I disabled the parity drive and tried uploading my movie to disk1. I ran comp from windows command line and it failed the test. Just thought I'd let you guys know (thought maybe parity had something to do with it). I want to try the method of downloading and decompressing as someone mentioned above, but I lack the knowledge of how to install wget so I'll have to wait until someone can help me out with that. Is there anything else I can test? Heck, I thought it may even be my flash drive but I tried another one (could only use 3 disks because it's not registered) but it failed the comp test as well. I'm serriously lost... all my hardware sounds like it's running fine... same with my network. What else could be corupting these files? Could it be the unRAID OS itself?
May 4, 200818 yr Author I just tried moving my 5 disks from one Athena bay to the other (I wasn't using the other one yet). I thought maybe the Athena backplane may be causing the issues, but switching didn't fix anything. I tried connecting 2 drives directly to the motherboard... but those threw errors as well when I compared them to the original file. I'm begininng to think this is software related now. I don't know what else I can do to check the hardware.
May 4, 200818 yr (Here's Rob talking about nForce 4 again, must be tiresome for others!) Your comp errors are precisely the kind of common data corruption problem that occurs with nForce 4 motherboards. Is it possible you have an nForce 4 or earlier board? If it is, trash it! Take a look at every byte change, and see if it is the same bit change every time. In your case above, both byte changes were identical, the 2 bit is stuck, in the same direction (4A to 48, and 52 to 50). Need more cases to confirm. Also, the addresses will show a pattern, and with enough cases, you can tell exactly which byte offset will have the bit stuck on, and how big the buffer or register is that contains the faulty byte. In the above case with only 2 points (0x84CDB560 and 0x425A3560), all I can say is the byte offset is 0x1560, probably within a 0x2000 or 0x4000 sized buffer. Need more points. That 0x1560 does seem very suspicious though. I would also like to recommend using 'fc /b' instead of comp. Both are good, but fc /b provides a better formatted display, and is not limited to 10 byte differences. Comp.com does handle wild cards, fc.exe does not. For easier testing, I usually create an FCB.BAT, with something like: @fc /b %1 %2\%1 >>diffs Then I put myself in one of the folders I'll be testing between, and type something like fcb testfile.mpg remotefolder, and then view the diffs file. To tell if the corruption is occurring on the read of the file, or the write of the file, or both, repeat an identical compare 3 or 4 times, more as needed. If the corruption is occurring during the read of the file, then the byte differences will be random, always different each run. If they are occurring during the write of the original file, then they will always repeat. If some byte differences repeat, and others are random, then the corruption is occurring with reads and writes. Generally, for a good reliable test, that consistently results in byte differences, you need a test file that is very large, 2 or more gigabytes. This is one of many cases where working in hex is easier than the alternative. Bit changes are obvious, and the address patterns will appear fairly quickly. You are looking for the lowest common divisor among all of the addresses that occur. That will give you the register or buffer size, and byte offset within that buffer, and help to determine what hardware or software component might be using a buffer of that size.
May 4, 200818 yr By the way, another user somewhat recently had similar problems, and it turned out that the unRAID machine was OK, but the Windows station he was using to copy from was an nForce 4 board, and causing the corruption.
May 4, 200818 yr If it is software related, it is only for you. Doesn't seem strange? I stand by my original opinion. Hardware. Could be anything. Best test is to install some other OS on the SAME machine (don't touch anything, cables etc.) and test. Maybe some live Linux distro. Or some lite Windows. Try to copy some (large enough) zipped file to the array. Then run test on the archive as is on the array AND after you copy it back to your local machine. Messed up archived files are easier to spot than going over a video. And to my original test/question (because I don't think I got a clear answer): - You have a local video file that works fine. - You copy it to the array. - Your copy on the array stops playing ok? - You copy BACK FROM the array. - Your local NEW copy, plays ok? New Q: - Same file same artifacts at same place ALL the time? - If you copy on different disk on the array, still get artifacts? - Same or different artifacts? Two more extreme tests (first easier): #1 - Mount (see wiki on how) a normal NTFS disk on the array, WITH the video. - Copy LOCALLY on the array from that disk. - Try to play it. #2 - Install one of these: http://p-nand-q.com/download/rfstool.html http://rfsd.sourceforge.net/ #2a - Copy the file over the network to the array as you did all this time. - Take AWAY the disk from the array and install it on your windows machine. - Access it using the tools above. - Does the file work? #2b - First take away the disk and install it on windows. - Access it using the tools. - Copy the file locally from windows. - Play it from reiserfs fisk (on windows). Plays ok? - Put it back to unRAID. - Try to play it over network. Plays ok? All these will help pinpoint the problem. But need time.
May 4, 200818 yr If it is software related, it is only for you. Doesn't seem strange? I stand by my original opinion. Hardware. Could be anything. I think we're all in agreement here. RobJ points to a very specific hardware issue with the chipset (and related driver). retry the wget (as below) and eliminate the windows station altogether. I tried copying wget-1.11.1-i486-1.tgz to my flash drive and once logged in via telnet running: installpkg wget-1.11.1-i486-1.tgz But it's saying: Cannot install wget-1.11.1-i486-1.tgz: package does not end in tgz Like I said before... I don't know my way around linux very well so dumb it down for me telnet to the unraid server, login as root cd to where you stored wget do an ls -l wget* if it is named incorrectly (as in two .tgz extentions) you can rename it with mv mv wget-1.11.1-i486-1.tgz.tgz wget-1.11.1-i486-1.tgz
May 4, 200818 yr Author (Here's Rob talking about nForce 4 again, must be tiresome for others!) Your comp errors are precisely the kind of common data corruption problem that occurs with nForce 4 motherboards. Is it possible you have an nForce 4 or earlier board? If it is, trash it! Take a look at every byte change, and see if it is the same bit change every time. In your case above, both byte changes were identical, the 2 bit is stuck, in the same direction (4A to 48, and 52 to 50). Need more cases to confirm. Also, the addresses will show a pattern, and with enough cases, you can tell exactly which byte offset will have the bit stuck on, and how big the buffer or register is that contains the faulty byte. In the above case with only 2 points (0x84CDB560 and 0x425A3560), all I can say is the byte offset is 0x1560, probably within a 0x2000 or 0x4000 sized buffer. Need more points. That 0x1560 does seem very suspicious though. I would also like to recommend using 'fc /b' instead of comp. Both are good, but fc /b provides a better formatted display, and is not limited to 10 byte differences. Comp.com does handle wild cards, fc.exe does not. For easier testing, I usually create an FCB.BAT, with something like: @fc /b %1 %2/%1 >>diffs Then I put myself in one of the folders I'll be testing between, and type something like fcb testfile.mpg remotefolder, and then view the diffs file. To tell if the corruption is occurring on the read of the file, or the write of the file, or both, repeat an identical compare 3 or 4 times, more as needed. If the corruption is occurring during the read of the file, then the byte differences will be random, always different each run. If they are occurring during the write of the original file, then they will always repeat. If some byte differences repeat, and others are random, then the corruption is occurring with reads and writes. Generally, for a good reliable test, that consistently results in byte differences, you need a test file that is very large, 2 or more gigabytes. This is one of many cases where working in hex is easier than the alternative. Bit changes are obvious, and the address patterns will appear fairly quickly. You are looking for the lowest common divisor among all of the addresses that occur. That will give you the register or buffer size, and byte offset within that buffer, and help to determine what hardware or software component might be using a buffer of that size. None of my boards are nForce boards. unRAID is using a Gigabyte GA-EP35-DS3R and the PC transferring to the unRAID server is using a Gigabyte GA-MA78GM-2SH. The unRAID is using the P35 chipset and the HTPC is using the 780G chipset. I created the FCB.BAT in my C:\ root directory with the command you posted inside that file. I then have the good working movie file in that same directoy. I typed: FCB MOVIE_NAME.ISO \\NAS\disk1\MOVIE_NAME.ISO I hit enter and it returned: FC: Cannot open \\NAS\disk1\MOVIE_NAME.ISO/MOVIE_NAME.ISO - No such file or folder What am I doing wrong?
Archived
This topic is now archived and is closed to further replies.