I had a drive go down, was getting the "UDMA_CRC_Error_Count" errors greater than zero, so I RMA'd the drive...or so I thought.  What I had actually done was send them a good drive, and when the new one came in, I got the invalid configuration error and would unRAID not let me start.  I tried to parity check it, but due to the bad drive, I was unsuccessful.  So fast forward to today, and I have RMA'd the correct drive and now I have all good drives (so I hope), including two drives that should be wiped clean, but the other drives should still have data on them?  I put it in and get the invalid configuration, so I do the TRUST MY ARRAY procedure, and now I can start it.  I start it, and it forces me to parity check, and of course my parity check freezes every time.  I have 6 1.5TB, so yea i presume it would take awhile, but is the webpage suppose to go down so I cannot even connect to it during this process? 

 

I run a:

 

2.5TB HDD x 6 WD

Quad Core 2.4 GHZ APU Fusion Processor w/ 6550D GPU

2GB DDR3 1333 RAM

650 Watt PSU

 

I have done this twice now and it does not even get past 0%, and have to do a hard reset.  The new hard drive was formatted via unRaid software.  This was done after I stopped the parity check, and restarted it. The parity is orange (would love to attach picture but it is bigger than size allows).  I have attached the syslog.txt, but what good it does since my last restart I am unsure what you will get from it...hoping something. 

 

Issues:

 

I want my array to start up and get past parity check, if all the data is on there it should be about 1TB worth of data.  If some was lost in the replacing of good drive by mistake, then there will be less...either way gotta get past parity check to find out.

 

If I have my data (or some) great, if not, then I need to know what was fragmented and worthless so I can start replacing it.  I do not know how data is stored on the hard drives, so if it was fragmented does it automatically go away or will it keep some of the data.  For the most part, the server holds .iso files so they are gigs worth of data.  I lost two drives, and I know unRaid does not account for this.  If you can help me recover what data has been saved to the parity, great, if not I have already reconciled that I may have lost everything due to my own stupidity, and will honestly just need help starting my array please.  If the data is all fragmented to the point that I will not have complete .iso files, then I would rather just start over fresh.  Give me options please.

 

I am pretty novice at all this, and I do not know much, so do treat me as such.  You ask what version I am using, and I do not even know how to tell this.  The unRaid webpage //tower/main says version 5.0 beta 14.  I do not even remember having a choice on which one it uses.  I have only had this server up for a month and the past month since then has been waiting for hard drives to come in.  I did buy a cold drive, so I do not have to go through this again, but anyway, is there anyone out there that can get my parity check to work so I can see what data has been lost/fragmented and start getting data back on there?  Thanks

syslog.txt

I don't know if this will work or not, but it honestly won't hurt anything any worse than it already is, so here goes.

 

Unassign the parity drive so it's not participating, then tell unraid to trust the array so you can start it.

 

On a positive note, individual files are never fragmented across multiple drives, so if you can copy the file off of the disk, it should be ok. Directories can be split so that some of the files are in one drive, and other files elsewhere, but if the file exists on a drive, it's the whole file.

I'm not sure why you did TRUST MY ARRAY. The array was decidedly untrustworthy. I think you wanted to click on the New Config utility.

 

The syslog is only 20 seconds long and does not show any problems.

 

If I'm understanding, you have two blank replacement drives, three data drives, and an invalid parity drive. If all of theses drives are good then the parity check should complete and the web interface should not crash.

 

How did you determine which drive had failed? It appears to still be in the array.

 

Post SMART reports for all the drives.

What specific make/model power supply are you using? 

Have you run a memory test? (preferably overnight)

 

It sounds like you have hardware issues that need to be resolved first.

 

The web-interface does not just "crash" (not unless something uses up all available memory)

Hard-resets are almost never the answer.  Forcing parity to be considered valid and trusting the array as you seem to have done is almost always the WRONG step to take when having problems.

 

UDMA CRC errors (checksum errors in the communicating with the drive) are indications of noise pickup on SATA connections.

(either bad data or power cables, too many splitters in line with disks, poor quality cables, data cables routed too close to power cables, noisy power supply voltages, bad memory, or, a bad disk.)

 

Joe L.

I am very glad to know that the data is not fragmented across all hard drives.  That was a relief to find out.  I did as Johnathanm suggested and was able to backup 61 of the 100 files that were originally on the 5 discs, now only data from the remaining 3 drives.  Looks like I lost 39 files from the 2 RMA’d drives?  Or are they still on the parity drive, and once I am able to get through a parity check they will return?  While I was copying over the data, errors started popping up, and I have saved my syslog for you to review.  Looks like another hard drive is having problems, hard drive 3 from the looks of it.

 

@dgaschk

 

“I'm not sure why you did TRUST MY ARRAY. The array was decidedly untrustworthy. I think you wanted to click on the New Config utility.”

 

Under the FAQ, it says, “How can I avoid rebuilding my parity or data drive when I know it is already good?  It is becoming known as the Validate Array or Trust My Array procedure. See this HowTo: Make unRAID Trust the Parity Drive, Avoid Rebuilding Parity Unnecessarily.”

 

I had to find a way to start my array because I was getting the invalid configuration error from the missing data discs (RMA’d ones).  From my point of view, all my data was on my parity still and it was good, so I just needed the array to start and it would reinvigorate disc 4 and disc 5.  I know from the FAQ that I did not want to restore my array (there was no button for this anyway), so it sounded perfect to me, so I followed the directions.

 

Now I wanted to avoid the parity check and start my array, and with jonathanm instructions I was able to start it by unassigning the parity drive.  You are correct that I have 2 blank RMA’d drives (disc 4 and 5) and 3 data drives (looks like 3 may be bad and is impeding my parity check), but why is my parity drive invalid?  I did not find any links to the NEW CONFIG utility when I was checking for answer on this problem, but it sounds promising.  I will await instructions before I start anything though.

 

I agree with you that the web interface should not crash and expected my parity check to run smooth once I got the RMA’d drives, but it does not appear to be.  I was hoping that the parity drive still has the data from disc 4 and disc 5 and is just waiting to pump that data into the new drives, but who knows, at least I was able to save 61/100 files (thanks johnathanm). 

 

The issue I may have now is if one of these last 3 data drives is now bad (appears to be disc3), can I trust the data that I just backed up?  With the files being so ginormous, it would only take a few bits off or missing and it would make the file unreadable…right?

What specific make/model power supply are you using? 

Have you run a memory test? (preferably overnight)

 

It sounds like you have hardware issues that need to be resolved first.

 

The web-interface does not just "crash" (not unless something uses up all available memory)

Hard-resets are almost never the answer.  Forcing parity to be considered valid and trusting the array as you seem to have done is almost always the WRONG step to take when having problems.

 

UDMA CRC errors (checksum errors in the communicating with the drive) are indications of noise pickup on SATA connections.

(either bad data or power cables, too many splitters in line with disks, poor quality cables, data cables routed too close to power cables, noisy power supply voltages, bad memory, or, a bad disk.)

 

Joe L.

 

Previous problem: before I RMA'd the right bad disc (if that makes sense), the first thing I tried was replacing sata cables.  I ran the smart report and it still gave me those same errors.  Hence why I RMA'd it.  I will look into the model of power supply I have.

 

Current problem:  I am now getting errors in disc 3, and when I try to run the smart report, the text file just gives me the first two lines, no reported data back.  I could attach the reports, but honestly seems pointless.  You want me to run a memory test over night, is there not a way I can try to correct the errors I am having on disc 3 first? 

syslog.zip

Power supply and motherboard:

 

GIGABYTE GA-A75M-S2V FM1 AMD A75 (Hudson D3) SATA 6Gb/s USB 3.0 Micro ATX AMD Motherboard

CORSAIR Enthusiast Series TX650 V2 650W ATX12V v2.31/ EPS12V v2.92 80 PLUS BRONZE Certified Active PFC High Performance Power

Power supply and motherboard:

 

GIGABYTE GA-A75M-S2V FM1 AMD A75 (Hudson D3) SATA 6Gb/s USB 3.0 Micro ATX AMD Motherboard

CORSAIR Enthusiast Series TX650 V2 650W ATX12V v2.31/ EPS12V v2.92 80 PLUS BRONZE Certified Active PFC High Performance Power

Not likely to be the power supply..  That is a very nice single-rail 12volt supply. 

Just wanted to eliminate it as a possible problem.  So many users have 700Watt 4-rail supplies that only have a single 18 Amp rail powering all their disks.  They frequently work fine until that to above 6 or 7 disks.

You are making so many wrong assumptions.

 

The UDMA_CRC_Error_Count errors in the SMART data will be there forever. Changing the cable may make the system work correctly but it doesn't correct the fact that these errors occurred in the past and were counted. I have a drive with 15 showing for the UDMA_CRC_Error_Count. I had it on a bad SATA cable so I changed the cable and the 15 still showing makes no difference on the drive operation.

 

unRAID does not store the data from the disks on the parity. unRAID uses the parity drive PLUS the data on every other drive to reconstruct a failed drive. In other words, unRAID uses the parity drive to give you protection against a SINGLE disk failure.

 

Using the trust my array procedure when a drive is changed is completely wrong. There are very few times you can use this and a failed drive isn't one of those times. In fact, for most users they should never attempt to use it.

 

You have lost all the data on the 2 drives you RMA'd. It is gone and can not be recovered. unRAID protects against a single disk failure. You have lost 2 disks so it can't do anything.

 

You need to track your drives vs assignments better. When you learn that say disk2 is bad then you need to know which physical drive is disk2. You basically get one chance to swap the bad drive for a good one so randomly swapping drives hoping for the best is a terrible idea. unRAID shows the serial number of the drive as part of the info so make use of that info.

 

 

I would say you now need to do this. Assign the parity drive and the old data drives to the slots you want them to be in. Use the initconfig command line (4.7 or beta) or New Config on the web interface (beta only) and then start the array. A parity build should start so let it complete. All the drives should show up and the data should be present.

 

Once that is done, download the preclear script and run it for both new drives. If they pass the preclear, then stop the array, assign the drives and start the array again. Confirm that ONLY the new drives show as unformatted and then press the format button to format them.

 

Peter

 

 

 

You are making so many wrong assumptions.

 

The UDMA_CRC_Error_Count errors in the SMART data will be there forever. Changing the cable may make the system work correctly but it doesn't correct the fact that these errors occurred in the past and were counted. I have a drive with 15 showing for the UDMA_CRC_Error_Count. I had it on a bad SATA cable so I changed the cable and the 15 still showing makes no difference on the drive operation.

 

unRAID does not store the data from the disks on the parity. unRAID uses the parity drive PLUS the data on every other drive to reconstruct a failed drive. In other words, unRAID uses the parity drive to give you protection against a SINGLE disk failure.

 

Using the trust my array procedure when a drive is changed is completely wrong. There are very few times you can use this and a failed drive isn't one of those times. In fact, for most users they should never attempt to use it.

 

You have lost all the data on the 2 drives you RMA'd. It is gone and can not be recovered. unRAID protects against a single disk failure. You have lost 2 disks so it can't do anything.

 

You need to track your drives vs assignments better. When you learn that say disk2 is bad then you need to know which physical drive is disk2. You basically get one chance to swap the bad drive for a good one so randomly swapping drives hoping for the best is a terrible idea. unRAID shows the serial number of the drive as part of the info so make use of that info.

 

 

Again this was the old problem.  This drive was RMA'd and now this is not the problem. I replaced the cables, still got the error, got a new drive with new cable, no error (on that drive..so far).  I understand that unRAID only accounts for one drive failure and I have created two.  The point is...what can I do now? There are so many questions that have been asked, that no one has responded to.  Everyone seems to be hung up on how I got here, and not moving me forward.  I get it, do not use "trust my array procedure", got it.  All I wanted is for my array to start and not hear "oh well it only accounts for a loss of 1 drive so you are hosed."  For me, trust my array saved 3 discs worth of data, so its hard for me to look at using that procedure as a bad thing...as it is better than you are just completed hosed scenario.  Though, I do not know if the backup data is solid, but right now it's at least something. 

 

 

I would say you now need to do this. Assign the parity drive and the old data drives to the slots you want them to be in. Use the initconfig command line (4.7 or beta) or New Config on the web interface (beta only) and then start the array. A parity build should start so let it complete. All the drives should show up and the data should be present.

 

Once that is done, download the preclear script and run it for both new drives. If they pass the preclear, then stop the array, assign the drives and start the array again. Confirm that ONLY the new drives show as unformatted and then press the format button to format them.

 

 

Peter, I appreciate your help, but you seem to be leading me down the path I was on, hence the name of this forum topic is "parity check freezes."  I would love my parity check to complete, but it is not.  Smart reports are not working either.  Disc3 seems to be the culprit according to the syslog, what do I do now?  The log keeps saying "FSCK?"  Is it asking me to repair it, to bypass it?  I have a cold drive unformatted now, but then we are going to get into the wrong configuration all over again, and I am not sure what procedure to do, as "trust my array" is clearly not favored, and then if I put in a new drive, then I am down to 2 original drives of data.

 

My thinking is that disc3 probably was in the process of writing, when web-interface would not longer load or see the server, and I hard reset.  So if that is a presumption, can I override that and save the drive and the data?  That would be nice, but again it is an assumption and I have been poor at those so far, hence why I am asking the forum.

 

Can anyone respond to: Is the backup data that I saved any good or is the fact that I have errors on Disc3 make that data suspect? 

 

I frankly do not even know what a preclear script is, and it sounds like you want me to clear all the data from all discs and format, and if that is the next step, so be it, but I cannot even get the parity check to run through to even get to that step. 

 

My ideal is to save disc3, with whatever is going on with it, keep the data on disc1,2, and 3 and start my array with 5 drives and start putting data back into the array.  Is this possible?

unRAID will recover from a single disk failure. 2 disks replaced = data lost. That's it, no if and or buts about it. I have no idea how you saved 3 disks of data by using the trust my parity procedure. The trust my parity procedure does nothing to recover any data so it pretty much impossible to use it and recover 1 disk of data let alone 3 disks of data.

 

You had a working array. Now you don't. You put 2 new disks into the array and now it's broken and locks up every time you try to start it and do a parity check. It's pretty simple to put 2 and 2 together and figure out that it's likely something to do with the new disks - bad replacement drive or bad SATA port or bad SATA cable or bad power cable. So, remove those disks and try without them. Then, you can go back and test them and see if one has issues.

 

Peter

 

unRAID will recover from a single disk failure. 2 disks replaced = data lost. That's it, no if and or buts about it. I have no idea how you saved 3 disks of data by using the trust my parity procedure. The trust my parity procedure does nothing to recover any data so it pretty much impossible to use it and recover 1 disk of data let alone 3 disks of data.

 

You had a working array. Now you don't. You put 2 new disks into the array and now it's broken and locks up every time you try to start it and do a parity check. It's pretty simple to put 2 and 2 together and figure out that it's likely something to do with the new disks - bad replacement drive or bad SATA port or bad SATA cable or bad power cable. So, remove those disks and try without them. Then, you can go back and test them and see if one has issues.

 

 

I had a working array with 5 discs and a parity.  Disc 4 and disc 5 are the new discs from the RMA process.  I was able to recover the data from disc 1, 2, and 3 and backed that up by starting the array without the parity drive so I avoided the parity check using command line initconfig (first step of trust my array procedure).  Anyway, the array starts fine with disc 1,2,3,4, and 5, and this is how I was able to save the data (see above).  So data does not equal lost, data on disc 4 and 5 equal lost...yes.  It is the parity check it cannot get through. 

 

According to the syslog, disc3 is the one that causing the hangups during the parity check.  If I remove disc4 and disc5, how is that going to make my parity check work if disc3 is the issue, unless you want me to use disc 1,2, and 4 or 1,2, and 5?  Let along the process of removing 2 discs from an array is going to give me the "dreaded" invalid configuration error on the web-interface.  My thought is to use trust my array procedure, but that seems to be frowned upon, so what is the correct procedure to make an array start with 3 drives when their should be 5?  Did anyone review my syslog.zip?  Can we try to repair disc3 first?  I am currently doing a memory test 86+ as requested. 

If it were me, and everything here is exactly as you have stated, I would proceed as follows.

 

Assuming what you posted first is still true, that you have 6 drives all identical in size, I'd start out by preclearing the two fresh drives at least 1 cycle, preferably 2 or 3 cycles. Google "preclear unraid" without the quotes if you haven't figured out what preclear is yet. I would then start a new array with only those two drives, one as parity, one as data. Let it complete a parity build on the blank array, shut down properly, bring it back up, and start another parity check. After that completes successfully, then create your shares and start copying your backup data onto the fresh array. After you compare the array contents to your backups and can trust the array to hold data, then I would start preclearing the other drives one at a time for as many cycles as you can stand, and adding them in to the active array one at a time. After you add each drive, I would do a full parity check to confirm that the array is still responding properly.

 

As far as recovering the data on drive 3, if you meant what you said in the first post about moving on with life and regenerating data that was lost from other sources, then I'd personally give up on drive 3. Data recovery will leave you with data you can't totally trust.

 

This course of action will take a couple weeks to fully implement, but when you are done, you should have an array you know inside and out, and can trust. You've already spent a bunch of time and effort chasing errors, if it were me I'd want confidence before I moved on.

 

If you decide to do the new array and build it back 1 drive at a time, stop and post a syslog and ask for help any time things don't go as they should. That way hopefully you can find the problem before it jeopardizes your data again.

 

This is just my opinion, feel free to disregard and try something else if you want, I'm hardly a senor expert here.

 

Jonathan

If it were me, and everything here is exactly as you have stated, I would proceed as follows.

 

Assuming what you posted first is still true, that you have 6 drives all identical in size, I'd start out by preclearing the two fresh drives at least 1 cycle, preferably 2 or 3 cycles. Google "preclear unraid" without the quotes if you haven't figured out what preclear is yet. I would then start a new array with only those two drives, one as parity, one as data. Let it complete a parity build on the blank array, shut down properly, bring it back up, and start another parity check. After that completes successfully, then create your shares and start copying your backup data onto the fresh array. After you compare the array contents to your backups and can trust the array to hold data, then I would start preclearing the other drives one at a time for as many cycles as you can stand, and adding them in to the active array one at a time. After you add each drive, I would do a full parity check to confirm that the array is still responding properly.

 

As far as recovering the data on drive 3, if you meant what you said in the first post about moving on with life and regenerating data that was lost from other sources, then I'd personally give up on drive 3. Data recovery will leave you with data you can't totally trust.

 

This course of action will take a couple weeks to fully implement, but when you are done, you should have an array you know inside and out, and can trust. You've already spent a bunch of time and effort chasing errors, if it were me I'd want confidence before I moved on.

 

If you decide to do the new array and build it back 1 drive at a time, stop and post a syslog and ask for help any time things don't go as they should. That way hopefully you can find the problem before it jeopardizes your data again.

 

This is just my opinion, feel free to disregard and try something else if you want, I'm hardly a senor expert here.

 

Jonathan

 

Fair enough.  It takes a full 24 hours to clear a 2.5 TB drive, so I am not sure how many of those I can stand.  My concern is still disc 3.  Should I RMA it at this point or attempt to preclear it and then hopefully it will be ok after I add back in?  Again, what of the data that I copied over and saved, it did include data from disc 3, can that data be trusted?  Can I use the initconfig and unassign disc 3 and then backup the data only from disc 1 and disc 2? 

Preclear will give you a smart report both before and after the preclear cycle, I'd post the results here if you can't decide whether or not to RMA the drive after that. I'd reserve judgment until you have a clean working array. It's possible the drives have never been your problem, and the rebuilding and testing should help root out any other issues.

 

You can preclear multiple drives at once to save time, I'd use the screen command to do a two or three at a time. Once again, google is your friend if you don't know what I'm talking about. Google "unraid preclear screen" without the quotes.

 

Until you have a handle on this whole situation, I'd save the syslog each time before you shutdown the array and keep them archived by date and time. That way if you start having issues again during the process you will have the syslogs to look back and try to find the issue.

 

Also, be VERY sure which physical drive you are working with at all times. Nothing sucks more than accidentally clearing the wrong drive. Preclear will give you a serial number on the confirmation screen, if you don't have the last few digits of the serial numbers marked on the drives where you can read them when you open the case or on their removable cages, I'd do that now.

 

Jonathan

I had a working array with 5 discs and a parity.  Disc 4 and disc 5 are the new discs from the RMA process.  I was able to recover the data from disc 1, 2, and 3 and backed that up by starting the array without the parity drive so I avoided the parity check using command line initconfig (first step of trust my array procedure).  Anyway, the array starts fine with disc 1,2,3,4, and 5, and this is how I was able to save the data (see above).  So data does not equal lost, data on disc 4 and 5 equal lost...yes.  It is the parity check it cannot get through.

 

LOL, the trust my parity procedure does nothing to "save" the data on a good data disk.

 

According to the syslog, disc3 is the one that causing the hangups during the parity check.

 

If it's disk 3 that has a problem then remove that disk and try without it.

 

Also, if it is disk3 then you have lost 3 disks of data.

 

 

LOL, the trust my parity procedure does nothing to "save" the data on a good data disk.

 

Very true, but what he meant was that without the parity disk assigned, he was able to start the array in a damaged state and copy data from the drives without the parity check hanging the machine and keeping him from copying anything.

 

He knows he lost data, at this point he just wants a running array he can trust.

 

At least that's what I interpreted from his posts.

 

Feel free to help out with better options. I'm just a newbie here myself.

 

Jonathan

 

He knows he lost data, at this point he just wants a running array he can trust.

 

At least that's what I interpreted from his posts.

 

 

Very true.  I am running the preclear scripts now, and disc3 will not budge from 0%.  Its like it is connected, but no data (zeros) being put through to it?  Motherboard port possibly?  I have no idea why I can see it, but nothing is going on with it.  Also does not explain how I was able to get data from it initially...then again maybe I did not.

 

He knows he lost data, at this point he just wants a running array he can trust.

 

At least that's what I interpreted from his posts.

 

 

Very true.  I am running the preclear scripts now, and disc3 will not budge from 0%.  Its like it is connected, but no data (zeros) being put through to it?  Motherboard port possibly?  I have no idea why I can see it, but nothing is going on with it.  Also does not explain how I was able to get data from it initially...then again maybe I did not.

can you get a "smart" report from it?  (you can do it even while being cleared.)  Sounds like the drive stopped responding.

 

Joe L.

I was running a second preclear, and my server appears to be frozen in step 10 of 10.  Reviewing the syslog, I keep getting ata bus errors.  I can power down from web interface, but have not yet in the hope that it will not be frozen.  Any ideas?

syslog-2012-02-23.txt

This usually indicate a bad SATA cable: http://lime-technology.com/wiki/index.php?title=The_Analysis_of_Drive_Issues#Drive_Interface_Issues

All 6!??

The problem may take several trials to isolate. Start with a 24 hour memtest.

Memtest was completed for 18 hours over night with no errors.

Please provide complete hardware specs.