A few days ago one of the 2TB disks in my array was set to disabled.  It appeared to be emulated using my 10TB parity disk.  I am confused as to why this happened.  I know that sata cables are the usual suspects but why was it good for so long (months)? 

I am using two HBA cards for the SAS to sata connection.  The full tower is pretty full of stuff and I am asking if I should shut it down, take it apart, reseat the cards, and replace the sata cables on the disks showing error.  I understand that Unraid uses the serial numbers of the drives for assignment in the array so mixing or swapping the sata cables should not be an issue.  Do you think this is advisable or worth doing?

I tried to recover from the disabled disk state by removing it and putting another 10TB drive in its place to do a rebuild. Within a short period, errors started appearing on two and then three of the array disks during the rebuild. The rebuild was completed after about 22 hours with about 500,000,000 errors in the parity check.  At this point, ALL of the array disks are showing NORMAL OPERATION (Green circle).  All the disks in the array are active and indicate a healthy SMART status.  However, they still show the errors and a significant amount of my media files will not play. Also, the SHARES tab indicates: There are no exportable user shares.

I am not sure how to proceed at this point.   Clearly, there is some data loss.  If I could somehow determine which media files are corrupted, I would delete and replace them.  If you could please give me a step-by-step approach on how to go forward I will do it.  Thank you very much for any help you can provide.  I just love this system but I want to get it to be stable.  If that means buying more or different hardware I am certainly willing to do so.  If I remember correctly those HBA cards were about $40 apiece and definitely not worth the hassle if they are causing my problems. Diagnostic attached.

plexnas-diagnostics-20241211-1601.zip

You should have canceled the rebuild, the errors look more like power/connection problem with disk2, check/replace cables.

JorgeB, many thanks. Next time that happens, I will cancel the rebuild as soon as errors pop up. Thank you for this advice.

 

After replacing the cable on disk 2, the system booted, and all the shares came back. Can you please tell me where in the diagnostic you were able to see the power/connection problem with disk 2?

 

After the reboot disk 5 is indicated to be Unmountable: Unsupported or no file system.   I guess I should run the filesystem status check on this drive and allow it to make changes.  Is that correct?  

 

Assuming I can get the array back up with all the drives, I am pretty sure I have some corrupted files.  Is there a plugin, app or methodology that will allow me to identify the corrupted files?

 

Again, thank you for your help.

 

Diagnostic attached.

plexnas-diagnostics-20241212-1052.zip

Because of the read errors on disk2, the rebuilt disk5 may have some corruptions, if you have a spare it may be worth to try and rebuild again, or see if it emulates and copy at least the most important data, you can also repair the filesystem for disk5, but there's likely some data corruption.

 

To see which files are corrupted with xfs, you would need to have pre-existing checksums.

Many thanks JorgeB, because of your help, I think this is very close to being Marked as Solved

 

Could you please tell me where in the diagnostics you found that disk 2 had a bad connection?

Is there a text string I should search for?  Being able to find this will hopefully keep me from bugging you for help.

 

I ran the filesystem repair on disk 5 with the -L option and the array started without errors.  

 

Should I run a parity check (with correction) since there were so many errors during the last check?

Is there anything else I should do before I run the parity check?

 

Does Unraid somehow automatically record pre-existing checksums?   If so how can I access them?

If not, how can I get these checksums to record somewhere after I get all my files working again?

 

The latest diagnostics are attached.

plexnas-diagnostics-20241212-1203.zip

46 minutes ago, demanding-chief3698 said:

Could you please tell me where in the diagnostics you found that disk 2 had a bad connection?

Is there a text string I should search for? 

In the syslog, but can't give you a string, it's by the type and frequency of errors, and they can differ with the controller used, and sometimes can really be a bad disk, that's why I say "looks like"

Thank you JorgeB,  I guess I will continue to ask your advice and console when I have these issues.

 

Please see questions below:

 

Should I run a parity check (with correction) since there were so many errors during the last check?

Is there anything else I should do before I run the parity check?

Only if you are happy with the current rebuild, as mentioned, 

2 hours ago, JorgeB said:

if you have a spare it may be worth to try and rebuild again, or see if it emulates and copy at least the most important data

 

JorgeB,  many thanks!  Your ability to point to the exact disk with the connection problems was the key to getting this fixed.   

I ran the parity check with corrections and it corrected over 293,000,000 errors.  It was my fault that I let it go so far without stopping the previous rebuild/parity check.  That will not happen again but it does show the ability of Unraid to fix itself.   I have marked this issue aa SOLVED.

You provide an outstanding service for my Unraid server which is why I am spending $250 on another license for another server I am building.

The issue regarding why the disk sata connection problem has pulled down my server twice is very disconcerting.   As I mentioned before I have a used HBA card with 8 SAS to sata cables.  I suspect the cables.   I will buy more expensive, more robust (and hopefully), more reliable cables to try and fix this. 

Again, many thanks!!