Hi all,

 

I recently went to upgrade my parity drive to a larger disk and ran into an issue about less than 1% into the process where i received a Warning message that reads

 

"Array has 1 disk with read errors" (error count is 219)

 

prior to replacing my parity drive i had no errors for any disks. 

 

what are my best next steps? do i stop this parity rebuild on the new parity disk, put back in the original parity drive, and replace the bad drive? or let this parity rebuild finish and then address? 

 

if the first option, what is the correct order of operations now that i've introduced a new parity drive effectively....

 

Thanks in advance

 

i'm running Unraid 6.11.1

 

i'll attach the diagnostics

mediaserver-diagnostics-20240118-1707.zip

here's the smart report download for the disk in question.

mediaserver-smart-20240118-1750.zip

Diagnostics already includes SMART for all attached disks. I encourage everyone to examine their diagnostics.

 

Why do you have ReiserFS on any of your disks? Have you been using Unraid since the V5 days?

Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk1 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk3 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk4 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk5 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk6 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk7 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk9 reiserfs

You will have to convert them to something else.

 

 

 

 

 

You should click on each of your WD disks and add attributes 1 and 200 for monitoring.

 

Disk10 SMART has

ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    124
197 Current_Pending_Sector  -O--CK   200   200   000    -    2

and syslog has

Jan 18 16:52:35 mediaserver kernel: critical medium error, dev sdo, sector 125053640 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0

and more of the same

 

SMART for other disks look OK, but

Jan 18 16:52:35 mediaserver kernel: md: recovery thread: multiple disk errors, sector=125053576

and more of the same so maybe a bad connection somewhere in addition to the disk10 problem.

 

 

1 hour ago, yessi_r said:

do i stop this parity rebuild on the new parity disk, put back in the original parity drive, and replace the bad drive?

Have you written anything to the array since you replaced parity?

You have nothing assigned as disk2, is that correct?

 

Many of your disks are too full. You should always keep some free space in case filesystem repair is needed.

 

Also, that particular old version of Unraid you are running has a bug in diagnostics collection for your User Shares.

 

You should disable Docker and VM Manager in Settings, and don't write anything to your server until your array is stable again.

1 hour ago, trurl said:

Diagnostics already includes SMART for all attached disks. I encourage everyone to examine their diagnostics.

 

Why do you have ReiserFS on any of your disks? Have you been using Unraid since the V5 days?

Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk1 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk3 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk4 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk5 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk6 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk7 reiserfs
Jan 18 16:51:24 mediaserver root: Fix Common Problems: Warning: ResierFS on disk9 reiserfs

You will have to convert them to something else.

 

I've been using unraid since 2012, I assume that was around the V5 days, I saw the notification about the phaseout in 2025 so that's on my to do list for sure. i've already started converting some

1 hour ago, trurl said:

You should click on each of your WD disks and add attributes 1 and 200 for monitoring.

 

Disk10 SMART has

ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    124
197 Current_Pending_Sector  -O--CK   200   200   000    -    2

and syslog has

Jan 18 16:52:35 mediaserver kernel: critical medium error, dev sdo, sector 125053640 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0

and more of the same

 

SMART for other disks look OK, but

Jan 18 16:52:35 mediaserver kernel: md: recovery thread: multiple disk errors, sector=125053576

and more of the same so maybe a bad connection somewhere in addition to the disk10 problem.

 

 

Have you written anything to the array since you replaced parity?

I'll be sure to look into how to add those (smart attribute notification's custom attributes field?) I'll see if i can reseat everything as well. though they're all in supermicro hot swap bays, but i'll check the individual connections. i haven't seen any new errors for that drive since a few min after the parity sync started for the new parity drive and i'm currently about 4.5hrs in. though my dockers are running

 

i don't believe i've written anything new to the actual array as i keep all my docker data on the cache, just phyical files should move once the mover initiates

1 hour ago, trurl said:

You have nothing assigned as disk2, is that correct?

 

Many of your disks are too full. You should always keep some free space in case filesystem repair is needed.

 

Also, that particular old version of Unraid you are running has a bug in diagnostics collection for your User Shares.

 

You should disable Docker and VM Manager in Settings, and don't write anything to your server until your array is stable again.

yes correct no disk currently in disk2 the plan was to move my old parity drive to that spot since it's larger than what was there previously and since that was one of the old reiserfs drives i converted to xfs recently. my disk are definitely pretty full, i try to keep enough space for the largest single file size i can see myself having as i heard that was the recommendation, i've disabled docker, do i need to disable vm manager as well? I don't have anything for my VM pointed to my array, but if there's something i'm not aware of i'll make sure to disable that as well.

with that said, what do i do now that i've disabled everything and haven't written anything to my array (that i'm aware of, space available is the same as it was prior to this swap)?

 

do i let the new parity drive rebuild finish? or do i revert back to my old parity drive and insert a replacement drive? 

You can quote a specific part of a post by selecting that part and clicking Quote selection on the popup, like this

11 hours ago, yessi_r said:

what do i do now

so you don't have to quote the entire post, and then bury your replies within the quote.

 

Run an extended SMART self-test on disk10.

3 hours ago, trurl said:

Run an extended SMART self-test on disk10.

Do I need to pause the parity sync and then run this test?

2 hours ago, yessi_r said:

Do I need to pause the parity sync and then run this test?

yes

1 hour ago, trurl said:

yes

Yep. Looks like disk 10 failed the extended test pretty much immediately. Similar to when the error popped up as soon as the parity sync started.

Do you have a new drive you can rebuild disk10 on?

4 hours ago, trurl said:

Do you have a new drive you can rebuild disk10 on?

yes.

On 1/18/2024 at 10:18 PM, yessi_r said:

do i revert back to my old parity drive and insert a replacement drive?

The replacement for disk10 must be at least as large as disk10, but cannot be larger than your old parity drive.

 

Is all of that true?

 

15 hours ago, trurl said:

The replacement for disk10 must be at least as large as disk10, but cannot be larger than your old parity drive.

 

Is all of that true?

 

yes, i've done drive replacements before, i've just never had to deal with a parity rebuild having an error while swapping parity drives. it's all sorted now. replacement drive rebuild completed with no errors.

 

greatly appreciate the help you provided!