Parity-Sync stops after 1.5% and never progresses

[RWayneRoss]

I just built my first Unraid server and am having a hard time tracking down the problem. I created a new array (six 14Tb drives) and have a 14Tb parity disk. The sync starts and within 4-5 minutes it has gone from 0.1% to 1.5% and it just stop progressing. Unraid shows it's still running but the only thing that changes is the estimated completion time, it has gone from 15 hours to 533 days.

I've done a full memory check (32Gb), letting it run overnight and had 4 full passes with no errors. Everything is brand new, even the drives.

Any help would be appreciated, here's the syslog, it seems to start having problems at about line 2773.

syslog.zip

[JorgeB]

Unraid driver is crashing, this appears to sometimes happen with some hardware/kernel combinations, try updating to v6.11.0-rc4, very different kernel might help.

[RWayneRoss]

Thank you @JorgeB, that has certainly helped. I've made it up to 3.5% so far. If it completes I'll mark your reply as the solution. It'll be wonderful to have a working server after trying to get it working for so long!

[RWayneRoss]

No success. After the upgrade to v6.11.0-rc4 the parity creation made it to 10.1% before dying again. Syslog attached, look like a GPF near line 1365.

 

home-server-syslog-20220826-1708.zip

[JorgeB]

Is this known good hardware, i.e., was it working reliably with a different OS before? If yes try v6.9, if not run memtest.

[bonienl]

Are you sure your PSU is up to the task?

A parity operation involves all disks and drains a lot more power than average usage.

 

[trurl]

Please post diagnostics instead of just syslog

[RWayneRoss]

4 minutes ago, trurl said:

Please post diagnostics instead of just syslog

Sorry, I'm new to unRAID, never thought of that. Here they are...

home-server-diagnostics-20220826-1525.zip

[trurl]

Please start the array and post new diagnostics

[RWayneRoss]

9 hours ago, trurl said:

Please start the array and post new diagnostics

Here they are. JorgeB suggested I try 6.9, I'll try that shortly as well.

home-server-diagnostics-20220827-0645.zip

[JorgeB]

Btrfs is already detecting some data corruption, for a new server not a good sign, start by running memtest.

[RWayneRoss]

44 minutes ago, JorgeB said:

Btrfs is already detecting some data corruption, for a new server not a good sign, start by running memtest.

I ran it overnight a couple of days ago. It ran 4 full passes on 32Gb with 0 errors.

[RWayneRoss]

I'm still on 6.11.0-rc4 and now I'm getting 100% CPU pegging on 2 of the 8 cores.

[JorgeB]

Try v6.9, but that and the fact that the Unraid driver is crashing with both v6.10 and v6.11 makes me suspect a hardware issue.

[RWayneRoss]

18 hours ago, bonienl said:

Are you sure your PSU is up to the task?

A parity operation involves all disks and drains a lot more power than average usage.

 

Yes, I just checked... it's a Corsair HX1050, 1,050 Watts.

[RWayneRoss]

Just downgraded to 6.9.2. I'll post results when I have some.

[RWayneRoss]

Version 6.9.2 did not help, in fact, made it worse. It ran for 18 minutes and I got:

 

Aug 27 08:24:31 Home-Server kernel: traps: emhttpd[2500] general protection fault ip:14ab5d246554 sp:14ab5cb85be8 error:0 in libc-2.30.so[14ab5d1e1000+16b000]

 

At this point I think it's a hardware incommutability.

[trurl]

Still seems like RAM. Try the new one at memtest86.com

[JorgeB]

19 hours ago, RWayneRoss said:

At this point I think it's a hardware incommutability.

Unlikely that it would be incompatible with 3 very different kernels.

 

[RWayneRoss]

19 hours ago, trurl said:

Still seems like RAM. Try the new one at memtest86.com

I downloaded the latest version at memtest86.com and ran the full test... 4 passes and zero errors. 

[RWayneRoss]

2 hours ago, JorgeB said:

Unlikely that it would be incompatible with 3 very different kernels.

 

Thanks @JorgeB. From what I've seen in the logs it always seems to be a GPF. If the memory has tested good, what else might cause those?

I do have (2) LSI 9207-8i controllers, could it be a driver problem with those? I'm pretty sure I have some other hardware RAID cards that can be set to JBOD, perhaps I should try those next?

Edit: I could also connect a few drives to the SATA ports on the motherboard and try to make a parity set.

 

Edited August 28, 2022 by RWayneRoss

[JorgeB]

16 minutes ago, RWayneRoss said:

I do have (2) LSI 9207-8i controllers, could it be a driver problem with those?

Don't think so, those are very common with Unraid, I have some myself.

 

Do you have another board/CPU combo you could test with?

[RWayneRoss]

2 minutes ago, JorgeB said:

Don't think so, those are very common with Unraid, I have some myself.

 

Do you have another board/CPU combo you could test with?

I do. It's a bit older than this setup, which might be good in this case. I'll put it together and give it a try. It'll take a day or more as I'll be tied up most of today with other duties.