Okay this is going to be a long story . So I started the upgrade to ESXi and a virtualised server.

 

Original setup:

unRaid 4.7

AMD Sempron 140 with Asus M4AGTD PRO, 4GB ram

14 disks + parity: mix of 2tb WD, hitachi and 1.5tb samsung, all slow spinners.

650W Corsair TX

Supermicro AOC SASLP MV8 x2

SIL 3123 2 port PCIe

Parity checks used to take about 7-8 hours.

 

I upgraded and added new disks, 6 new 2TB WD and Hitachi green drives, more of the above basically. Swapped the controllers for some IBM M1015s.

unRaid 5.04

Asrock 970  Extreme4, AMD FX 8320, 32GB ram

IBM M1015 x2

SIL 3123 2 port PCIe

Parity check now taking 12 hours

 

In between the two I removed a lot of plugins, Ran everything through ESXi which worked fine but then started getting red balls on random disks and hangups requiring a hard power down. The drives used to be either the ones on the SIL controller and/or the motherboard. I suspect either the SIL controller passthrough or most likely an underpowered PSU. First time I ran unraid bare metal and everything came back good, parity check ran all good as well but took about 12 hours.

 

Second time back in ESXi same thing again so I've gone back to a bare metal unraid but this time it came back as parity invalid, so I rebuilt parity (I know I could have just trusted parity). I then rechecked parity and also took about 12 hours and all good again.

 

I've removed two empty disks from the array and clicked trust parity and started a check with no correct again (coming up with some 149 errors which I'm not sure about as the drives were empty) I'll parity check again and correct parity later but so far it is also looking like 12 hours.

 

Is this normal behaviour? I've searched and there is mention that adding more disks will slow things down a bit but by how much? More than 50%?

Also the IBM M1015's are supposed to be faster, PCIe x8 as opposed to the Supermicros which where PCIe x4.

The only differences (assuming motherboard, CPU and RAM makes little difference) is v5.04, more dirves, different controller.

 

I plan on running the tunables.sh later but would like to try it when I'm back in ESXi with less RAM an when I have a new PSU to check the controllers aren't dropping red balls due to lack of power.

 

Thanks Josh

First, you should not "Trust Parity" when you've actually removed some disks !!

An "empty" disk is NOT all zeroes ... so every non-zero bit will result in a parity sync error.

 

If you do what you did, you need to run a correcting parity check immediately after, so you'll have good parity for the remaining array disks.

 

As for the speeds ... I'm not surprised at a modest increase, but I think 50% is excessive.  Something has changed that's causing that ... the trick is to isolate it one step at-a-time.

This could be a controller difference; a cable; or a bandwidth issue.

 

The first thing I'd do is remove half of your memory ... i.e. only populate two of your slots.    This significantly reduces the loading on the memory bus, and can make a notable difference in memory performance on unbuffered systems.  If you've encountered a memory bandwidth issue, this will likely resolve it.    Whether it helps or not with this specific issue; I'd leave it that way, as it will improve the reliability of your memory.

 

Next, I'd revert to your original set of disks, so the only difference in the system is the changed controllers.    See what your parity sync/parity check speed is with that configuration.  See if it reverts to the previous speeds.    The answer to this will help focus on what you need to try next.

 

Thanks, correcting parity check was on the list to do after this non correcting one has run. Maybe I'm too over cautious but I like to see what needs to correct before doing something irreversible.

I'll remove more disks back to the previous setup and I can try the memory but the plan is still to virtualised at some point.

I'll test with less disks and get back.

Josh

 

Sent by tapatalk

 

 

Okay just for information but I've cut back to the original drives and parity check was still slow but then putting it back into ESXi with 4GB ram has sped it up back to 8 hours which I'm happy with.

 

Thanks josh

So this runs faster in ESXi than with bare metal ?? !!

(Using the exact same hardware)

 

It did but only because I limited it to 4gb ram. Bare metal was still 32gb and it was a rebuild. Previous setup, bare metal was probably a fraction quicker. so assuming controllers didn't make a big difference ESXi just a fraction slower.

Josh

 

Sent by tapatalk