Writes capped to 60mb/s even with reconstruction writes enabled

[robinkoehler]

Hi all, 

 

I've spent the weekend trying to troubleshoot this, sorry if I'm missing something obvious. My array has parity enabled, and writing to the array is capped to around 60mb/s. I have gigabit ethernet, the connection between PC and unraid is confirmed to work (and I can write to RAM at full 115mb/s until RAM is filled and it drops to 60mb/s). I am not using a cache pool.

 

• If I disable parity, the array writes at full 115mb/s as well 
• I have reconstruction writes enabled for the share, and tried user share vs disk share as well
• I have tried high water, fill up and most free with the same result
• I have tried the TipsAndTweaks dirty ratio % set to 1% with the same result
• I have confirmed that write caching is enabled on all drives (they are SATA, but I still double checked)
• All disks are spinning. Parity sync can run at full disk speed (185+mb/s for all disks read, and 185+mb/s write on the parity disk)
• All disks are identical (4TB Seagate Ironwolf). I have 5 data disks and 1 parity disk in the array
• I have tried different test file sizes which are all larger than system RAM
• I have tried different computers on the network

 

Am I not supposed to get almost full write speed of the drives (or at least 1GB ethernet saturation in my case) with turbo writes enabled?

 

Again, sorry if I don't understand this correctly. I have searched the forum and googled everything I could think of in the last two days, and am at the end of my knowledge and skill level here.

 

Thank you kindly in advance for your help and guidance!

 

Robin

Edited November 20, 2022 by robinkoehler

[itimpi]

You will never get anywhere near the drives speeds when writing to the parity protected array.   You will find this section of the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page might give you an insight into why.   When the array does not have parity then the network speed (as you found) is more likely to be the limiting factor

 

I must admit though that I would expect something nearer 80MB/s unless something else (e.g. disk controller) is limiting throughput.   A general rule is that I would expect the write speed in Turbo mode to be similar to what you get from a parity check (as long as there is no other I/O going on at the same time).

[Vr2Io]

You have perform all necessary troubleshoot step and really clear and great.

 

Does CPU reach 100% at any core ?

 

Then pls attach diagnostic file.

Edited November 20, 2022 by Vr2Io

[robinkoehler]

3 minutes ago, itimpi said:

I must admit though that I would expect something nearer 80MB/s unless something else (e.g. disk controller) is limiting throughput.   A general rule is that I would expect the write speed in Turbo mode to be similar to what you get from a parity check (as long as there is no other I/O going on at the same time).

 

Thank you for the reply. I am running parity sync now at 196mb/s

[robinkoehler]

4 minutes ago, Vr2Io said:

Pls check does all disk write cache already enable in SMART.

Yes, if I run sdparm -g WCE /dev/sdd on each drive, they all return "1", indicating it's enabled on every drive

 

6 minutes ago, Vr2Io said:

Does CPU reach 100% at any core ?

Never more than 47% on any core during copy operation (Ryzen 2700x).

 

6 minutes ago, Vr2Io said:

Then pls attach diagnostic file.

 

I have attached it to this post

diagnostics-20221120-2151.zip

[itimpi]

Just now, robinkoehler said:

 

Thank you for the reply. I am running parity sync now at 196mb/s

Then I am surprised that the writing in Turbo mode was so slow   Are you sure that nothing else was trying to access the array as that would slow down the write speed.   
 

If you attach your system’s diagnostics zip file to your next post in this thread taken while you are getting the 60MB/s write speed it might allow somebody to give some useful feedback.

[robinkoehler]

1 minute ago, itimpi said:

Then I am surprised that the writing in Turbo mode was so slow   Are you sure that nothing else was trying to access the array as that would slow down the write speed.   
 

If you attach your system’s diagnostics zip file to your next post in this thread taken while you are getting the 60MB/s write speed it might allow somebody to give some useful feedback.

 

Thank you again! I have created a diagnostics file just now during a file copy operation to the array. Parity sync was disabled during the test. Please see attached:

 

diagnostics-20221120-2156.zip

[Vr2Io]

37 minutes ago, robinkoehler said:

All disks are identical (4TB Seagate Ironwolf). I have 5 data disks and 1 parity disk in the array

Also some comment on this, 4TB disk usually not high performance if compare to high capacity disk. If almost full and writing at inner track then speed will drop a lot. How about the usage of writing disk ? Would you posible found a less usage disk and perform same test and verify any difference. Pls use disk share for test.

 

I am at work now, once available will try to check diagnostic.

Edited November 20, 2022 by Vr2Io

[robinkoehler]

Just now, Vr2Io said:

Also some comment on this, 4TB disk usually not high performance if compare to high capacity disk. If almost full and writing at inner track then speed will drop a lot. How about the usage of writing disk ? Would you posible found a less usage disk and perform same test and verify any difference. Pls use disk share for test.

 

Thank you. The disks are still completely empty and I can write on the array in full speed of my network connection if parity is disabled. Parity sync is running at over 185 mb/s read for the data drives, and write for the parity drive.

[Vr2Io]

That's strange all show excellent state, but got slow writing speed.

[JorgeB]

11 minutes ago, robinkoehler said:

Parity sync was disabled during the test.

Parity is not yet synced, let it sync first, only after that will turbo write work correctly.

[robinkoehler]

Just now, JorgeB said:

Parity is not yet synced, let it sync first, only after that will turbo write work correctly.

 

Thank you Jorge, I'll let it finish overnight and report back (5hrs remaining)

[Vr2Io]

4 minutes ago, JorgeB said:

Parity is not yet synced, let it sync first, only after that will turbo write work correctly.

Never think that 👍

Edited November 20, 2022 by Vr2Io

[robinkoehler]

Hi Jorge, you have rescued me yet again! After the parity was completely synched, the array now happily writes at up to 150mb/s, which is higher than my network bandwidth and allows me to transfer files at the maximum speed my setup allows. 

 

Your help and knowledge is highly appreciated. 

 

Take care and best wishes, 

 

Robin