UNRAID slow on writing, WHY! (2)

[grumpy]

Original post

 

Not sure with almost 24 hours and 70 views with no response if others are having the same issue as me not seeing the text unless you click the button Reveal hidden contents. I have no idea why it is hidden.  

 

But here is the text portion of the post.

 

I have just discovered that my network slowdown is my Unraid. (I have been ignoring for long time)

Using BlackMagic Speed test between local and the 2 NAS's really shows the poor performance from UNRAID to the point I'm reconsidering my decision to continue with Unraid. Just put in a second Synology and be done with it.

 

But first I thought I would ask for help/pointers in correcting this major slowdown.

 

There are 4 images.

Local PC Samsung 860 EVO M.2 2TB, Write 478MB - Read 520MB

Synology populated with 8 Iron Wolf 10 TB, Write 564MB - Read 602MB

Unriad share with cache and share Write 109MB - Read 588MB

Unraid without cache (NC), Write 14MB - Read 598MB

4 (1 as Parity) Seagate 16TB HDD Exos X16 7200 RPM 512e/4Kn SATA 6Gb/s 256MB Cache 3.5-Inch Enterprise Hard Drive (ST16000NM001G)

Cache Drive Western Digital 4TB WD Red SA500 NAS 3D NAND Internal SSD

 

The Synology is extremely close to the local speed (beating it)

But the Unraid is extremely slow

Iperf3 is about 7 GBs to Unraid ; not optimum but not the problem.

 

 

 

Unraid system:    Unraid server Pro, version 6.11.5
Motherboard:    Dell Inc. 002KVM, Version: A01, s/n: /GH1V9Z2/CNFCW0098C00GC/
Processor:    Intel® Xeon® W-2155 CPU @ 3.30GHz
HVM:    Enabled
IOMMU:    Enabled
Cache:    L1-Cache = 0 KiB (max. capacity 0 KiB)    
    L2-Cache = 0 KiB (max. capacity 0 KiB)    
    L3-Cache = 0 KiB (max. capacity 0 KiB)    
Memory:    32 GiB DDR4 Single-bit ECC (max. installable capacity 3072 GiB)
    DIMM1: Hynix HMA82GR7JJR8N-VK, 16 GiB DDR4 @ 2666 MT/s
    DIMM2: Hynix HMA82GR7JJR8N-VK, 16 GiB DDR4 @ 2666 MT/s
Network:    bond0: fault-tolerance (active-backup)
    eth0: 10000 Mbps, full duplex, mtu 1500
    eth1: interface down
Kernel:    Linux 5.19.17-Unraid x86_64
OpenSSL:    1.1.1s
P + Q algorithm:    23849 MB/s + 44160 MB/s

 

[Squid]

You need to post your diagnostics

 

[grumpy]

Diagnostics File

 

tower-diagnostics-20230721-1452.zip

[grumpy]

Nobody have any ideas/suggestions?

[Tzekang]

In my case, i have a speed limit issue when added a parity drive.

 

After parity drive added, my write speed was capped at 70mb/s. So now in my use case, i would make the share to use cache, then move to array. 

[grumpy]

@Tzekang You are correct that there is a LARGE penalty for using parity, and of course the cache drive will improve transfer speed up to a point. But I just did another test between PC, Synology and UNRAID (with and without cache) and I'm still not satisfied with the results. Instead of using a benchmark that I do not understand I went with a real world copy and paste. A directory containing 12 video clips 4K mp4. And the results were better but Synology (with no cache, weak cpu, less memory) still kicked Unraid's butt with and without cache.

 

The whole reason to use Unraid over Synology is power. (Yea for some it is dollars) Any CPU any motherboard allows you to do more than just a NAS, which was my use case. But the NAS part is not performing up to par, read and write speeds are below my needs/expectations.

If Synology and maybe Qnap can do it, why not Unraid?

 

I thought of changing hardware, but in reality the Xeon CPU rarely goes over 10%, 32 GIG of ram is more than enough and the 7200RPM enterprise class HD are fast enough. The problem is software, somewhere there is a bottle neck. In the short term I will enable cache (4TB SSD) on my Media share and get some improvement. I just haven't done that as SSDs have a habit of no warning dying and I download a bunch of ahhh! ISO's.

 

Maybe I should put a cache drive in the Synology just to see what it really can do, but the CPU, memory is it weak point. The Synology just mirror the Unraid data currently.

 

[Squid]

2 hours ago, Tzekang said:

In my case, i have a speed limit issue when added a parity drive.

 

After parity drive added, my write speed was capped at 70mb/s. So now in my use case, i would make the share to use cache, then move to array. 

It's not capped, but rather a trade-off of how the system works.  With Unraid set to use Read/Modify/Write in Disk Settings, write speeds take a hit but your power savings are better as only the drive involved in the write and parity drive are involved.  With Reconstruct Write enabled, there is no degradation in speed at the expense that every drive in the array has to spin up to write anything to any drive.  Cache Pools and Use Pool settings for the share are the usual method for combining the best of both worlds.

 

3 hours ago, grumpy said:

Nobody have any ideas/suggestions?

The best user for advice on this and any tips / tweaks is currently on vacation.

[JorgeB]

What speed to you get writing directly to the array with turbo write enabled?

[grumpy]

@JorgeB

 

Unraid without cache (NC), Write 14MB - Read 598MB is the speed directly to the array.  (Which would be 1 drive bypassing cache but not parity)

But I do not know what or how to enable turbo speed!

[JorgeB]

Enable turbo write and re-test writing directly to the array.

[grumpy]

@JorgeB

I marked it as a solution as this maybe the best I can expect from Unraid. Synology is still faster but they write to all disks sharing the love evenly. I have no cache in the Synology.

 

Test:

151 GB dir with 10 video files is my test now.

Before turbo write MB/s

180 read 75 write

After turbo write

230 read 220 write

Cached enabled (4 TB SSD)

490 Read 380 Write

 

Iperf-3 7.79 Gbits/second

Edited July 31, 2023 by grumpy

[JorgeB]

27 minutes ago, grumpy said:

After turbo write

230 read 220 write

This is normal for an unRAID array, data isn't striped, so it's limited to the speed of your slowest disk at the position, for better performance you can create for example a zfs raidz pool, of course it also has some disadvantages, like not easy expansion for example.