Hi all, 

 

I'm experiencing a really slow writing speed on my Seagate IRONWOLF PRO 20TB (ST20000NT001) disks: 
 

# dd if=/dev/zero of=/mnt/disk1/test  bs=128k count=10k
10240+0 records in
10240+0 records out
1342177280 bytes (1.3 GB, 1.2 GiB) copied, 54.2975 s, 24.7 MB/s

 

with old western digital red 8tb disk (WDC_WD80EFZX) and 8tb seagate (ST8000VN0002) all seems fine:

 

dd if=/dev/zero of=/mnt/disk4/test  bs=128k count=50k conv=fdatasync
51200+0 records in
51200+0 records out
6710886400 bytes (6.7 GB, 6.2 GiB) copied, 4.56653 s, 1.5 GB/s

 

all disks have the same FS (ZFS), and the two old (but well working) disks have some SMART errors too. 

littleboy-diagnostics-20231113-1124.zip

Edited November 13, 20232 yr by skler

Wait for the parity sync to finish, then enable turbo write and re-test.

the parity sync requires 20 days, it's going at 10mb/s 😞  I've paused it 

Edited November 13, 20232 yr by skler

3 minutes ago, skler said:

requires 20 days

That suggests another problem, or the array is being used, but you cannot test speeds with parity invalid, you can remove parity if want to test before.

works well without parity 

Add parity back to start the parity sync, make sure no other array activity is going on, and if it's still slow post new diags.

ZFS with Parity Disk? A killer as we know already!

What he sees is just the "these things do not go well with each other" thing.

 

On 11/13/2023 at 2:39 PM, JorgeB said:

Add parity back to start the parity sync, make sure no other array activity is going on, and if it's still slow post new diags.

 

Installed all disks, activated parity again and is all slow again 😕 

 

I did:

- a preclear of disks with an average speed of 180/200mb/s

- writing to cache is really fast (ZFS raid0)

- writing on a single disk (excluded from the array) seems good (ST8000VN0002 (sdo) from diagnostic) 

 

When I write into the array (with or without  turbo mode) I'm on average speed of 25mb/s 

 

dd test is not accurate, I've activated a lot of cache on ZFS

 

cat /boot/config/zfs.conf
# limetech - Default ARC size is MemTotal/8 (in bytes)
# I set 128GB
options zfs zfs_arc_max=128000000000

 

 

littleboy-diagnostics-20231213-1005.zip

There's something writing to disks 3, 5, 6 and 7, turbo write will not work with multiple disk writes, are you using most free as the allocation method?

When diagnostic was done there were:

- The mover running, moving from cache to media share in disks 1,2,4,5,6 (media in with high-water conf, screenshot attached) 

- GUI Moving files, from disk3 to cache (with acceptable performance of 100mb/s)

- The TimeMachine backup on disk 7, now stopped (sorry I didn't saw it)

 

I can't turn off the mover. But the problem is that if I do 2 simple mv, when the mover doesn't runs es:

mv /mnt/disk1/mybigfile.iso /mnt/disk3/

mv /mnt/disk2/mybigfile.iso /mnt/disk4/

 

The parity disk (sdc) has a maximum of write speed of 25-30mb/s.

 

I don't have any special io-wait, maybe this can help:

 

 

littleboy-diagnostics-20231213-1219.zip

Edited December 13, 20232 yr by skler

Any time there are writes going on to 2 or more disks turbo write won't work and write speed will be much slower because parity is being updated at the same time for all disks, you can only test turbo write if writing to a single disk.

18 minutes ago, skler said:

When diagnostic was done there were:

 

As usual with HDDs you never want to do 2 things at the same time, it'll always be slower than doing them one after the other.

 

19 minutes ago, skler said:

mv /mnt/disk1/mybigfile.iso /mnt/disk3/

mv /mnt/disk2/mybigfile.iso /mnt/disk4/

 

Without parity that wouldn't be a problem since each drive is only doing one thing, but here parity needs to be written in parallel for each, so that's 2 simultaneous operations.

41 minutes ago, JorgeB said:

Any time there are writes going on to 2 or more disks turbo write won't work and write speed will be much slower because parity is being updated at the same time for all disks, you can only test turbo write if writing to a single disk.

 

you know what you do. 

 

thanks! 

it is an acceptable result, but there is something I can do to improve it? the disk can write at 230mb/s 

Run the diskspeed docker controller test to check for any bottlenecks.