Internal transfer speed seems slow. Thoughts?

[ksignorini]

I'm rsync-ing a 20GB VM image from my SSD cache (SATA III) to my array (this disk is on a SATA I controller) through the share location (/mnt/user/domains archive) for the receiving drive directory.

 

With --progress turned on I'm only getting 25-30MB/s speeds.

 

Does this make sense?

[1812]

A little low. You could enable turbo write to try to help with that. 

[BRiT]

Depends on your hardware.

[trurl]

Seems to me you are doing the User Share Copy Bug, since /mnt/cache/domains is part of the user share at /mnt/user/domains.

[ksignorini]

So stick to copying from user share to user share, rather than disk location to user share or user share to disk location, even at the command prompt? (/mnt/user/XXXX... to /mnt/user/YYYY...)

 

Or is disk to disk location better? (/mnt/diskX/dirXXX... to /mnt/diskY/dirYYY...)

 

(From what I gather at the linked post.)

 

(Oh, and nothing seems to be clobbered. The files are there after any of my copies, the correct sizes, etc.)

 

Correct?

Edited February 27, 2017 by ksignorini

[gubbgnutten]

1 hour ago, ksignorini said:

So stick to copying from user share to user share, rather than disk location to user share or user share to disk location, even at the command prompt? (/mnt/user/XXXX... to /mnt/user/YYYY...)

 

Especially at the command prompt! But yes, to stay safe - stick to user share to user share and/or disk to disk and never mix user and disk.

 

2 hours ago, trurl said:

Seems to me you are doing the User Share Copy Bug, since /mnt/cache/domains is part of the user share at /mnt/user/domains.

Sure, but it is not part of "/mnt/user/domains archive" so it is not triggered by this transfer. Still best to point out the risk of mixing user shares/disk shares anyway, of course.

[ksignorini]

Just now, gubbgnutten said:

Especially at the command prompt! But yes, to stay safe - stick to user share to user share and/or disk to disk and never mix user and disk.

 

Sure, but it is not part of "/mnt/user/domains archive" so it is not triggered by this transfer. Still best to point out the risk of mixing user shares/disk shares anyway, of course.

 

Yeah, I kind of figured this didn't apply, however, it's good to know. I'll stick to disk-to-disk and user-to-user share copies from now on. Perhaps a warning about this was noted somewhere, but I must have missed it.

 

Thanks!

[ksignorini]

2 hours ago, BRiT said:

Depends on your hardware.

 

How so? And is there official Turbo Write documentation?

[1812]

10 hours ago, ksignorini said:

 

How so? And is there official Turbo Write documentation?

 

https://forums.lime-technology.com/search/?type=all&q=turbo+write

[BRiT]

11 hours ago, ksignorini said:

 

How so? And is there official Turbo Write documentation?

 

Are your hard drives 5400rpm? 5900rpm? 7200rpm? Are they older 1TB? 2TB? 3TB? 4TB? 5TB? 6TB? 8TB? 10TB? Are they Archival Drives? PMR? SMR? What is the Areal Density? Are they connected via Sata I? Sata II? Sata III? Are they connected over PCI Controller? PCI-Express? Motherboard via DMI I? DMI II? Do you have Dual Parity? How old is the CPU?

 

TL,DR; Depends on your Hardware.

Edited February 28, 2017 by BRiT

[ksignorini]

15 minutes ago, BRiT said:

TL,DR; Depends on your Hardware.

 

Fair enough. All 7200 RPM, SATA III except the one in the example above, that I'm copying to.  It's SATA I (but I said that in OP, I think). It is, however, on PCI (not mobo, not PCIe). The drive copying from is SSD cache on SATA III. Single parity. RAID 1 cache.

 

So yeah, slower hardware connection. But even SATA I is 150 MB/s. 20-30 MB/s is a far cry from that. Heck, it's a far cry from half of that.

 

This is an AMD FX-8320 system with 16GB DDR3 RAM.

Edited February 28, 2017 by ksignorini

[BRiT]

Your maximum upper bound is much lower than Sata 1 sspeeds. PCI has a theoretical maximum speed of 133 MB/s shared across ALL DEVICES on that bus. Actual throughout will be slightly lower, more around 80% of the theoretical maximum. If you have 2 drives on PCI such as a Data Drive and the Parity Drive your absolute fastest speed for writes would be 66 MB/s.  Since writing to drives in unraid requires 4 operations for every write to the array (read old data, read old parity, write new data, write new parity) rotational latency of the drives also come into play.

 

If you're also writing to 2 other data drives at the same time, then using 4 drives on PCI at same time has theoretical maximum speed of 33 MB/s. Actual throughput would be closer to 24-26 MB/s.

 

If you have Dual Parity with Parity and Data on PCI then your maximum throughput is 44 MB/s. Also Dual Parity is very cpu intensive on older cpu systems.

 

Move all drives off PCI for speed gains.

 

 

Wow, I havent had to explain how horrible PCI bus is in ages, it feels like 8 years, back when unraid 4.5 was the current version.

[ksignorini]

Only one of the drives is PCI. There are two on PCIe (SATA III) and six on mobo SATA III. Until I can afford much larger drives, I need to keep these in place.

 

I could go without the one on PCI if I had to, but I don't use it for much other than ISO's and backups of VM's.

 

I'm going to do some testing (similar copy operation) but stick to the on-board and PCIe drives to see how they compare. Thanks for the great explanation, BRiT.