I am relatively new to this forum. Administrators, please let me know if I am violating any rules and/or posting in the wrong section.

I hope you can help me with a performance issue. I will try to describe it as clearly as possible:
 

I have a ProLiant DL380 Gen9 and a ProLiant ML310e Gen8.2. These two servers are directly connected to each other via two 10GB network cards.

Until recently, I was running TRUENAS Scale on both servers. Using the built-in Data Replication, I created backups from the DL380 to the ML310e. I achieved an almost constant speed of 5GB per second.

I have now installed UNRAID 7.0.1 on the DL380, while the other still runs TrueNAS Scale. The UNRAID array consists of 14 hard disks of 8TB each, formatted with the XFS file system, with one parity disk and two SSD cache drives (mirrored) using BTRFS.

I mount the data on the other server via an SMB share, and using Rsync (user script), I am now transferring the data back from the ML310e to the DL380. When I configure the cache as the first drive and the array as the second for the relevant share, I achieve a transfer speed of 2GB per second max. When the cache gets full, the speed collapses. I am using CA Mover Tuning, and I’ve set it to write from cache to array when the cache exceeds 50% capacity.

If I don’t use the cache and have construct write enabled, I barely reach 1GB per second transfer speed.

I changed the network settings for the relevant port from 1500 MTU to 9000 MTU, but this did not bring significant improvement.
 

Does anyone have suggestions for what I could try to improve the speed? Could it be related to the Rsync protocol?

Edited March 27, 20251 yr by Gerco

48 minutes ago, Gerco said:

posting in the wrong section

You posted in the Multi-language section. I have moved this to General Support

49 minutes ago, Gerco said:

I am using CA Mover Tuning, and I’ve set it to write from cache to array when the cache exceeds 50% capacity.

Mover is intended for idle time. It is impossible to move from fast cache to slow array as fast as you can write to fast cache. If you need to write more at one time than cache can hold, don't cache.

 

In addition to not caching initial data load, you can go without parity then build it after initial load completed.

2 hours ago, Gerco said:

If I don’t use the cache and have construct write enabled, I barely reach 1GB per second transfer speed.

I am afraid this is about all you can expect if writing to the Unraid style parity protected array.   To understand why it can be worth reading this section of the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page. The Unraid type of array is optimised for things other than speed as it is typically used for WORM (Write Once, Read Multiple) type access such as media files.
 

You can get better performance if you use a pool for the storage, and for that you have a choice of BTRFS or ZFS type pools (and you can have multiple pools if wanted).  With Unraid 7 the Unraid parity protected array is now optional so you can if you want just have a mixture of pools to meet your needs.

 

in your case I would have suggested that you do the initial data load onto the Unraid type array without a parity drive assigned and without caching as then you avoid the overheads of updating parity in realtime, although you would still be limited by the write speed of individual drives as Unraid does not use striping on that type of array. The parity could then be added afterwards.   For normal running you could then add caching to get better perceived write performance as long as the data does not exceed the size of the pool being used for caching.  The mover would then typically transfer the data overnight (relatively slowly) when the system is otherwise idle.   

@itimpi

Thank you. This is very helpful.

I have a full copy of my data, I could start over again. 

 

I wouldn't say my use is WORM but more like Write more or lease regularly Read Many. I used ZFS in TrueNas scale; the performance was good, as mentioned earlier, but extending the pool was a drag. Extending a pool in BTFRS is easy; ZFS requires adding a new Vdev with the same number of disks. That's why I am reluctant to use ZFS. 

I was unaware that Mover only becomes active when the server has little to do, but I had indeed already noticed that moving cache to array was not very fast.

Edited March 27, 20251 yr by Gerco
Went through ZFS and BTFRS docs

12 minutes ago, Gerco said:

I was unaware that Mover only becomes active when the server has little to do, but I had indeed already noticed that moving cache to array was not very fast.

Mover cannot work any faster than writing directly to the array.    It is normally scheduled to run in the middle of the night.  It is aimed at the Use Case where you write a limited amount of data every day so that the cache pool never fills up completely before mover can kick in.

 

14 minutes ago, Gerco said:

If BTRFS and/or ZFS pools are easy to extend, also uneven-sized drives,

BTRFS pools are easy to extend and can contain mixed size drives.    ZFS pools are more restrictive although they are probably better performance wise and more robust against hardware issues.

@itimpi

Thank you once more. 🙂 

It'll be a toss-up between flexibility and less robustness (BTFRS) and performance, robustness, but less flexible (ZFS).

 

I'll sit back with a glass of wine to think about this.