After removal of BTRFS cache pool i can't get above 110mb/s to unraid cache from clients

[je82]

I have a very strange issue with my unraid installation.

 

I previously ran 2x 1tb ssd in a btrfs raid partition as a cache for 3 years and never had any speed problems but i wanted to have the volume encrypted and also save 1x ssd so i changed this pool to a single 1tb ssd with xfs encrypted instead.
 

Everything works fine, here is the hickup.

 

Network speed from outside rack to clients = 2500mbit

Network speed from inside rack between servers = 10000mbit

 

Quote

 

TEST 1 : From client outside rack to unraid inside rack:

 

Sending files over network directly to unraid cache over smb = 110mb/s

Receiving files over network directly from unraid cache over smb = 250mb/s

 

 

Next test, setting up a virtual machine inside unraid with a smb share on the same nic/network and try the speeds:

 

Quote

 

TEST 2 : From client outside rack to unraid virtual machine inside rack:

 

Sending files over network to smb share on virtual machine hosted by unraid = 250mb/s

Receiving files over network from smb share on virtual machine hosted by unraid = 250mb/s

 

Third and final test is to send a file from another server inside the rack that has a 10gbit connection directly to the unraid cache.

 

Quote

 

TEST 3 : From client inside rack to unraid cache inside rack:

 

Sending files from 10gbit connected server inside rack directly to unraid cache = 110mb/s

Receiving files from 10gbit connected server inside rack directly from unraid cache = 870mb/s

 

 

As you can see clearly in the last test, something is very off, even on 10gbit connectivity i am getting 110mb/s when sending to the cache, but receiving i am getting 10gbit speeds.

 

The issue started the same day i removed my btrfs cache raid partition and changed it into a xfs encrypted. I have already swapped the SSD to a brand new one because i intially though this issue must have been due to 3 years of wear and then xfs encryption on top of that was causing weird slowdowns, brand new ssd, exact same issue.

 

Edited September 27, 2022 by je82

[je82]

Interesting read here:

Is unraid just that slow with xfs encrypted? It seems like a crazy performance hit, i have been running my array encrypted for years but i never write directly to the array (the mover does that at night) so the 110mb/s limiter that may be in place somewhere isn't a big problem there, but its definitely a huge issue if on the cache.

 

Is there a limit hardcoded into unraid how fast it can write to a xfs encrypted volume?

 

Edited September 27, 2022 by je82

[je82]

Think i answered my own question, i just expected it to be faster, my cpu literally sleeps while writing at 110mb/s to the xfs encrypted cache volume so to me it seems like there is a hardcoded limitation either inside the xfs encrypted protocol or inside unraid for whatever reason.