Network Speed Help

[johnmatt]

Hi all,

 

I have looked around the forums, as this is a question that comes up a lot - but I believe answers are all sort of specific to individuals networks/configurations.

 

I am getting less than ideal file transfer speeds.  I'm also fairly novice with Unraid (ie, might need specific instructions if you want terminal commands/reports)

 

I'll include a graph of my file transfer (from computer, to server).  It started at 110 mb/s and then dropped to ~40.  I did this the other day however, and it was ALL over the place - up and down and up and down, dropping as low as 12-14 mb/s?!?!

 

Anyways, hoping someone can help me diagnose!

 

THANK YOU in advance,

 

John

[itimpi]

The speed of 100MBps is roughly what you get when you are using a Gigabit network as that is the maximum the network supports.

 

a speed of ~40MBps is not atypical for writing directly to a parity protected array (i.e. to a share that is not set up to use a cache drive).    That is because the architecture of unRAID means each ‘write’ to an array drive in unRAID actually involves 2 reads and 2 writes.    You then need to take into account the linux Level caching which can mean writes vary between these extremes.

 

[johnmatt]

Ah.. ok, so do you think a cache SSD would resolve this issue?

 

What are your thoughts on the inconsistent (all over the place) results that happened the other day?  

 

Here's a graph of the writes... a little inconsistent, but probably not an issue?

[bonienl]

This is when I copy a 6GB file to my cache pool. It saturates my 1Gbps link.

 

[Frank1940]

Please understand that there is a lot more to copying a file to an array then just moving the bits across the network.  The OS has to decide which disk to store the file on, It then has to create the file, allocate the disk space, then move the file from the RAM buffer to the actual disk, updating the file allocation table as work proceeds and when the file is safely stored on the disk, the file has to closed.  If you have a gaggle of small-size files, all of this overhead can easily be more time than required to actually write the bits that consist of the data.  If you are moving 40GB blu-ray iso files, this overhead is insignificant and you may well be network bound.  

 

But there is always be some process in a data moving setup that will be the bottleneck.  An ssd  setup as a cache drive is probably the best option if you want to transfer at 100GB/s+ data rates.  I can tell that my 320GB spinner cache drive is not up to the task.  I can also tell you that the background processes on your PC can impact the transfer rate also.

[bonienl]

3 minutes ago, Frank1940 said:

Please understand that there is a lot more to copying a file to an array then just moving the bits across the network.

 

Absolutely. I just wanted to show that with the correct setup it is easy to saturate a 1 Gbps link. unRAID itself does not impose a limitation.

[Frank1940]

11 minutes ago, bonienl said:

 

Absolutely. I just wanted to show that with the correct setup it is easy to saturate a 1 Gbps link. unRAID itself does not impose a limitation.

 

With a 128GB of RAM are you sure that this 6GB file was not simply being buffered to RAM?  The reason I mention this is that I use ImgBurn to create and load .iso files on my server.  Since ImgBurn was written to be a CD/DVD/Bluray disk burning program, it waits until the file has been completely transferred onto the physical media before it says that it has finished its task of writing the data.  When I use it to make an ISO on the server, it will take several extra seconds AFTER the data transfer to the server has ended before the message that the transfer is complete occurs.  This is the time that it takes to move the data from the various RAM buffers to the final location on the physical disk.  The more RAM you have, the bigger those buffers will be.  

[bonienl]

10 minutes ago, Frank1940 said:

The more RAM you have, the bigger those buffers will be

 

That all depends how the system is setup. I have the following in my go file.

# limit disk caching memory (2GiB/1GiB)
sysctl -qw vm.dirty_bytes=2147483648
sysctl -qw vm.dirty_background_bytes=1073741824

 

[johnmatt]

Thanks all for the help.  I'll try a cache SSD and see how that goes