Hi,

 

I recently built a new Unraid server - see my post here: link.

 

I'm having trouble reaching 10Gbit speeds on Unraid via a the share. Internally, I test the drive speed and get much faster results - see below:

 

I have two test computers, one running Unraid (new build), and one running FreeNAS (existing computer).The Unraid computer has a 512GB Samsung 860 Pro NVME drives, which is capable of >3GB/s read and ~2GB/s write (verified via dd if=/dev/zero of=test10GB bs=1M count=10k)

 

Computer A: FreeNAS, with 10Gbit card --> NFS Share "ShareA"

Computer B: Unraid, with 10Gbit card --> NFS Share "ShareB"

 

When reading/writing to NFS  on FreeNAS, from Unraid, I get 800MB/s+. For example:

B <--- ShareA = 900MB/s (read)

B ----> ShareA = 800MB/s (write)

 

The problem is when I read/write to Unraid's share.

A <----- ShareB = 200MB/s (read)

A ----> ShareB = 300MB/s (write)

 

As mentioned above, when I go locally on the Unraid computer and write to the NVME drive, I get >3GB/s for both read and write. Additionally, when I try and use the NFS share locally on Unraid, I get slow speeds (B <----> ShareB) - around 400MB/s. As such, it seems like it's related to the share, rather than the disk.

 

Any help would be greatly appreciated - thank you!

See if any of these help:

 

1-Change NIC mtu do 9000 (unRAID server and any other computer with a 10GbE NIC)
2-Go to Settings -> Global Share Settings -> Tunable (enable direct IO): set to Yes

5 hours ago, johnnie.black said:

See if any of these help:

 

1-Change NIC mtu do 9000 (unRAID server and any other computer with a 10GbE NIC)
2-Go to Settings -> Global Share Settings -> Tunable (enable direct IO): set to Yes

Thanks - I've tried #1 - and it helped provide a 10% boost (actually included that in the numbers I posed above).

 

Also tried #2 - for some reason it made my transfers slower. In the description above, it went from:

 

A <----- ShareB = 200MB/s (read)

A ----> ShareB = 300MB/s (write)

 

To:

 

A <----- ShareB = 150MB/s (read)

A ----> ShareB = 200MB/s (write)

8 hours ago, johnnie.black said:

See if any of these help:

 

1-Change NIC mtu do 9000 (unRAID server and any other computer with a 10GbE NIC)
2-Go to Settings -> Global Share Settings -> Tunable (enable direct IO): set to Yes

Just an update:

 

Testing read cache directly --> 2.0GB/s

dd if=/mnt/cache/test10GB of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 5.3521 s, 2.0 GB/s

Testing from cache via "user" share --> ~1.0GB/s

 

dd if=/mnt/user/test10GB of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 10.8258 s, 992 MB/s

Testing from NFS share (keep in mind local IP for 10GB adapter is 192.168.100.200)

mkdir nfs_unraid_test                                
mount 192.168.100.200:/mnt/user/Movies/ nfs_unraid_test
cd nfs_unraid_test/
dd if=test10GB of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 13.8409 s, 776 MB/s

 

However, when testing from the FreeNAS machine (Unraid mounted over NFS):

dd if=test10GB of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes transferred in 3.015465 secs (3560783549 bytes/sec)

 

I read you OP to quickly and didn't notice Freenas was on the other end, it's really difficult to test like that, I also have a Freenas server and both are connected to my desktop with 10Gbe and Freenas is slower than unRAID, I can get practically line speed reading/writing to unRAID and only about 600/800MB/s read/write speed to Freenas (and it took some tuning to get those speeds), if you search Freenas forums you'll see that most can't get line speed with it, to better test and see where the problem is you'd need to connect unRAID directly to a Windows desktop.

Edited July 15, 20178 yr by johnnie.black

7 hours ago, johnnie.black said:

I read you OP to quickly and didn't notice Freenas was on the other end, it's really difficult to test like that, I also have a Freenas server and both are connected to my desktop with 10Gbe and Freenas is slower than unRAID, I can get practically line speed reading/writing to unRAID and only about 600/800MB/s read/write speed to Freenas (and it took some tuning to get those speeds), if you search Freenas forums you'll see that most can't get line speed with it, to better test and see where the problem is you'd need to connect unRAID directly to a Windows desktop.

 

I understand - the thing is, when reading/writing FreeNAS's share from Unraid, I get very fast speeds. The problem is reading Unraid's share from FreeNAS.

 

I will try from a Windows computer and let you know.