Mellanox Connectx2 Issues


Recommended Posts

7 minutes ago, Hannibal said:

update: got new cards connection is rock solid no issues like the previous cards had but my transfer speeds are abysmal... tried to transfer a 11GB file and only saw a max of 80 Mb/s...

try some of this, i have this in my go file, edit as needed!

 

ethtool -G eth0 rx 8192 tx 8192;ifconfig eth0 down;sleep 1;ifconfig eth0 up
route add default gw 192.168.1.1

sysctl -w net.nf_conntrack_max=700000

sysctl -w net.ipv4.tcp_timestamps=0
sysctl -w net.core.netdev_max_backlog=250000
sysctl -w net.ipv4.tcp_sack=1
sysctl -w net.core.rmem_max=4194304
sysctl -w net.core.wmem_max=4194304
sysctl -w net.core.rmem_default=4194304
sysctl -w net.core.wmem_default=4194304
sysctl -w net.core.optmem_max=4194304
sysctl -w net.ipv4.tcp_rmem="4096 87380 4194304"
sysctl -w net.ipv4.tcp_wmem="4096 65536 4194304"
sysctl -w net.ipv4.tcp_low_latency=1
sysctl -w net.ipv4.tcp_adv_win_scale=1

 

 

btw transferring to cache drive i assume?

 

Tim

 

Link to comment
  • Replies 83
  • Created
  • Last Reply

Top Posters In This Topic

Well i'm dumb to say the least that transfer speed was between two mechanical drives (WD red in my unraid box to a WD green in my windows machine) but however just transferred from my WD green to my cache in unraid and saw about 400 mb/s still seems a little low for a 10gb connection.... 

Link to comment
Just now, Hannibal said:

Well i'm dumb to say the least that transfer speed was between two mechanical drives (WD red in my unraid box to a WD green in my windows machine) but however just transferred from my WD green to my cache in unraid and saw about 400 mb/s still seems a little low for a 10gb connection.... 

Id recommend looking at running iperf to test line speed, samba and unraid has some overhead to it.

 

https://elkano.org/blog/testing-10g-network-iperf/

 

Tim

 

Link to comment
26 minutes ago, Hannibal said:

alright to adjust this on the unraid side and sorry if this is a dumb question but do i just SSH into the unraid machine? still kinda new to unraid so was not sure if they're is a terminal built into unraid that im missing somewhere...

im going to assume your using windows, download putty and SSH to IP of unraid server, username is 'root'

 

edit the go file:

nano /boot/config/go

do the needful and save (cnrl x)

 

Link to comment
3 minutes ago, Hannibal said:

Yes I am using windows and I have putty... Another thing I was wondering about is if you can use the mellanox mlxup firmware tool within a putty SSH to the unraid machine... 

nope, you can try but there are a lot of libraries that are missing because unraid is a stripped down version of linux.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.