Bladedude Posted March 4 Share Posted March 4 8 hours ago, JorgeB said: Are use using fiber with transceivers or DAC cables? DAC cables Quote Link to comment
JorgeB Posted March 4 Share Posted March 4 I'm sorry, I don't see how that can be an Unraid problem, driver is not detecting a link, and it won't work until it does, and find it very strange that TrueNAS Scale would detected a link with exactly the same hardware, it uses the same driver. Quote Link to comment
Bladedude Posted March 4 Share Posted March 4 (edited) 7 minutes ago, JorgeB said: I'm sorry, I don't see how that can be an Unraid problem, driver is not detecting a link, and it won't work until it does, and find it very strange that TrueNAS would detected a link with exactly the same hardware. Thanks, but help me understand: If I reboot the server, without touching the server or hardware, and only change the boot device, TrueNas Scale boots and detects the Mellanox NIC with DAC cables and receives a reserved IP. Unraid does not. Same HW, one OS brings up NIC ok, but another OS does not. Sounds to me like software. Edited March 4 by Bladedude Quote Link to comment
JorgeB Posted March 4 Share Posted March 4 Not sure, you missed my edit, TrueNAS scale uses the same driver, so don't see how that is possible, but not saying it's not happening. Quote Link to comment
Bladedude Posted March 4 Share Posted March 4 Yup. Fair enough. Let me 100% validate on my end again. I'll reboot into TrueNas Scale and grab a screenshot and dmesg on boot and see if that can help. Quote Link to comment
simo241 Posted April 24 Share Posted April 24 (edited) On 1/28/2024 at 12:00 PM, simo241 said: I been able to show the Mellanox 10gb Card trought the below procedures: # lspci | grep Mellanox 01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, PCIe 2.0 5GT/s] (rev b0) #echo ib > /sys/bus/pci/devices/0000\:01\:00.0/mlx4_port1 #echo eth > /sys/bus/pci/devices/0000\:01\:00.0/mlx4_port2 Configured the Eth1 to work with windows Pc , only 3 uge problem: 1 - After reboot Eth2 disappear , configuration need to be done again. 2 - if i resume from sleep or i dont use server for a while port eth1 is going down. 3 - Transfer from Unraid to Windows is very slow only 120 Mb/s , from windows to unraid the speed is about 1,5Gb/s Hi everyone , anybody can help me on point # 2 , i been able to resume the ETH port trough a script but how can i do it in automatically after resume from sleep and not manually every time , any suggestion? Also i still experience very slow transfer speed Edited April 24 by simo241 Quote Link to comment
simo241 Posted April 28 Share Posted April 28 On 4/24/2024 at 2:54 PM, simo241 said: Hi everyone , anybody can help me on point # 2 , i been able to resume the ETH port trough a script but how can i do it in automatically after resume from sleep and not manually every time , any suggestion? Also i still experience very slow transfer speed Any help please Quote Link to comment
Bladedude Posted May 4 Share Posted May 4 (edited) Data when booting server and Mellanox ConnectX-3 Pro into TrueNas scale and ethernet comes up, but ethernet does come up when I reboot into Unraid. admin@supermicro[~]$ uname -srm Linux 6.1.63-production+truenas x86_64 admin@supermicro[~]$ /sbin/ifconfig enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.X.201 netmask 255.255.255.0 broadcast 192.168.X.255 inet6 XX prefixlen 64 scopeid 0x20<link> inet6 XX prefixlen 64 scopeid 0x0<global> ether 9c:dc:X txqueuelen 1000 (Ethernet) RX packets 47076 bytes 49359473 (47.0 MiB) RX errors 0 dropped 1 overruns 0 frame 0 TX packets 16362 bytes 6274827 (5.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 admin@supermicro[~]$ sudo dmesg | grep -i enp1s0 [ 9.744788] mlx4_core 0000:01:00.0 enp1s0: renamed from eth0 [ 9.836760] mlx4_core 0000:01:00.0 enp1s0d1: renamed from eth0 [ 11.154076] mlx4_en: enp1s0: Link Up [ 48.685128] mlx4_en: enp1s0: Steering Mode 1 [ 48.740749] mlx4_en: enp1s0: Link Up admin@supermicro[~]$ netstat -i Kernel Interface table Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg enp1s0 1500 57271 0 1 0 20159 0 0 0 BMRU admin@supermicro[~]$ lspci | grep Ethernet 01:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro] Edited May 4 by Bladedude Quote Link to comment
bmartino1 Posted May 4 Share Posted May 4 to use Mellanox ConnectX cards, you need to install the plugin and run the firmware: while the infiniband for the ring form 1 dac to mutil doesn't work in unraid. a single interface direct sac to sac fiber line can still carry the full bandwidth when setup correctly. Quote Link to comment
Bladedude Posted May 4 Share Posted May 4 (edited) 13 minutes ago, bmartino1 said: to use Mellanox ConnectX cards, you need to install the plugin and run the firmware: while the infiniband for the ring form 1 dac to mutil doesn't work in unraid. a single interface direct sac to sac fiber line can still carry the full bandwidth when setup correctly. Thanks @bmartino1. I'm not using infiniband, just 10GbE ethernet via DAC cable. Will that work as is? Or do I need fiber transceiver and cable? Edited May 4 by Bladedude Quote Link to comment
bmartino1 Posted May 4 Share Posted May 4 yes ich777 made that plugin for that exact setup. Tested with a freinds melnox 10G fiber and dak to a mikrotek switch. install plugin via app, may need to reboot settings confirm via firmware that ist up and check unraid setting netwrok for a igbc interface. Quote Link to comment
Bladedude Posted May 5 Share Posted May 5 (edited) @bmartino1Thanks. I had already installed the Mellanox Firmware Tools plugin. I assumed the interface would be enabled by default on install. It wasn't, but all I had to do was enable the interface. The board has 4 GbE + 2 SFP+, so I missed that on the network page. 🤦♂️ Once enabled, it came right up with no issues. Thanks again! Feature request / bug fix would be to enable, by default, on install any/all interfaces. Edited May 5 by Bladedude 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.