Partially working: Thunderbolt Networking to Macbook Pro

[loosegoose]

After a few rough starts, I have successfully achieved my goal of building a NAS with working Thunderbolt networking connectivity to a Macbook Pro.

 

I've been able to read and write to my encrypted NVME raidz array at 2GB/s with this setup.

 

Unraid 6.12.13

 

Hardware:

* ASUS X670-P WIFI

* Ryzen 7600

* Kingston 32GB DDR4 RAM ECC

* 3x 4TB NVME ZFS raidz

* 4x 16TB HDD

* ASUS ThunderboltEX 4

 

The ThunderboltEX 4 card is finicky and I had issues with soft reboots hard-locking until I changed PCIe slots.

 

Once plugged in, the thunderbolt0 interface needs to be configured and brought up:

 

ifconfig thunderbolt0 up

ifconfig thunderbolt0 192.168.22.1 # or whatever IP you want

ifconfig thunderbolt0 mtu 9000 # if you want jumbo frames, although this doesn't appear to affect performance for me

 

You'll need to manually set the Thunderbolt Bridge interface on macOS to a relevant IP (192.168.22.2).

 

I haven't figured out how to automatically bring the interface up on Unraid yet, and there are still some reliability issues which appear to be due to the card, or potentially older Linux kernel version on current stable Unraid.

 

EDIT: I also tried the ASUS USB4 PCIe as well, hearing that it was apparently more reliable for Thunderbolt, however only the USB part of it works (at least in Unraid 6.12.13), which is likely due to `thunderbolt: probe of 0000:0c:00.0 failed with error -110` in `dmesg`. I have to yet test with a newer kernel.

Edited November 7 by loosegoose

[vigs]

Dropping my solution in here for everyone. I've only been testing this for the last few hours but it seems to survive reboots and work automatically.

 

Few changes from the solution above. I use a udev rule to bring up the thunderbolt interface any time a thunderbolt cable is connected. That same rule also adds the interface to br0 so it has access to the rest of the network through the unraid box. That has the added benefit of not needing to use static ips on the thunderbolt interface or the mac. The Mac can just get an ip from DHCP and you don't need to mess with separate subnets. Finally, it sets the mtu of the thunderbolt interface to 9000 which squeezes out a little extra performance especially at 40g speeds. I also set the bridge to spanning tree just in case there could be any loops since the mac would also be connected to the network with wifi. Unsure if that's necessary but it's working for me so I'm rolling with it.

 

my udev rule that i save at /boot/rules/99-thunderbolt.rules

SUBSYSTEM=="net", KERNEL=="thunderbolt0", ACTION=="add", \
    RUN+="/sbin/ip link set thunderbolt0 up", \
    RUN+="/sbin/ip link set thunderbolt0 master br0", \
    RUN+="/sbin/ip link set dev br0 type bridge stp_state 1", \
    RUN+="/sbin/ip link set thunderbolt0 mtu 9000"

 

I added the following to my /boot/config/go to do all the same stuff at boot as well as copy over the udev rule and enable it so that it works until next reboot

# Ensure the thunderbolt0 interface is up and part of the br0 bridge
/sbin/ip link set thunderbolt0 up
/sbin/ip link set thunderbolt0 master br0
/sbin/ip link set dev br0 type bridge stp_state 1
/sbin/ip link set thunderbolt0 mtu 9000

# Copy the thunderbolt rules file
cp /boot/rules/99-thunderbolt.rules /etc/udev/rules.d/
chmod 644 -R /etc/udev/rules.d/99-thunderbolt.rules 2> /dev/null

# reload udev rules
udevadm control --reload-rules
udevadm trigger

 

So far it's working for me and I hope it works for you. Please improve on it if you know more than I do! (I don't know much...this is a combination of ChatGPT and some other helpful posts I found)

Edited November 28 by vigs