Jump to content

Inclusion of nvmet-tcp and nvmet-rdma kernel modules for NVMe-oF over RDMA.

Recommended Posts



I've been looking at Unraid as an alternative for my NAS, now that it supports ZFS, and the way TrueNAS Scale is heading.


One reason I'm currently running latter currently is that it's Linux based and ships the nvmet-rdma (and nvmet-tcp) module(s) out of the box. With this I can use NVMe-oF, and that via RDMA on my Mellanox ConnectX3 cards. Over on my Windows desktop, I'm using the Starwind NVMe-oF initiator. NVMe-oF is a bit faster than iSCSI over TCP, and even more so when using RDMA (when things are hot in ARC, at least in read, it can almost hold a candle to my Samsung 970 Evo on my current HAS hardware, per Crystal Diskmark).


Unraid ships with nvmet (at least per bzImage) but none of the necessary transport modules, other than nvmet-loop. If the modules from the title were included, one could set it up NVMe-oF via either TCP or RDMA.


Then I could figure out the plugin stuff and get the Python-based nvmetcli tool into Unraid, to configure things the easy way (instead of echoing into /sys/*).


One reason I'm doing it the harder way instead of just iSCSI is maximizing any performance to run games (especially those that stream a lot of data) from a ZVOL over the network.


(The best case would be native support with UI and everything, but I figure this isn't going to happen anytime soon, due to demand, initiator availability and everything.)


Thanks for considering.

Edited by Glowtape
  • Upvote 1
Link to comment
  • 11 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...