billington.mark Posted May 25, 2018 Share Posted May 25, 2018 (edited) So I spotted this today... a single slot x16 PCI-e expansion card for 4x NVMe drives: https://www.scan.co.uk/products/asus-hyper-m2-card-pcie-30-x16-4x-m2-pcie-2242-60-80-110-slots-intel-vroc-support-for-asus-x299-moth https://www.asus.com/uk/Motherboard-Accessory/HYPER-M-2-X16-CARD/ Does anyone have one of these running on a non-X299 board? Would be very very useful if thats the case! At that price im tempted to just get it and see if it works anyway... Ideally it would still split each NVMe into its own IOMMU group too.. but i think im asking a bit much there! Edited May 31, 2018 by billington.mark Quote Link to comment
unrateable Posted May 25, 2018 Share Posted May 25, 2018 considering this a buy too. I guess as long as your mainboard support PCIe bifurcation (check your bios) it will work and split the lanes up. on top of that individual iommu groups would be great. tell me more about it ! not many reports out there....btw anybody with supermicro 2xnvme >pcie adapter experience here ? Quote Link to comment
billington.mark Posted May 26, 2018 Author Share Posted May 26, 2018 I get the option to change the lane layout for my x16 slots (x16 or x8x8 or x4x4x4x4), so im hoping its supported.... Ive ordered the part and i'll report back once ive fitted it. For anyone curious this is the setup im using: ASRock - EP2C602-4L/D16 2x E5-2670 Quote Link to comment
LammeN3rd Posted May 28, 2018 Share Posted May 28, 2018 On 5/26/2018 at 8:15 PM, billington.mark said: I get the option to change the lane layout for my x16 slots (x16 or x8x8 or x4x4x4x4), so im hoping its supported.... Ive ordered the part and i'll report back once ive fitted it. For anyone curious this is the setup im using: ASRock - EP2C602-4L/D16 2x E5-2670 In that case it will work without a problem! the Asus Hyper M2 is a really nice affordable piece of hardware, I did not check if my bios supported PCIe bifurcation so i ended up buying 3 of them. they are not that much more expensive as a card for 1 M2 SSD and they are way nicer! (I had PCIe issues with cheap Chinese PCIe to M2 cards) Quote Link to comment
eschultz Posted May 28, 2018 Share Posted May 28, 2018 Sounds like this card (or multiple) will work on some X399 ThreadRipper motherboards even: Quote Link to comment
billington.mark Posted May 28, 2018 Author Share Posted May 28, 2018 From my google-fu on this, it seems PCI-e bifurcation is a feature in most CPUs from the last few years, its just a case of your BIOS giving you the option to configure the x16 slot(s) to 4x4x4x4x, which only seems to be available in higher-end gaming or workstation motherboards. Its a public holiday in the UK today, so I'll not get this delivered until later in the week to test unfortunately. Quote Link to comment
billington.mark Posted May 31, 2018 Author Share Posted May 31, 2018 So its arrived and fitted, and.. Success! You DO NOT need an x299 based motherboard for this card to work, only a montherboard\cpu capable of splitting an x16 slot into 4x4x4x4x. This was found in my PCIe settings in the BIOS. Even better, each NVMe device is still in its own IOMMU group: IOMMU group 47: [144d:a802] 81:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 (rev 01) IOMMU group 48: [144d:a808] 82:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 Only thing i had to to after installing was to update a VMs XML which has the 970evo passed through to it with the new PCIe address. So, ive free'd up a PCIe slot, and have the option to add 2 more NVMe devices in the future without much hastle at all. 1 Quote Link to comment
Tybio Posted May 31, 2018 Share Posted May 31, 2018 Found a review of the ASRock version: ASRock Ultra Quad M.2 Card Read more: https://www.tweaktown.com/reviews/8542/asrock-ultra-quad-2-card-16-lane-aic-review/index.html They make a big deal about how the orientation of the slots in this is better...but really these are just adapters. I'd think the only real delta would be in heat dissipation and fan noise. Quote Link to comment
uldise Posted June 1, 2018 Share Posted June 1, 2018 has anyone some success with this or similar card on a Supermicro X9 board? Quote Link to comment
billington.mark Posted June 1, 2018 Author Share Posted June 1, 2018 1 hour ago, uldise said: has anyone some success with this or similar card on a Supermicro X9 board? What model board do you have? Quote Link to comment
uldise Posted June 1, 2018 Share Posted June 1, 2018 18 minutes ago, billington.mark said: What model board do you have? i have Supermicro X9DRH-7F.. Quote Link to comment
billington.mark Posted June 1, 2018 Author Share Posted June 1, 2018 5 minutes ago, uldise said: i have Supermicro X9DRH-7F.. Page 4-10 of your motherboard manual.... Clearly says that bifurcation is supported on a couple of your PCIe slots. Check your BIOS settings and see if you have the option to change things on your x16 slots. Quote Link to comment
uldise Posted June 1, 2018 Share Posted June 1, 2018 2 hours ago, billington.mark said: Page 4-10 of your motherboard manual.... i think it is not so clear. i asked SM support some time ago about their AOC-SLG3-2M2 which is very similar and also need PCIe bifurcation to work properly. Look at the answer: I'm sorry to tell you that you are not able to use the NVMe SSD with your current system since our X9 series only has BIOS that supports EFI only or Legacy only. You can try it out with your own discretion. To use NVMe, you must have PCI-e slot where the NVME SSD installed with EFI enabled. Otherwise, the system will not detect it, usually there is an option to set only one slot as EFI if you need Legacy for your other cards, but since this X9 board is relatively old it must not have that feature. Our X9 motherboard series only has BIOS that supports either Legacy mode only or EFI mode only, you can’t combined. Add-on Card AOC-SLG3-2M2 only compatible with the following motherboards as shown below. It will not work with any other motherboards and you can try it out with your own discretion. https://www.supermicro.com/products/accessories/addon/AOC-SLG3-2M2.cfm i'm not sure about my other cards: i have one IBM 1015 and one QLogic BR-1020 10Gbit Ethernet card. Quote Link to comment
uldise Posted June 2, 2018 Share Posted June 2, 2018 anyone else with Supermicro X9 and PCIe bifurcation? Quote Link to comment
billington.mark Posted June 2, 2018 Author Share Posted June 2, 2018 I think they're getting at the point that you won't be able to boot from an nvme device as it won't be detected as a bootable device in the bios... Which if you're using it with unraid, that's not an issue. That's how I read that response anyway. Quote Link to comment
uldise Posted June 2, 2018 Share Posted June 2, 2018 yes, this was a question a asked them next: "if i don't need NVME SSD as boot device, but just wanna use it as VMWare ESXi datastore, do i still need a bios support for it?" and they replied: "Unfortunately, no." so i really did not understand their answer, and was not asked any further.. Quote Link to comment
LammeN3rd Posted June 3, 2018 Share Posted June 3, 2018 You need Bios support for PCIe bifurcation without only the first NVMe SSD will work. Bios support for NVMe is an different thing and would be needed if you want to boot from your NVMe drive, since unRAID boots from USB NVMe boot support is not needed! Quote Link to comment
boxer74 Posted September 21, 2018 Share Posted September 21, 2018 Did you dig further into this with the X9 board? Curious to know the result. Quote Link to comment
Alphahelix Posted September 21, 2018 Share Posted September 21, 2018 Did you dig further into this with the X9 board? Curious to know the result.+1Sent from my SM-G955F using Tapatalk Quote Link to comment
kron Posted November 12, 2018 Share Posted November 12, 2018 I went ahead and picked up one of these and enabled bifurcation on PCIe slot 5 (CPU2 Slot5). I setup 8x 8x since I'm only using 2 SSD's for now. Unraid doesn't detect these drives so I must be doing something wrong.. Page 4-10 of the manual states, this option isn't present on the latest bios of 3.3 IIO 1 PCIe Port Bifurcation Control This submenu configures the following IO PCIe Port Bifurcation Control settings for IIO 1 PCIe ports to determine how the available PCI-Express lanes to be distributed between the PCI-Exp. Root Ports Board: X9DRi-LN4F+ Manual: https://www.supermicro.com/manuals/motherboard/C606_602/MNL-1258.pdf Quote Link to comment
tjb_altf4 Posted November 12, 2018 Share Posted November 12, 2018 Have you tried running in x4/x4/x4/x4 mode to see if that fixes the issue? If your on a slot that doesn't have that option, you may have the m.2s connected to the "second half" of the x8 instead of the "first half", changing the slots the m.2s are connected to may fix the issue. Quote Link to comment
kron Posted November 12, 2018 Share Posted November 12, 2018 27 minutes ago, tjb_altf4 said: Have you tried running in x4/x4/x4/x4 mode to see if that fixes the issue? If your on a slot that doesn't have that option, you may have the m.2s connected to the "second half" of the x8 instead of the "first half", changing the slots the m.2s are connected to may fix the issue. Just flipped it to x4/x4/x4/x4 and the drives still don't show up for preclearing. The m.2's are in slots 1 and 2 on the hyper adapter. I'll try moving that to 1 and 3 with x8/x8 tomorrow. Quote Link to comment
uldise Posted November 12, 2018 Share Posted November 12, 2018 are you booting with UEFI? i don't know if this is related, but when i asked about bifurcation on X9 boards to SM support some time ago, they answered, it's only works if you boot in UEFI mode.. Quote Link to comment
kron Posted November 12, 2018 Share Posted November 12, 2018 (edited) 6 hours ago, uldise said: are you booting with UEFI? i don't know if this is related, but when i asked about bifurcation on X9 boards to SM support some time ago, they answered, it's only works if you boot in UEFI mode.. Yes, PCIe mode is set to UEFI only. Other settings remained the same for Slot 5 (x4/x4/x4/x4). Still didn't pull up the drive in System Devices -moved drive 2 from slow 2 to 3 on the hyper drive. Set to x8/x8 and neither show. Maybe try another slot but the manual seems to state that all of the x16 slots are the same.:?!? Edited November 12, 2018 by kron additional details Quote Link to comment
JonathanM Posted November 13, 2018 Share Posted November 13, 2018 2 hours ago, kron said: Yes, PCIe mode is set to UEFI only. That wasn't the question. 8 hours ago, uldise said: are you booting with UEFI? Please verify that you have enabled UEFI boot mode on the USB stick, and you have legacy boot disabled in the BIOS. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.