Issue with VT-D and Unraid 6.8.3


Recommended Posts

I think this has more to do with hardware incompatibility than unraid, but here it goes:

 

I have an Intel DQ77MK mobo (Q77 chipset, basically server grade Z77) with 10 drives (ultimately it'll become 12 once I add another SSD for cache pool, and an extra parity drive). Long story short, with a GPU (1660 Ti) installed for PLEX, the only other ports I had available for a SATA expander were a PCIe x4 or x1. That basically ruled out any usable LSI SAS cards which are mostly x8 for their 8 port variants so I went with this Syba card instead: https://www.sybausa.com/index.php?route=product/product&product_id=1006 model SI-PEX40137.

 

BIOS recognizes all my drives just fine as well as Windows 10 with VT-D on, but as soon as I boot Unraid or Mint, the kernel freaks with the errors shown in the dmesg output. I took a screenshot and can go back and post the full kernel buffer later; it's initializing my parity at the moment, and I'm a bit worried re-enabling it would screw up my array. Once booted with it on, only half the drives end up being shown. As soon as I turn VT-D off in the BIOS, no more errors and the kernel boots much faster.

I guess I don't absolutely need VT-D, but it's one of the reasons I bought this board. Any thoughts? Clearly, it does not like the card but Windows seems to do just fine with it (it does have proper Marvell drivers though). Thanks.

20200603_080047.jpg

Edited by nimaim
Link to comment

EDIT: No go for me .. doesn't looks like the kernel likes the card at all, which sucks because Windows handles it just fine. Time to find another solution for the 8 data drives since I feel I will definitely need GPU passthrough (VT-D enabled) for a Plex container.

Is it even worth continuing the initialization of the parity drive since all the drive assignments will change with a new card, or should I just let it finish? Also, any suggestions for an x1 or x4 card that will reliably work with 8 drives in unraid? Again, all LSI SAS 8 port cards are x8 so that's out of the question.

Link to comment
24 minutes ago, johnnie.black said:

AFAIK there are no good x4 8port controller options.

Yea I've looked all over ... this seems to be accurate. There is one very roundabout way to do it with SAS cards though, which is to pair a LSI 9211-4i variant card (the only x4 SAS card out) with a SAS expander powered externally with a riser card (PITA but may be my only solution at this point without tearing apart my current set up). Figured it was worth sharing: 

 

.. 

Edited by nimaim
Link to comment
4 minutes ago, nimaim said:

There is one very roundabout way to do it with SAS cards though, which is to pair a LSI 9211-4i variant card (the only x4 SAS card out) with a SAS expander

Yes, that's a possibility, I have a couple of those, and at one time had one of those connected to 32 disks with an expander.

Link to comment
25 minutes ago, Michael_P said:

The RES2SV240 can be powered with a 4 pin molex connector

Interesting... that is definitely a cleaner solution. Have you used this? Any idea if that will work with a 9211 4i? Is the PCI-E port on it just used for power or do you need another spare PCI-E port on ur mobo for this? I assume molex is the alternate way to power this (or is it in addition to?)

Edited by nimaim
Link to comment

Thanks @Michael_P. Never even heard of this card so appreciate your recommendation. Got one on the way. I'll have to find some nifty way of mounting this thing in my Node 804. Definitely a cleaner solution than the PCI-E riser/extender option. What a PITA just to get a bunch of drives working.

Link to comment

@Michael_P Great minds think alike, that was exactly my idea. Pretty sure I will never use the PCI slot. I'm a 3d printer hobbyist so going to design a PCI bracketed "tray" that I can screw in and zip tie this board around.

 

@johnnie.black if only i had that much room in my chassis, lol. It's a miracle any small consumer case (microATX at that) fits like 12 drives as it is. The Node 804 is dual chamber so maybe can mount that card on the HDD cage side, but it's pushing it.

Edited by nimaim
Link to comment

@johnnie.black Side question on the same topic, while I'm waiting for my expander to come in, can I still set up my disks with this Syba card and VT-D disabled? Or not recommended as the SAS card will reassign these drives? It'd be nice to have everything ready to go once it comes in since parity drive initialization takes a full day.

Link to comment
4 minutes ago, nimaim said:

can I still set up my disks with this Syba card and VT-D disabled?

It should be mostly fine with VT-d disable, but they are also known for sometimes dropping disks without a reason, though they work more or less reliable for some, I would say it should be low risk to use for a little while.

Link to comment
  • 3 weeks later...

Wow I got screwed on two cards on Ebay and finally got a RES2SV240 that works. First seller sent the completely wrong card, second one I went with a cheaper IBM SAS expander I was going to power off a riser card and that ended up being a fake. Did a test run with Molex power on this third one and works well, other than it getting insanely hot in just 15-20 minutes of starting the parity sync.


@Michael_P I'm guessing for the fan, you took the heatsink off, put a thermal pad and then the fan?

Edited by nimaim
Link to comment

Yeah, that's the problem with Ebay, Amazon too sometimes. I had good luck with my HBAs on Ebay, but got my expander from Newegg

 

14 hours ago, nimaim said:

@Michael_P I'm guessing for the fan, you took the heatsink off, put a thermal pad and then the fan?

 

I just slapped the fan on top. Wasn't terribly concerned with heat, just figured couldn't hurt to have a bit of air on it.

 

If I see a heat sink, I have a compulsion to slap a fan on it.

Link to comment

Thanks! Looks like the H110 (9211-4i) + RES2SV240 SAS expander is a good solution to get ~20 disks working off a single PCie x4 port. Limited bandwidth of course but more than enough for mechanical drives. Highly recommend this over the Marvell garbage.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.