scs3jb

I went through so many iterations of head scratching with the two cards, seems that I forgot that the X13SAE-F only supports single cards in slot 7 (CPU) because of some crazy switching (Slot 4 and Slot 7 both need to be occupied for 8x, or a single 8x must be in slot 7). Thanks as always!

A bit of a botched upgrade, I was replacing a PLX M.2 card as my board doesn't support bifrucation but a new HBA card had the wrong cables. When I tried to reinstall the PLX card again my system just refused to boot. Tried bios upgrades, etc but its stuck. Now I've decided to temporarily run on a SATA SSD for cache but disaster... unraid can no longer see my drives! The drives are all listed in my bios (22 + 1 SSD + 3x M.2), but I've had no luck trying to get UNRAID to see them. The controller in question is seen but throwing errors: Any suggestions? beyonder-nas-diagnostics-20240320-0255.zip

Hello, So I wanted to talk about this. For the life in me I can not POST with the SAS3224 and QUAD M.2 anymore. Can not explain how it booted before and not now. The M.2 drops one by one, and when I tried to switch out the M.2 for another HBA, the motherboard won't POST. Tried upgrading firmware and playing with the bios but nothing, as soon as I put another card in SLOT7 it seems to have given up on having a PCI-e 3.0 8x and a PCI-e 3.0 8x (over 16x). I haven't gone all the way back to 2.0 yet. I had tried a HBA 9600-24i but it couldn't see any drives. I think its cabling, I thought I'd bought 2x 05-60005-00 cables for 4x U.2, but they seem to be non-Broadcom knock-offs. I can only see 05-60005-00 working with HBA 95xx, guess i'll email broadcom

Oh crikey! This if going to be fun. I just had 9600 arrive today and I'm waiting on the cables. I guess I'm going to be playing with the firmware tool and hoping everything works. Thanks for the reply.

Sorry to bump, are you running the Broadcom BCM HBA 9600 in IT mode? I was looking at purchasing but couldn't it pre-flashed to IT mode or much documentation. Wanting to split a pci-e 4.0 16x port for 4x U.2 drives and the HBA 9xxx looks like the best way to do it with my board (intel, you suck... all this money and no bifurcation!?!?). Looks like its working but wondering if you bought it in IT mode, flashed yourself, or running with raid?

I looked but I need four devices unfortunately 😔. I could go down pci-e splitter but would still need bifurcation

I have a spare PCI-e 4.0 16x slot I'd like to use to put some U.2 drives in. My motherboard does not support bifurcation. I'm currently using commercial M2 drives for scratch disks, but I'm killing those pretty fast and have had to RMA them several times. My aim is to have some high speed cache for downloads and temporary / scratch but with some good endurance rating behind it, then stick these in a ZFS/btrfs array. I've only managed to find a PCI-e 3.0 PEX controllers that will let me get 4x U.2 drives (PCI-e 4.0 bifrucation cards seem to be 10x the price, around 1.3k GBP!): https://www.aliexpress.com/item/1005005671021299.html?spm=a2g0o.productlist.0.0.16b5a873zNdaEl&mp=1 https://www.aliexpress.com/item/4000029811733.html?spm=2114.12010612.8148356.16.20ee71776HmCGk I was looking at Samsung PM9A3 2.5" U.2 Enterprise SSD/Solid State Drives, but I understand these would run well under the 6GB/s read speeds they should provide, as at PCI-E 3.0 it would be limited to 3.9 GB/s (985MB/s*4) which is nearly 2/3rds the performance. My questions: Am I overstating the performance drop for PM9A3 if i put them in PCI-e 3.0? If no, is there a high endurance/performance U.2 drive that would be cheaper and better than the PM9A3 (4tb ~ 500 GBP, 2TB ~ 200 GBP) if i'm limited to PCI-e 3.0 speed. Am I doing something stupid and there's a better way to get 4x enterprise flash drives into that PCI-e 4.0 port? Thanks in advance

Darn. Holy smokes, broadcom one is 700+ euros 😶 This one looks a bit more economical: https://www.ebay.co.uk/itm/304481053593 but i'm guessing there's a catch (PCIE3.. slow?)?

Note: My motherboard does not have bifurcation I wanted to upgrade my flash scratch disks and am becoming weary of commercial M.2. I would like to use datacenter grade U.2 flash drives. I however do not have the appropriate ports on my motherboard. I wanted to use a Gen4 PCI-E fullsize card slot, but I'm unsure if my motherboard would see all 4x devices if I used something like this: amazon link / ebay link / aliexpress link I'm guessing by split they mean 'requires a motherboard with bifurcation'? If they don't work, does anyone know of a device that would work to let me use a PCI-E Gen4 slot for 4x U2 without needing to sell a kidney?

I have tried the following: 1. Removing the SAS controller - no luck, so there's no sharing, etc going on. 2. Removing and reseating the m.2 drive 3. Adding some sysconfig boot parameters

I've raised a support request with AsRock. Whilst pulling up the details this feels like it might be on the slackware/unraid side, in the BMC it looks fine (I can see both the PCI-E 4 and m.2 slots in use) whilst unraid can't see the drive... very odd.

No you are on to something, thank you, reading the block diagram its sharing the pci-e 3x and the m.2 and there's: Trying to figure out what BOM option with M2_1 means now!

OcuLink for the U.2 SSD Drives, nvme is via the motherboard (PCH). In terms of sharing one PCI-E sharing/collision, one OcuLink (PCI-E7 / CPU), the rest are all PCH and don't share... in theory. I wouldn't expect PCH to collide with anything. The M.2 slot is pci-e 3.0 4x and SLOT4 is empty (only other PCIe lane). I am using the PCIe5.0 for a SAS controller but that's on CPU. PCIe Expansion Slots (SLOT7 close to CPU) SLOT7 PCIe4.0 x4* [CPU] SLOT6 PCIe5.0 x16 [CPU] SLOT4 PCIe3.0 x1 [PCH] Note *SLOT7 share lanes with OCuLink4. OCuLink4 will disable when SLOT7 is populated Other PCIe Expansion Connectors M.2 slot 1 M-key (PCIe3.0x4) [PCH]; support 2280/2260/2242/2230 form factor OCuLink 1 OCuLink (PCIe4.0 x4)* [CPU] 1 OCuLink (PCIe4.0 x4 or 4 SATA 6Gb/s) [PCH] 1 OCuLink (PCIe4.0 x4) [PCH] 1 OCuLink (PCIe3.0 x4) [PCH] Note *SLOT7 share lanes with OCuLink4. OCuLink4 will disable when SLOT7 is populated Thoughts? You think a SAS controller on a CPU port could class with a m.2 slot on PCH? I don't think it can be the OcuLinks since bios is showing Oculinks and m.2 in the bios. I guess i can pull it when the parity rebuild finishes, but it seems a bit odd. Maybe I didn't spot something in hidden in the manual? https://download.asrock.com/Manual/W680D4U-2L2TG5.pdf

And I also just built out an AsRock W680D4U-2L2T/G5 with an i9 13900k and 128GB DDR. Runs a bit hotter but ipmi / remote environment is better than supermicro and its got PCI-E 5.0, but less nvme slots and only PCI-e 3x for that. For some reason it lists the base clock at 2970MHZ instead of 3000MHZ Currently battling with the m.2 not being detected, but it seems to be compatible and working fine with unraid apart from that.

I used this because my motherboard doesn't support bifrucation: https://www.amazon.co.uk/gp/customer-reviews/RPQHJPZ4LP8L4?ref=pf_ov_at_pdctrvw_srp it supports 3x at gen3 speeds super stable... 4x causes one to drop out though which i discovered after purchase. There's an updated model but i was unable to find it for purchase that supports 4x. Its all transparent. If you don't mind spending a bit more get a u.2 expansion card (or oculink) and use u.2 enterprise ssds. Scan.co.uk has a 1.92TB enterprise samsung which supports 2TB read/write per day for 5 years for ~200 brexit beans which is much better suited to a nas environment.