Omid

This is less of an issue but I've just noticed the duration is inconsistently reported between the two as well. The GUI shows "16 hr, 35 min, 56 sec" whilst the notification shows "16 hours, 47 minutes, 6 seconds". Comparing the duration of random historical parity checks against old email notifications confirms this isn't a one-off.

v6.12.4 The email notification received after a regular scheduled parity check had the subject line "Notice [UNRAID] - Parity-Check finished (0 errors) " which, according to the unraid GUI, is wrong, unless "errors" here is referring to something else. Event: Unraid Parity-Check Subject: Notice [UNRAID] - Parity-Check finished (0 errors) Description: Duration: 16 hours, 47 minutes, 6 seconds. Average speed: 132.4 MB/s Importance: warning versus This is potentially dangerous, as it means administrators won't be informed of sync errors unless they look at the GUI. Is this a bug? As a side note, there doesn't seem to be anywhere which indicates if these errors were "detected" or "corrected" (defined by how the parity check was run). Is this a feature request?

I think I have a very similar scenario and I can't find a way forward ☹️ slowly going from "this will be a fun challenge" to "this is very frustrating". I had iGPU and Quick Sync working fine with the W680 ACE IPMI motherboard. I also had video output working through the IPMI card, both via the KVM and with a monitor plugged in to the card's VGA port. I then added an LSI 9201-8i HBA card and really struggled to get it working but finally got there with help from the community. At least that's what I thought, until a couple of weeks later when I noticed unRAID couldn't see the iGPU and all transcoding was done by the CPU (an easy oversight). tl;dr: I needed to enable CSM for the HBA to properly initialise and attached drives to be detected by unRAID. I never checked if the iGPU was detected with the HBA installed and CSM disabled, because I never started the array due to drives not being detected, but I have a strong feeling this is caused by enabling CSM and not simply having the HBA card in. Trying to move forwards, I'm a little suspicious of the Graphics Configuration page in BIOS, because it makes no reference to CPU graphics at all. It's not one of the options in the drop-down and the iGPU Multi-Monitor field isn't there at all (I'm almost certain I recall it being enabled when I first installed this build, before all the HBA and CSM madness). Here is what I see: I've tried booting with all 3 options but nothing changes. It's as if the iGPU isn't detected or initialised during POST/boot. Any advice, comments or thoughts would be appreciated, before I go crazy(!). Thanks in advance!

Before: After: This was the solution and I got everything up and running back then, but I learned of a problem after a few days that was difficult to notice at first... My iGPU isn't detected/active/working anymore. I've not rebooted, checked BIOS settings or tried anything yet but I know it's not working because the GPU Statistics plugin now just shows "Vendor command returned no data." and the output from the intel_gpu_top command aligns with this. root@unRAID:~# intel_gpu_top No device filter specified and no discrete/integrated i915 devices found root@unRAID:~# ls -ltrah /dev/dri/ total 0 drwxrwxrwx 2 root root 60 Dec 16 02:15 by-path/ drwxrwxrwx 3 root root 80 Dec 16 02:15 ./ crwxrwxrwx 1 root video 226, 0 Dec 16 02:15 card0 drwxr-xr-x 16 root root 4.6K Dec 17 14:37 ../ I know it was working for many months until I installed the LSI HBA card, so it's either the presence of the card or enabling CSM (or both?). I'm going to try the following: Just a simple reboot (not expecting this to fix it) Unplug VGA cable from the IPMI card and reboot Plug something in to the motherboard's VGA/HDMI port and reboot Play with BIOS settings (?) Any thoughts/ideas from people here?

@JimmyGerms Thank you so much for trying this out for me. I'm super confused at the different result but appreciate the effort nonetheless. I already had Fast Boot disabled 🙂 but thanks for the inputs, @maydaytek! I've actually shared all of my BIOS changes in an earlier post on this thread (in case anyone is interested). Nope, I've never put a dGPU in this build. Only using the 13700K's iGPU. @jlarmstr Wait, what? Supplemental power for the 9207-8i?

Resolved! CSM and legacy mode PCIe did it. Will post again later...

@golfsands7 Here are the IOMMU groups on my Asus Pro WS W680-ACE IPMI (ATX, not mATX), in two different formats: IOMMU Group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:a780] (rev 04) IOMMU Group 1 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a703] (rev 01) IOMMU Group 2 00:01.0 PCI bridge [0604]: Intel Corporation Device [8086:a70d] (rev 01) IOMMU Group 3 00:06.0 PCI bridge [0604]: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port [8086:a74d] (rev 01) IOMMU Group 4 00:0a.0 Signal processing controller [1180]: Intel Corporation Raptor Lake Crashlog and Telemetry [8086:a77d] (rev 01) IOMMU Group 5 00:0e.0 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller Intel Corporation [8086:a77f] IOMMU Group 6 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller [8086:7ae0] (rev 11) IOMMU Group 6 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake-S PCH Shared SRAM [8086:7aa7] (rev 11) IOMMU Group 7 00:15.0 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 [8086:7acc] (rev 11) IOMMU Group 7 00:15.1 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #1 [8086:7acd] (rev 11) IOMMU Group 7 00:15.2 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #2 [8086:7ace] (rev 11) IOMMU Group 8 00:16.0 Communication controller [0780]: Intel Corporation Alder Lake-S PCH HECI Controller #1 [8086:7ae8] (rev 11) IOMMU Group 8 00:16.3 Serial controller [0700]: Intel Corporation Device [8086:7aeb] (rev 11) IOMMU Group 9 00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11) IOMMU Group 10 00:1a.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #25 [8086:7ac8] (rev 11) IOMMU Group 11 00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:7ac0] (rev 11) IOMMU Group 12 00:1b.4 PCI bridge [0604]: Intel Corporation Device [8086:7ac4] (rev 11) IOMMU Group 13 00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 [8086:7ab8] (rev 11) IOMMU Group 14 00:1c.1 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #2 [8086:7ab9] (rev 11) IOMMU Group 15 00:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #9 [8086:7ab0] (rev 11) IOMMU Group 16 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:7a88] (rev 11) IOMMU Group 16 00:1f.3 Audio device [0403]: Intel Corporation Alder Lake-S HD Audio Controller [8086:7ad0] (rev 11) IOMMU Group 16 00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake-S PCH SMBus Controller [8086:7aa3] (rev 11) IOMMU Group 16 00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH SPI Controller [8086:7aa4] (rev 11) IOMMU Group 17 01:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9215 PCIe 2.0 x1 4-port SATA 6 Gb/s Controller [1b4b:9215] (rev 11) IOMMU Group 18 02:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Western Digital WD Black SN850X NVMe SSD [15b7:5030] (rev 01) IOMMU Group 19 03:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Western Digital WD Black SN850X NVMe SSD [15b7:5030] (rev 01) IOMMU Group 20 05:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Western Digital WD Black SN850X NVMe SSD [15b7:5030] (rev 01) IOMMU Group 21 06:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-LM [8086:125b] (rev 06) IOMMU Group 22 07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-LM [8086:125b] (rev 06) IOMMU Group 23 08:00.0 PCI bridge [0604]: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge [1a03:1150] (rev 06) IOMMU Group 23 09:00.0 VGA compatible controller [0300]: ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000] (rev 52) IOMMU Group 23 09:01.0 IPMI Interface [0c07]: ASPEED Technology, Inc. Device [1a03:2402] (rev 01) Alternative format: Group 0: [8086:a780] [R] 00:02.0 VGA compatible controller Raptor Lake-S GT1 [UHD Graphics 770] Group 1: [8086:a703] 00:00.0 Host bridge Device a703 Group 2: [8086:a70d] [R] 00:01.0 PCI bridge Device a70d Group 3: [8086:a74d] [R] 00:06.0 PCI bridge Raptor Lake PCIe 4.0 Graphics Port Group 4: [8086:a77d] 00:0a.0 Signal processing controller Raptor Lake Crashlog and Telemetry Group 5: [8086:a77f] 00:0e.0 RAID bus controller Volume Management Device NVMe RAID Controller Intel Corporation Group 6: [8086:7ae0] 00:14.0 USB controller Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller USB: [05ac:0250] Bus 001 Device 004 Apple, Inc. Aluminium Keyboard (ISO) USB: [05ac:1006] Bus 001 Device 002 Apple, Inc. Hub in Aluminum Keyboard USB: [046b:ff10] Bus 001 Device 009 American Megatrends, Inc. Virtual Keyboard and Mouse USB: [046b:ffb0] Bus 001 Device 008 American Megatrends, Inc. Virtual Ethernet. USB: [046b:ff01] Bus 001 Device 007 American Megatrends, Inc. Virtual Hub USB: [1b1c:1c1e] Bus 001 Device 006 Corsair HX1000i Power Supply USB: [05e3:0608] Bus 001 Device 005 Genesys Logic, Inc. Hub USB: [0781:5567] Bus 001 Device 003 SanDisk Corp. Cruzer Blade USB: [1d6b:0002] Bus 001 Device 001 Linux Foundation 2.0 root hub USB: [1d6b:0003] Bus 002 Device 001 Linux Foundation 3.0 root hub [8086:7aa7] 00:14.2 RAM memory Alder Lake-S PCH Shared SRAM Group 7: [8086:7acc] 00:15.0 Serial bus controller Alder Lake-S PCH Serial IO I2C Controller #0 [8086:7acd] 00:15.1 Serial bus controller Alder Lake-S PCH Serial IO I2C Controller #1 [8086:7ace] 00:15.2 Serial bus controller Alder Lake-S PCH Serial IO I2C Controller #2 Group 8: [8086:7ae8] 00:16.0 Communication controller Alder Lake-S PCH HECI Controller #1 [8086:7aeb] 00:16.3 Serial controller Device 7aeb Group 9: [8086:7ae2] 00:17.0 SATA controller Alder Lake-S PCH SATA Controller [AHCI Mode] Group 10: [8086:7ac8] [R] 00:1a.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #25 Group 11: [8086:7ac0] 00:1b.0 PCI bridge Device 7ac0 Group 12: [8086:7ac4] [R] 00:1b.4 PCI bridge Device 7ac4 Group 13: [8086:7ab8] [R] 00:1c.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #1 Group 14: [8086:7ab9] [R] 00:1c.1 PCI bridge Alder Lake-S PCH PCI Express Root Port #2 Group 15: [8086:7ab0] [R] 00:1d.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #9 Group 16: [8086:7a88] 00:1f.0 ISA bridge Device 7a88 [8086:7ad0] 00:1f.3 Audio device Alder Lake-S HD Audio Controller [8086:7aa3] 00:1f.4 SMBus Alder Lake-S PCH SMBus Controller [8086:7aa4] 00:1f.5 Serial bus controller Alder Lake-S PCH SPI Controller Group 17: [1b4b:9215] [R] 01:00.0 SATA controller 88SE9215 PCIe 2.0 x1 4-port SATA 6 Gb/s Controller Group 18: [15b7:5030] [R] 02:00.0 Non-Volatile memory controller Western Digital WD Black SN850X NVMe SSD Group 19: [15b7:5030] [R] 03:00.0 Non-Volatile memory controller Western Digital WD Black SN850X NVMe SSD Group 20: [15b7:5030] [R] 05:00.0 Non-Volatile memory controller Western Digital WD Black SN850X NVMe SSD Group 21: [8086:125b] [R] 06:00.0 Ethernet controller Ethernet Controller I226-LM Group 22: [8086:125b] [R] 07:00.0 Ethernet controller Ethernet Controller I226-LM Group 23: [1a03:1150] [R] 08:00.0 PCI bridge AST1150 PCI-to-PCI Bridge [1a03:2000] [R] 09:00.0 VGA compatible controller ASPEED Graphics Family [1a03:2402] [R] 09:01.0 IPMI Interface Device 2402

Help! I've just hooked up an LSI 9201-8i HBA to this motherboard and Unraid won't boot. I get POST with the Asus splash screen showing, then the IPMI card initialising with the firmware/OData configuration step, as usual, but then it changes to a completely black screen. No cursor, blinker or anything. I left it 10+ mins and confirmed on the router that the server has not connected. The screen shows black through IPMI and with a VGA monitor attached. Have I missed any special steps? It's my first time with an HBA card. I added it to the top PCIe 5.0 x16 slot. Next steps: Check BIOS to downgrade PCIe version, although I thought PCIe was good at backwards-compatibility Done: I forced PCIe Gen2 - no success Remove the HBA card just to see if I can get a boot again Done: Confirmed - successful boot Try a different PCIe slot Done: I tried one of the PCIe 3.0 slots and also tried it with Gen2 forced - no success In the 5.0 slots, Q-LED would mostly stay on 40 for ~10 seconds and briefly switch to 39 for just a couple of seconds, before returning to 40 [repeat]. In the 3.0 slots, Q-LED would instead mostly stay on 38 and briefly switch to 37 or 39 but never 40. I assume the slight variation in Q-LED codes is due to the PCIe 5.0 lanes running to the CPU whilst the PCI 3.0 lanes via PCH (?). Regardless, these codes don't help much and only seem to confirm that there's some issue with the HBA card...unless someone can interpret this better than I can? Anyone have any other ideas? Could it be the card's firmware? Would an older firmware have such a significant negative impact? How can I check the current firmware version? EDIT: Based on this thread I have the following next steps. Interestingly, what resolved the OP's issue in that thread was switching from an ASRock board to an Asus one 🤦‍♂️ Check boot device order (I stupidly haven't checked this yet) Done: looks good, no different to when card is detached Check and disable "legacy storage boot" in BIOS I can't find this setting anywhere -- any help? Insulate some PCIe pins (sounds scary) Done: no success @golfsands7 I'm unfamiliar but will try to get that information for you when I'm up and running again. See below

Damn, I wish I had seen this video before! @casperse This should be great info for you!

I decided to keep my NH-D15 and instead moved the IPMI card to one of the x16 slots at the bottom of the board. It's annoying, I know, but won't be a problem for me as I've only got 1 other card in there right now and it's for storage. Honestly, I think that, with a bit of work, I could create a ~2mm gap between the cooler and the card if it's in the x1 slot. I just didn't have the time when I was doing it. If you're really creative and up for the challenge, I think you could make it work with NH-D15 and top slot. It was really close; I had to look very closely to see how small the gap was. Ultimately, I wasn't happy or comfortable with closing the case with that configuration. When I was thinking of options, I was exploring a combination of the following: Mounting the cooler just a couple of mm higher and screwing it down Place a thin insulation layer (plastic/rubber sheet) on top of the IPMI board Maybe shave/fold/clip off a mm of the cooler's fins File down/clip off some of the soldered tips of the IPMI board As you would probably agree, none of these are great solutions and I really dislike the last two. There's also the idea of using a riser cable/card of some sort, but I didn't really explore that much, since I would also have to figure out how the network cable would reach the card.

That's very strange. My attempt to update the BMC firmware was unsuccessful (from 1.0.12 to 1.1.33), so I can't really comment. I use the same credentials to access these BIOS pages. I will point out that the page header in your screenshot looks different to mine. Overall, I'm very happy with this motherboard. I personally find the number of M.2, PCIe slots and SATA/SAS ports perfect for my Unraid use case. I don't believe there's any bifurcation either; if there is, I've not come across it yet with my configuration. I'm less impressed by the IPMI functionality, but, coming from Supermicro's integrated IPMI, I've decided that they all generally suck (unless you're using some enterprise-grade IPMI such as Dell's DRAC or HPE's iLO). I'm not sure where you got that picture from, but the IPMI expansion card is only PCI 3.0 x1 and the very first slot on the motherboard is also PCI 3.0 x1 (look directly below the CMOS battery). That makes it the most ideal slot for the IPMI expansion, as long as you have enough clearance from your cooler. That's unfortunately not the case for me: Hmmm, no, not necessarily. +1 to all of @Daniel15's points

Has anyone here paired this motherboard with a SAS HBA card? Is 9207-8i still an appropriate go-to for these newer motherboards? Any performance/bandwidth bottlenecks with putting 8 HDDs through it via 2 SFF-8087 cables?

@smokeless-grind5220 Yes and that's how I'd recommend making any BIOS changes remotely. This web version is very basic. I wouldn't use it for anything more than a read-only tool to check effective values. A lot of the popular settings are on pages that aren't even reachable (because of the empty page bug). @Daniel15 It could be, yeah, although I'd expect the BIOS changelog/release notes to mention that. What version is your BMC Firmware? Honestly, you're not missing much with this BIOS feature. I can't see myself ever having the confidence in using it.

@smokeless-grind5220 I have the ATX version, but I assume the IPMI functionality is very similar, if not the same. I'm sharing a lot of screenshots below for you. Please note that I do not have any fans. temperature sensors or PSU connected to the IPMI card; only the BMC and USB headers are attached. I need to see if I can supress all of these events, warnings and alerts. It would be ideal if I could completely disable the monitoring/sensors. Are the features "complete"? -- I would say "no". it's about what you would normally expect with these types of products. The UI/UX is equivalent to something I'd throw together during a weekend to test some new frontend framework or for some hackathon project of some sort 🤣 With all that said, it works! More feedback and comments below... This page seems useless. Just templated text/fields. Nothing can be clicked. H5Viewer takes you into a separate tab where you can fully access and interact with the system: The Remote Control page has one button to reach the Remote Media page: Clicking on "BIOS" on the top right (header of every page) opens a new tab which ALMOST mirrors the actual BIOS: Most sections that are nested more than 2 or 3 pages deep seem to always return blank, so I wouldn't rely on this as a way to reliably and accurately make BIOS changes I think this might possibly maybe be the only page and setting I would ever potentially change using this tool...maybe 🙃 There are some pages and settings missing. For example, the entire "Ai Tweaker" tab/menu is missing I hope this helps!

PSA: There is a very new BIOS update available (2802) for these motherboards. Links to download pages below: Pro WS W680 ACE IPMI (ATX, PCIe IPMI card) Pro WS W680M ACE SE (mATX, integrated IPMI) I just jumped from 2305 to 2802 (released 2023/10/06, just 2 days ago) -- the first 26+ hours have been fine and stable. Also, ICYMI: the page also includes BMC Firmware releases (2023/08/04) for both motherboards, although they don't seem to share versions. I actually just tried to upgrade from 1.0.12 to 1.1.33 but was unsuccessful. The file was accepted via the upload form, the device restarted and presented its login page after a couple of minutes, but everything still shows it's on 1.0.12. For anyone interested, here are the only BIOS settings I've changed for my server (i7-13700K, 64 GB DDR5 ECC, ~15 containers) Ai Tweaker > Asus Performance Enhancement 3.0 > Disabled (from Enabled) I value stability over any OC/changing of limits Boot > Boot Logo Display > Disabled (from Auto) Personal preference for a server Boot > Fast Boot > Disabled (from Enabled) For compatibility with unRAID Advanced > System Agent (SA) Configuration > Graphics Configuration > Primary Display > PCIe (from Auto) See this post and the few following it. I only have an iGPU, no discrete GPU Advanced > Onboard Devices Configuration > Q-Code LED Function > Auto (from POST Code) Why not? Useful for when you need to troubleshoot Advanced > Onboard Devices Configuration > Bluetooth and WiFi > Disabled (from Enabled) Neither are available on this board anyway Advanced > Onboard Devices Configuration > SlimSAS Configuration > SATA (from PCIE) Only if you're using the SlimSAS connector (or plan to) Tools > Armory Crate > Disabled (from Enabled) lol IPMI > In-band driver type > Linux (from Windows)