Tybio

I can confirm, I get the same result as @rinseaid; Not ideal, but as this build is plugging a hole in the product lines until 7nm solutions come out for me, I can deal with it for a few years :). If this was intended to be a 10 year build, I'd have a different opinion.

I'll put it in, but stick to using the script for now

Hey, quick question. I just added the apcupsd exporter and I'm getting an error. Not sure if this is because I'm using a cyberpower UPS or what: Traceback (most recent call last): File "/src/apcupsd-influxdb-exporter.py", line 36, in <module> 'BATTV': float(ups['BATTV']), KeyError: 'BATTV' This is what I have in my apcaccess if that matters: root@Storage:/mnt/user/AppData/Config/scripts# apcaccess APC : 001,033,0821 DATE : 2019-02-01 15:40:38 -0800 HOSTNAME : Storage VERSION : 3.14.14 (31 May 2016) slackware UPSNAME : Storage CABLE : USB Cable DRIVER : USB UPS Driver UPSMODE : Stand Alone STARTTIME: 2019-01-27 04:40:04 -0800 MODEL : OR1500PFCLCD STATUS : ONLINE LINEV : 120.0 Volts LOADPCT : 11.0 Percent BCHARGE : 100.0 Percent TIMELEFT : 68.0 Minutes MBATTCHG : 10 Percent MINTIMEL : 10 Minutes MAXTIME : 600 Seconds OUTPUTV : 120.0 Volts DWAKE : -1 Seconds LOTRANS : 100.0 Volts HITRANS : 132.0 Volts ALARMDEL : No alarm NUMXFERS : 1 XONBATT : 2019-01-30 05:02:27 -0800 TONBATT : 0 Seconds CUMONBATT: 8 Seconds XOFFBATT : 2019-01-30 05:02:35 -0800 SELFTEST : NO STATFLAG : 0x05000008 SERIALNO : GX1HS2000054 NOMINV : 120 Volts NOMPOWER : 1050 Watts END APC : 2019-02-01 15:40:46 -0800

Very interesting, Going to do some testing today, I have a small monitor I can connect for the moment as I don't have a display emulator plug handy...but I should be able to replicate the setup. Will report back.

Nope, the only thing you lose is remote console...all other functions of IPMI seem to work fine.

The 2018 report is out: https://www.backblaze.com/blog/hard-drive-stats-for-2018/ Looks like 10TB is the sweet spot in terms of reliability and size (Not cost tho ;).

Yea, the P2000 would be in a windows VM and passedthrough so it has no impact at the unraid level. It looks like with an iGPU and BMC, you have either/or. With a PCI-E vid card and BMC you can have both but it is driver dependent, but I've not tested this and have no intention too. If I can't get iGPU and BMC then I only have two paths: 1> accept IPMI is mostly pointless for me 2> I have to move away from iGPU

Confirmed, Supermicro has stated this is the expected behavior...you can't have a remote console and iGPU. I think I'll likely be in the market for a P2000 at some point later this year, looking toward the next release of Ryzen or TR. If I'm going to have all these limitations, I might as well go for PCI-E lanes so I can work around them....sad to say.

I'll keep you updated on the IPMI...and I also agree on the VM solution, it has some down-sides, but honestly...it is looking like a VM with a P2000 would have been a much better solution for this moment in time...then I could have remote console, and gone with any CPU I might like down the road

So I've been working with Supermicro on the IPMI problem, and bad news. Right now it seems you have a choice...use the BMC's Video adapter and get remote console /OR/ use the iGPU. If you use the IPMI video adapter, you don't seem to have access to the iGPU once the OS is booted... This is a REAL issue for me, one I'm working with their support team to sort out...but at this point, it looks like remote console OR hardware transcoding...but not both. I hope there is a way around that or this board was freaking pointless :).

Ok, been running a few days and it is working flawlessly. Thanks again for the update!

Thanks to the IPMI plugin I now have solid fan control, the update makes this board 100% functional for a quit unraid server! The only outstanding issue left is with the KVM setup. I can't get any console redirection to work via HTML5 or Java applet. THink I'm going to open a case with supermicro.

You need to set the lower thresholds for the fans with the sensor editor. The bmc will keep kicking them up to full speed. Set them to 200 or 300 rpms I had them fixed for all the headers (100/100/100)...it looks like the bmc reset is what was needed. I forgot I could do that from the UI!

Odd, I did a full reset...meaning not just a reboot but powering down the system and unplugging it to force the BMC to reset and it seems to work now! Sorry for the false alarm....testing to make sure each side responds properly ATM.

As a note, I set both the minimum PWM to 50%...as far as I know that should be fine for the FANs in my case....it was able to go lower with CPU control, so I don't think I'm hitting the erred state...it just feels like the IPMI full speed setting is being re-applied or something.

Ok, I re-wored my fan headers this morning, the system side fans are all in the "CPU" ports, and the drive side is in a "SYS" header. The result is all fans on full, it sounds like they are spinning down a bit when the timer hits for the IPMI plugin, but then they spool back up to full. Is there a special switch I need to dig out to stop the MB from resetting them to 100%? 2019-01-02 06:10:05 Starting Fan Control 2019-01-02 06:10:05 Setting fans to full speed 2019-01-02 06:10:15 Fan:Temp, FAN1234(40%):HDD Temp(28°C), FANA(40%):HDD Temp(28°C) 2019-01-02 06:11:26 fan control config file updated, reloading settings 2019-01-02 06:11:26 Fan:Temp, FAN1234(43%):CPU Temp(30°C), FANA(40%):HDD Temp(28°C) 2019-01-02 06:11:46 fan control config file updated, reloading settings 2019-01-02 06:11:46 Fan:Temp, FAN1234(46%):CPU Temp(33°C), FANA(48%):HDD Temp(28°C) 2019-01-02 06:12:46 Fan:Temp, FAN1234(47%):CPU Temp(34°C), FANA(48%):HDD Temp(28°C) 2019-01-02 06:13:06 fan control config file updated, reloading settings 2019-01-02 06:13:56 fan control config file updated, reloading settings 2019-01-02 06:13:56 Fan:Temp, FAN1234(61%):CPU Temp(33°C), FANA(48%):HDD Temp(28°C) 2019-01-02 06:14:16 Fan:Temp, FAN1234(59%):CPU Temp(31°C), FANA(48%):HDD Temp(28°C) 2019-01-02 06:15:57 Fan:Temp, FAN1234(60%):CPU Temp(32°C), FANA(48%):HDD Temp(28°C) 2019-01-02 06:16:17 Fan:Temp, FAN1234(60%):CPU Temp(32°C), FANA(45%):HDD Temp(27°C) 2019-01-02 06:16:47 Fan:Temp, FAN1234(59%):CPU Temp(31°C), FANA(45%):HDD Temp(27°C) 2019-01-02 06:17:27 Fan:Temp, FAN1234(62%):CPU Temp(34°C), FANA(45%):HDD Temp(27°C) 2019-01-02 06:17:37 Fan:Temp, FAN1234(59%):CPU Temp(31°C), FANA(45%):HDD Temp(27°C)

I just updated, looks good! I'll open the system and re-position the fans today or tomorrow and turn on control to test. Thanks for the quick update!

Confirmed! I've got a 250G NVME drive in the top slot by the CPU and a 4x card installed. There is also a drive on every on-board SATA port, though that shouldn't matter as it is an NVMe. As a note, I tried it in the other M.2 port and it DID knock out the 4x slot (So I had no net)

Waiting on the M.2 to come in...I don't have a handy spare one at the moment. Should have it this weekend.

This diagram from the manual says it better than I could

There is some question as to if that "~" is an or. As each M.2 slot IS 4x PCI-E..it wouldn't make sense to disable 8 channels because you are using a 4x slot. I believe this means that if you use PCI-E 4x slot then you can use ONE of the M.2 slots. If you use U.2 (4x PCI-E lanes) you can only use M.2#1 if you use BOTH U.2 and PCI-E 4x then you can't use either. I'm not sure about this, which is why I want to test it...but it would make no sense to disable both of them to use half the lanes for an expansion card

Same setting for all fans: Section 808_CPU_FAN1 ## Possible values: Yes/No Enable_All_Event_Messages Yes ## Possible values: Yes/No Enable_Scanning_On_This_Sensor Yes ## Possible values: Yes/No Enable_Assertion_Event_Lower_Critical_Going_Low Yes ## Possible values: Yes/No Enable_Assertion_Event_Lower_Non_Recoverable_Going_Low Yes ## Possible values: Yes/No Enable_Assertion_Event_Upper_Critical_Going_High Yes ## Possible values: Yes/No Enable_Assertion_Event_Upper_Non_Recoverable_Going_High Yes ## Possible values: Yes/No Enable_Deassertion_Event_Lower_Critical_Going_Low Yes ## Possible values: Yes/No Enable_Deassertion_Event_Lower_Non_Recoverable_Going_Low Yes ## Possible values: Yes/No Enable_Deassertion_Event_Upper_Critical_Going_High Yes ## Possible values: Yes/No Enable_Deassertion_Event_Upper_Non_Recoverable_Going_High Yes ## Give valid input for sensor type = Fan; units = RPM Lower_Non_Critical_Threshold 100.000000 ## Give valid input for sensor type = Fan; units = RPM Lower_Critical_Threshold 100.000000 ## Give valid input for sensor type = Fan; units = RPM Lower_Non_Recoverable_Threshold 100.000000 ## Give valid input for sensor type = Fan; units = RPM Upper_Non_Critical_Threshold 25300.000000 ## Give valid input for sensor type = Fan; units = RPM Upper_Critical_Threshold 25400.000000 ## Give valid input for sensor type = Fan; units = RPM Upper_Non_Recoverable_Threshold 25500.000000 ## Give valid input for sensor type = Fan; units = RPM; 'None' to not use hysteresis Positive_Going_Threshold_Hysteresis 100.000000 ## Give valid input for sensor type = Fan; units = RPM; 'None' to not use hysteresis Negative_Going_Threshold_Hysteresis 100.000000 EndSection

IOMMU group 0: [8086:3ec6] 00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers (rev 07) IOMMU group 1: [8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 07) [8086:1905] 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 07) [1000:0087] 02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05) IOMMU group 2: [8086:3e96] 00:02.0 VGA compatible controller: Intel Corporation Device 3e96 IOMMU group 3: [8086:1911] 00:08.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th Gen Core Processor Gaussian Mixture Model IOMMU group 4: [8086:a379] 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10) IOMMU group 5: [8086:a36d] 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10) [8086:a36f] 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10) IOMMU group 6: [8086:a368] 00:15.0 Serial bus controller [0c80]: Intel Corporation Device a368 (rev 10) [8086:a369] 00:15.1 Serial bus controller [0c80]: Intel Corporation Device a369 (rev 10) IOMMU group 7: [8086:a360] 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10) IOMMU group 8: [8086:a352] 00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10) IOMMU group 9: [8086:a340] 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0) IOMMU group 10: [8086:a32c] 00:1b.4 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 (rev f0) IOMMU group 11: [8086:a338] 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0) IOMMU group 12: [8086:a33d] 00:1c.5 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #6 (rev f0) IOMMU group 13: [8086:a33e] 00:1c.6 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #7 (rev f0) IOMMU group 14: [8086:a33f] 00:1c.7 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #8 (rev f0) IOMMU group 15: [8086:a328] 00:1e.0 Communication controller: Intel Corporation Device a328 (rev 10) IOMMU group 16: [8086:a309] 00:1f.0 ISA bridge: Intel Corporation Device a309 (rev 10) [8086:a348] 00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10) [8086:a323] 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10) [8086:a324] 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10) [8086:15bb] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-LM (rev 10) IOMMU group 17: [1d6a:d107] 04:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02) IOMMU group 18: [8086:1533] 06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) IOMMU group 19: [1a03:1150] 07:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 04) IOMMU group 20: [10e3:8113] 09:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (rev 01)

This new Supermicro board is turning out to be a little odd...so I thought I'd start a thread to note interesting information about it that come up as I refine my new deployment. 1> The VGA port is NOT tied to the intel processor, it is tied to a "BMC" video adapter 2> On first boot, I got no video from HDMI or Display port with no addon cards. Had to force the board to use the iGPU by changing from "Auto" to "IGFX" in the bios 3> The fans don't follow the normal SM standard. There are 2 zones, but rather than "FAN#" and "FANA" you get 2 CPU headers and 3 FAN headers. This is being looked at in the IPMI plugin thread to see if it can be supported. 4> My LSI card would not allow drives to mount when in the PCI-E slot nearest the CPU. I moved it and everything worked fine. I'm fairly sure this was one of those transient errors you get when building a new system, but I wanted to mention it. 5> IPMI seems to be losing remote console access, it starts up, but never shows the console screen. It has worked intermittently and having an HTML5 version is HEAVEN...when it works. 6> The fan thresholds are crap, I'll post my settings to precent the "Fan Cycling" condition in a reply. 7> I'm going to test an NVMe drive soon, the docs read that I should be able to us /1/ of the NVMe slots while using the 4x slot (10GB Ethernet card)...but I want to check that. 8> IOMMU groups are not "Bad" but not great. Audio is burried and there is a real lack of USB ports exposed.