pottlepaul

I am not sure if it would be helpful to anyone, but I moved the system share from the cache to the array as a temporary measure and it seems to have helped a lot. I went from an average of 160 GB / day of write to around 60 GB / day. Not perfect, but there is a good chance that is what my dockers are actually using as I have a lot running. Just wanted to share what I found to prevent beating on my SSDs a little.

I am also seeing the problem. unRaid 6.8.3 Intel 1TB 660Op installed 7/2019 Power on hours: 6451 Data Written: 41.2 TB Cache only used for dockers and VMs.

First off, thanks for any help with this. I am in the process of moving everything from a ESXi host to unRaid and so far am really impressed with the software. I am now trying to set up a pfSense VM and want to pass through one of my quad NICs to it. I have tried adding this to the XML but I get an error and the VM won't boot. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/> </source> </hostdev> This is the error I get: Execution error internal error: process exited while connecting to monitor: 2017-08-30T20:03:15.375733Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: vfio: error, group 15 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver. 2017-08-30T20:03:15.375774Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 15 2017-08-30T20:03:15.375799Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: Device initialization failed I assume this is because the NIC is tied in to unraid. I have tried to adding my NIC to pci-stub, but due to all four NICs having the same vendor info (14F1:880) I loose all four NICs on reboot. I have done a bit of searching and am having a hard time finding a way around this. Is it possible to only withhold 1 of the 4 NICs? Any ideas would be fantastic. My system devices output is below and I have bolded the NIC I am trying to pass through. Let me know if I can add anything else that would be helpful. PCI Devices and IOMMU Groups IOMMU group 0 [8086:3406] 00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 13) IOMMU group 1 [8086:3408] 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) IOMMU group 2 [8086:340a] 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) IOMMU group 3 [8086:340b] 00:04.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 4 (rev 13) IOMMU group 4 [8086:340c] 00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13) IOMMU group 5 [8086:340d] 00:06.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 6 (rev 13) IOMMU group 6 [8086:340e] 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) IOMMU group 7 [8086:3410] 00:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev 13) IOMMU group 8 [8086:342e] 00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13) [8086:3422] 00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) [8086:3423] 00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) IOMMU group 9 [8086:3430] 00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:3431] 00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:3432] 00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:3433] 00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:3429] 00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:342a] 00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:342b] 00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) [8086:342c] 00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) IOMMU group 10 [8086:2937] 00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02) [8086:2938] 00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02) [8086:293c] 00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02) IOMMU group 11 [8086:2934] 00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) [8086:2935] 00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) [8086:293a] 00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) IOMMU group 12 [8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92) [102b:0532] 08:03.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a) IOMMU group 13 [8086:2918] 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02) IOMMU group 14 [14e4:1639] 01:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20) [14e4:1639] 01:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20) IOMMU group 15 [14e4:1639] 02:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20) [14e4:1639] 02:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20) IOMMU group 16 [1000:0060] 03:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 04) IOMMU group 17 [1000:0072] 05:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03) IOMMU group 18 [14f1:8880] 06:00.0 Multimedia video controller: Conexant Systems, Inc. CX23887/8 PCIe Broadcast Audio and Video Decoder with 3D Comb (rev 0f) IOMMU group 19 [8086:2c40] fe:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05) [8086:2c01] fe:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05) IOMMU group 20 [8086:2c10] fe:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05) [8086:2c11] fe:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05) [8086:2c14] fe:02.4 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 1 (rev 05) [8086:2c15] fe:02.5 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 1 (rev 05) IOMMU group 21 [8086:2c18] fe:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05) [8086:2c19] fe:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05) [8086:2c1a] fe:03.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller RAS Registers (rev 05) [8086:2c1c] fe:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05) IOMMU group 22 [8086:2c20] fe:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05) [8086:2c21] fe:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05) [8086:2c22] fe:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05) [8086:2c23] fe:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05) IOMMU group 23 [8086:2c28] fe:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05) [8086:2c29] fe:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05) [8086:2c2a] fe:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05) [8086:2c2b] fe:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05) IOMMU group 24 [8086:2c30] fe:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05) [8086:2c31] fe:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05) [8086:2c32] fe:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05) [8086:2c33] fe:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05) IOMMU group 25 [8086:2c40] ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05) [8086:2c01] ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05) IOMMU group 26 [8086:2c10] ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05) [8086:2c11] ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05) [8086:2c14] ff:02.4 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 1 (rev 05) [8086:2c15] ff:02.5 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 1 (rev 05) IOMMU group 27 [8086:2c18] ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05) [8086:2c19] ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05) [8086:2c1a] ff:03.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller RAS Registers (rev 05) [8086:2c1c] ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05) IOMMU group 28 [8086:2c20] ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05) [8086:2c21] ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05) [8086:2c22] ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05) [8086:2c23] ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05) IOMMU group 29 [8086:2c28] ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05) [8086:2c29] ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05) [8086:2c2a] ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05) [8086:2c2b] ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05) IOMMU group 30 [8086:2c30] ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05) [8086:2c31] ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05) [8086:2c32] ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05) [8086:2c33] ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05) CPU Thread Pairings cpu 0 <===> cpu 8 cpu 1 <===> cpu 9 cpu 2 <===> cpu 10 cpu 3 <===> cpu 11 cpu 4 <===> cpu 12 cpu 5 <===> cpu 13 cpu 6 <===> cpu 14 cpu 7 <===> cpu 15 USB Devices Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub Bus 001 Device 003: ID 0781:5530 SanDisk Corp. Cruzer Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 005 Device 002: ID 0624:0248 Avocent Corp. Virtual Hub Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub SCSI Devices [0:0:0:0] disk SanDisk Cruzer 1100 /dev/sda 8.00GB [1:0:32:0] enclosu DP BACKPLANE 1.07 - - [1:2:0:0] disk DELL PERC 6/i 1.22 /dev/sdb 639GB [2:0:0:0] enclosu DELL MD1000 A.04 - - [2:0:1:0] disk ATA HITACHI HUA7210S AC4A /dev/sdc 1.00TB [2:0:2:0] disk ATA SAMSUNG HD103UJ 1113 /dev/sdd 1.00TB [2:0:3:0] disk ATA HITACHI HUA7210S AC4A /dev/sde 1.00TB [2:0:4:0] disk ATA HITACHI HUA7210S AC4A /dev/sdf 1.00TB [2:0:5:0] disk ATA HITACHI HUA7210S AC5A /dev/sdg 1.00TB [2:0:6:0] disk ATA HITACHI HUA7210S AC4A /dev/sdh 1.00TB [2:0:7:0] disk ATA HITACHI HUA7210S AC4A /dev/sdi 1.00TB [2:0:8:0] disk ATA HITACHI HUA7210S AC4A /dev/sdj 1.00TB