brin6thepayne

Well wouldn't you know. My GPU kink was the culprit here too. I swapped some cards around and removed one, so the vfio addresses changed, and vfio bound the sata controller instead of where the old gpu was......

Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:07:00.0: version 3.0 Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:07:00.0: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl SATA mode Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:07:00.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs pio slum part Sep 15 20:41:06 BIG-BOSS kernel: scsi host1: ahci Sep 15 20:41:06 BIG-BOSS kernel: ata1: SATA max UDMA/133 abar m2048@0xf6d00000 port 0xf6d00100 irq 61 Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:0c:00.0: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl SATA mode Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:0c:00.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs pio slum part Sep 15 20:41:06 BIG-BOSS kernel: scsi host2: ahci Sep 15 20:41:06 BIG-BOSS kernel: ata2: SATA max UDMA/133 abar m2048@0xf7400000 port 0xf7400100 irq 63 Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:0d:00.0: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl SATA mode Sep 15 20:41:06 BIG-BOSS kernel: ahci 0000:0d:00.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs pio slum part Sep 15 20:41:06 BIG-BOSS kernel: scsi host3: ahci Sep 15 20:41:06 BIG-BOSS kernel: ata3: SATA max UDMA/133 abar m2048@0xf7300000 port 0xf7300100 irq 65 Sep 15 20:41:06 BIG-BOSS kernel: ata1: SATA link down (SStatus 0 SControl 300) Sep 15 20:41:06 BIG-BOSS kernel: ata3: SATA link down (SStatus 0 SControl 300) Sep 15 20:41:06 BIG-BOSS kernel: ata2: SATA link down (SStatus 0 SControl 300) Are you sure this says all SATA is down? This is on an ASUS X570 Pro-ACE which has 4x sata + u.s which can supply 4x sata via sas. Only using the four sata slots on the motherboard, nothing connected in the u.2 slot.

Updated to beta now. No dice. Attaching both diagnostics and log. Hey, JorgeB thanks for taking your time to look at this with me. I'm really pulling my hair out after spending an entire day working out a gpu kink only for this to start after that issue was resolved. big-boss-diagnostics-20200915-2042.zip big-boss-syslog-20200915-1842.zip

Yeah it's weird

EDIT: It was vfio binding Hello. On a reboot all my drives disappeared. This has happened earlier, so I rebooted (this fixed it last time) but it didn't work. They are not visible as unnasigned devices. They are visible under SATA in my bios. I've reseated the sata power and data connectors on both ends. No errors in log. I'm dumbfounded. big-boss-syslog-20200915-1815.zip

EDIT: I got exactly zero replies on this, but figured out my issue was unraid grabbing my gpu for gui output since my new build doesnt have igpu. It was a common error, I found it by googling segments of the repeated log error one at a time.

Diagnostics.zip

This might look like a mess, but it's from the vfio page in settings. Group 25 is the GTX 1070 which I think is the cause of these issues. IOMMU Address Bound Reset Vendor ID Description Group 0 00:01.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 1 00:01.2 1022:1483 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge Group 2 00:02.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 3 00:03.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 4 00:03.1 1022:1483 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge Group 5 00:04.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 6 00:05.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 7 00:07.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 8 00:07.1 1022:1484 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] Group 9 00:08.0 1022:1482 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge Group 10 00:08.1 1022:1484 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] Group 11 00:08.2 1022:1484 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] Group 12 00:08.3 1022:1484 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] Group 13 00:14.0 1022:790b SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) 00:14.3 1022:790e ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) Group 14 00:18.0 1022:1440 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 00:18.1 1022:1441 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 00:18.2 1022:1442 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 00:18.3 1022:1443 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 00:18.4 1022:1444 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 00:18.5 1022:1445 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 00:18.6 1022:1446 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 00:18.7 1022:1447 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 Group 15 01:00.0 1022:57ad PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream Group 16 02:00.0 1022:57a3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge Group 17 02:03.0 1022:57a3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge Group 18 02:04.0 1022:57a3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge Group 19 02:08.0 1022:57a4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge 06:00.0 1022:1485 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP 06:00.1 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller USB devices attached to this controller: Bus 002 Device 002: ID 2516:0051 Cooler Master Technology Inc. AMD SR4 lamplight Control Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub 06:00.3 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller USB devices attached to this controller: Bus 004 Device 003: ID 1b1c:1b50 Corsair CORSAIR K63 Wireless USB Receiver Bus 004 Device 002: ID 046d:c53f Logitech, Inc. USB Receiver Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Group 20 02:09.0 1022:57a4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge 07:00.0 1022:7901 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Group 21 02:0a.0 1022:57a4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge 08:00.0 1022:7901 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Group 22 03:00.0 10de:128b VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1) 03:00.1 10de:0e0f Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1) Group 23 04:00.0 8086:1539 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) Group 24 05:00.0 10ec:816e Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. Device 816e (rev 1a) 05:00.1 10ec:8168 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 1a) 05:00.2 10ec:816a Serial controller: Realtek Semiconductor Co., Ltd. Device 816a (rev 1a) 05:00.4 10ec:816d USB controller: Realtek Semiconductor Co., Ltd. Device 816d (rev 1a) USB devices attached to this controller: Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub 05:00.7 10ec:816c IPMI Interface: Realtek Semiconductor Co., Ltd. Device 816c (rev 1a) Group 25 09:00.0 10de:1b81 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) 09:00.1 10de:10f0 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) Group 26 0a:00.0 1022:148a Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function Group 27 0b:00.0 1022:1485 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP Group 28 0b:00.1 1022:1486 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP Group 29 0b:00.3 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller USB devices attached to this controller: Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 007 Device 002: ID 0781:5583 SanDisk Corp. Ultra Fit Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Group 30 0b:00.4 1022:1487 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller Group 31 0c:00.0 1022:7901 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Group 32 0d:00.0 1022:7901 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

I have recently migrated from old hardware, and in setting up my new server I'm faced with a VM issue. My gaming Windows VM won't boot, it goes into repair mode but as usual can't repair itself. Through some troubleshooting I've found out it's my GTX 1070 doing some buggering, but I don't know what buggering is happening. It's bound through VFIO (with the VFIO-PCI CFG plugin) and is alone in the IOMMU group only with the soundcard on the graphics card (0000:09:00.0 and .1). My log is filled with endless lines of this: Sep 12 17:04:44 <HOSTNAME> kernel: vfio-pci 0000:09:00.0: BAR 1: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref] Sep 12 17:04:44 <HOSTNAME> kernel: vfio-pci 0000:09:00.0: BAR 1: can't reserve [mem 0xd0000000-0xdfffffff 64bit pref] Some things: 1) It worked on my old rig with VFIO binding in Syslinux configuration 2) No changes made to the vm .xml in migration other than re-picking the gpu with its new pcie address 3) The VM works with my secondary gpu (GT 710) which is bound in the same way, and in it's own IOMMU group 4) Tried with a backup vdisk image to no prevail. I'm doing more troubleshooting tomorrow, but I'm at a loss from where to go from here. Going to try VFIO in Syslinux like my old system had, but I can't see how that should make a difference.

Turned off bonding and bridging on eth1 and adding eth1 to br0 made br0 the default interface :shrug: Is there a way to mark this solved or something?

# route outputs Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default X.X.X.1 X.X.X.0 UG 0 0 0 br1 So the br1 interface is default (bonded to eth1). My goal would be to set br0 as default. Alternatively add eth0 to br1 (but that feels alot like using duct tape) So, does anyone know how to change default interface?

I moved most of my server functions off my server and onto bare metal for the 24/7 operated things, so I could tinker about without ruining something vital for the network. I did a clean install and reconfigured the network according to screenshots I took beforehand (it's identical to what it used to be, I've quadruple checked). Everything worked exactly as it should before the reset configured exactly the same. I have a 10Gb/s PCIe network card and that's the connection I've set up and used. The only cable between the switch and the server is CAT6 in the PCIe RJ45 jack. My problem now when trying to f.ex ping 8.8.8.8 the command window tries to ping through the eth1 (motherboard) 192.x.x.69 instead of eth0 (PCIe) 192.x.x.60 like it used to. Why? Isn't eth0 the first priority? System info Command line