Endy

I've been struggling to get Bitwarden to work. The docker is installed and I can access it by IP, but not by reverse proxy. My ISP blocks port 80, but not port 443 so I am using Cloudflare to force 443. First I followed SpaceInvader's video and was using SWAG. I just get an Error 522 from Cloudflare. (Everything else through SWAG was working just fine.) Then I switched everything over to using HAProxy in Pfsense. Instead of using the custom network, I gave each of the docker containers their own static IP addresses. Again, I can get everything working, but Bitwarden gets Error 522 from Cloudflare. HAProxy stats shows that Bitwarden is up. I've searched and searched, but I can not find an answer. Any ideas?

I've had SWAG working for awhile now (For Nextcloud, Emby, etc.), but I am trying to get it to work for Bitwarden, but I am getting a 522 error. Cname has been created and added to SWAG for the LE certificate. Bitwarden container is on the same proxynet as SWAG. I can reach Bitwarden using the localip:port, but not by domain name. I'm using the default bitwarden.subdomain.conf file with a minor change to location /admin. I changed the Bitwarden container name to just bitwarden from bitwardenrs to match the file. # make sure that your dns has a cname set for bitwarden and that your bitwarden container is not using a base url # make sure your bitwarden container is named "bitwarden" # set the environment variable WEBSOCKET_ENABLED=true on your bitwarden container server { listen 443 ssl; listen [::]:443 ssl; server_name bitwarden.*; include /config/nginx/ssl.conf; client_max_body_size 128M; # enable for ldap auth, fill in ldap details in ldap.conf #include /config/nginx/ldap.conf; # enable for Authelia #include /config/nginx/authelia-server.conf; location / { # enable the next two lines for http auth #auth_basic "Restricted"; #auth_basic_user_file /config/nginx/.htpasswd; # enable the next two lines for ldap auth #auth_request /auth; #error_page 401 =200 /ldaplogin; # enable for Authelia #include /config/nginx/authelia-location.conf; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; set $upstream_app bitwarden; set $upstream_port 80; set $upstream_proto http; proxy_pass $upstream_proto://$upstream_app:$upstream_port; } location /admin { return 404; } location /notifications/hub { include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; set $upstream_app bitwarden; set $upstream_port 3012; set $upstream_proto http; proxy_pass $upstream_proto://$upstream_app:$upstream_port; } location /notifications/hub/negotiate { include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; set $upstream_app bitwarden; set $upstream_port 80; set $upstream_proto http; proxy_pass $upstream_proto://$upstream_app:$upstream_port; } } I'm drawing a blank on what to try next.

The 10TB WD Elements are on sale for $149 today for those who subscribe to Neweggs promo emails.

I just (11/10) picked up a couple of 10TB WD Elements for $149 each from Newegg. I think Amazon may have had them for the same price at the same time as well.

I have a Dell H310 flashed to IT mode. I just plugged it in and it worked. It was previously used in my old server using forward breakout cables and now it's connected to the backplane in my CSE-846.

@mafoster I took a quick look and some of it is out of my realm of experience because I have not tried to set up a vm for Catalina or using an AMD video card. There is one thing I did see. It looks like you were able to stub the video card, but you may want to follow this video: https://www.youtube.com/watch?v=QlTVANDndpM&t=601s Be aware that changes made to the vm in xml are overwritten if you make any other changes using the gui view.

I am not even seeing bluetooth listed in the devices (entirely possible it's there and I am just not seeing it) and I don't use it, so I don't think I am gonna be much help there. The problem with the onboard audio is the same as with that 3rd usb device. It may be possible to get it working, but I haven't tried it. More info here:

I think there have been a couple. One by jonp and one by Spaceinvader One. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/General Storage/NVIDIA.GTX1080Ti.11264.170118.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </hostdev> I think this is your problem. You left off multifunction='on'. So it should look like this: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/General Storage/NVIDIA.GTX1080Ti.11264.170118.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/> </hostdev> And then you may or may not need to change the Bus of the last device because of the number gap from Bus 4 to Bus 6. I'm not sure how that is handled by the vm and it may not be a problem at all. I know that I have seen it somewhere, but I just don't recall where.

I didn't really go into actually setting up Windows and installing drivers. Spaceinvader One has a video for that as well. But basically this is where the VirtIO drivers come in and I'd have to watch the video again to tell you what exactly is needed. Yes, the slots were the same. On a Windows 7 vm, the bus was the same but the slots were different, just like in the video. I do not know why that is. But basically, because the graphics and sound are one device and that's the way we want them to be seen by the vm, both the bus and the slot need to match. This is one of those things that I know I have dealt with in the past, but my memory fails me to the specifics. Again, I think this may be in the video for setting up a Windows 10 vm. I think I have also recently read a post about that here on the forum somewhere? I would say with the new vfio-pci.cfg plugin, it is much much easier now to determine which usb port is which and plan accordingly. But it's also good to have options.

I’ve seen a lot of people asking questions and having problems getting graphics cards and onboard usb controllers passed through on Ryzen X570 systems. I had some issues myself that thanks to a couple of people here on the forums I was able to get resolved. This guide is about setting up a Windows 10 vm using a single graphics card with onboard usb passed through. Because I have managed to get it working, I thought it might be a good idea to write a little guide based on my setup in case it can help some other people. While this was specific to my motherboard and hardware, there are parts that should be applicable for other hardware combinations. Let’s start with my hardware. I’m using a Ryzen 7 3800X with the AsRock X570 Taichi. The graphics card is a Geforce GTX 1070. BIOS First step is to make sure the bios is up to date. At the time I am writing this the newest bios for my motherboard is 2.70 dated 12-9-2019. This is the AGESA version: Update AMD AGESA Combo-AM4 1.0.0.4 Patch B Once updated, boot into the bios once then restore default settings, save, reboot and go back into the bios. To enable IOMMU I found this at level1techs: https://forum.level1techs.com/t/x570-taichi-iommu-groups/145762/2 After some trial and error, I found that in order to passthrough the motherboard usb controllers, I had to change some usb options from Auto to Enabled. Because there is one controller that can’t be passed through, not all of these probably need to be changed, but I’m not sure which ones are for which controllers. Doesn’t seem to be causing me any issues at the moment. Go to Advanced>AMD CBS>FCH Common Options>USB Configuration Options and change XHCI Controller to Enabled. While there go to MCM USB enable and set XHCI2 enable and XHCI3 enable to Enabled. Next is Advanced>AMD CBS>Chipset Common Options>Chipset USB Configuration Options. Change XHCI Controller0 enable and XHCI Controller1 enable to Enabled. The last thing is to set the boot device to the Unraid usb non uefi. (Supposedly booting the usb using uefi can cause problems, but I have not experienced those myself so I can’t comment on that. This is the better safe than sorry approach.) Save settings and reboot. Isolating Cores This next step may be considered optional, but it’s about making sure the vm has the best chance of performing as best it can. In Unraid go to Settings>CPU Pinning. At the bottom you can choose which CPU cores you would like to isolate. Unraid like to use core 0, so I stay away from that one and usually start at the other end. Keep in mind to match the hyperthreads together. In my case I went with cores 4-7 and the matching hyperthreads 12-15. Syslinux Next, in Unraid, go to the Main tab and click on Flash then scroll down to Syslinux Configuration. To edit these click on Menu View and it will change to Raw View. I like to have a separate menu option for hardware passthrough so I just copy everything for ‘label Unraid OS’, delete ‘menu default’ under ‘label Unraid OS’, then paste in the new menu item with the other menu items. Make sure to rename it and that it says ‘menu default’ so that’s the one that Unraid will boot into automatically. The only other change is to add ‘video=efifb:off’ after ‘ initrd=/bzroot’. I admit that I am not sure on the details of why this is necessary, but I believe it has to do with passing through the only graphics card in the system. label Unraid OS HWPT menu default kernel /bzimage append isolcpus=4-7,12-15 initrd=/bzroot video=efifb:off vfio-pci.cfg To stub the graphics card and usb controller I highly recommend using the VFIO-PCI CFG plugin. So far it works really well and makes things much easier. Plus, it gives useful information like if your device can be reset and also which devices are attached to which usb controllers (very useful for figuring out which usb ports belong to which usb controllers). The plugin can be found in Community Apps. Once installed you can find it in the Settings tab. Go there and it’s a good time to double check that the IOMMU groups are looking like they should. Then it’s as easy as selecting which devices you want to stub. Check the graphics card and it’s audio controller. For the onboard usb controllers, it’s tempting to try using the one that is off on in it’s own group. In my case that was this one: Group 33 10:00.3 [RESET] 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller However, that one will not work and was one of the problems I ran into, but thanks to an explanation by @Skitals it makes sense why that one would not work. The address is Bus:Device:Function. Notice that the Function for that usb controller is 3. That means that despite being in it’s own IOMMU group, it belongs to another device and therefore, can’t be passed through. Good thing there are two other usb controllers on this motherboard in this group: Group 20 03:08.0 [RESET] 1022:57a4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 0a:00.0 [RESET] 1022:1485 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP 0a:00.1 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller 0a:00.3 [RESET] 1022:149c USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller Select the three Oa:00 devices. Make sure the usb drive that Unraid uses is not in one of the usb ports for those two controllers and then reboot. For reference, here’s a picture that shows the back panel of the X570 Taichi. The ports colored in red belong to the usb controller that’s off in it’s own group. The yellow and green colored ports belong to the two other usb controllers. I have not yet tried to figure out which controllers the mother usb headers belong to. vm xml – graphics card Before creating the vm, we’ll need a dump of the vbios. While it is possible to download from techpowerup, that may or may not be the right vbios for your exact card. I found it was easiest to put Hiren’s BootCD on a usb drive and swap that with the unraid usb drive, boot into that, dump the vbios with GPUZ and save that to the usb drive. Take that usb drive to another computer and edit the vbios with a hex editor to remove the header as seen in this Spaceinvader One video: https://www.youtube.com/watch?v=1IP-h9IKof0&t=438s Put the Unraid usb drive back in the server and start it up. (Now is a good time to make sure it’s still booting into Legacy mode and not uefi as I noticed mine had switched at this point.) Put the vbios file somewhere on your Unraid server so that the vm will be able to access it. (Maybe a vbios folder in with your ISO folders.) Time to create a Windows 10 vm. Go to the VMS tab, Add VM, Windows 10. I’ll just mention the things that I actually change and the rest of the settings will stay at defaults. Logical CPUs - change this to match the the cores you isolated earlier Initial and Max Memory – change these to the desired amount of RAM for the vm, I’m using 8192 MB for both Machine – Q35 OS Install ISO – point this to your Windows 10 install ISO VirtIO Drivers ISO – might want to make sure you have the latest version (virtio-win-0.1.171.iso as of the time I am writing this) They can be found here under Direct downloads virtio-win iso: https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/index.html Note: You can set your default virtio iso in Settings>VM Manager. Graphics Card – set to your graphics card Graphics ROM BIOS – point this to the vbios file you got from your graphics card earlier Sound Card – set to the graphics card audio controller Other PCI Devices – I had some issues with this but so far I have found two ways that seem to work. The first is to just select the usb controller that ends in .3. (In the reference photo above, I believe it’s the usb ports colored in yellow.) The other way is to select the 2 usb controllers and the non-essential instrumentation. Then the xml also needs to be edited which will be explained below. Uncheck Start VM after creation Click Create Now the xml of the vm needs to be edited, so go in to edit it and click on Form View to change it to XML View. Now we need to follow this video by Spaceinvader One. There is one thing I noticed that was different for my setup and that was that it was the Bus that was different and not the Slot, so I went ahead and changed the Bus to match. https://www.youtube.com/watch?v=QlTVANDndpM&t=601s Usb Controller If both usb controllers were selected, then the xml needs to be edited in a similar fashion as to the way the graphics card xml was edited. Here’s mine for an example: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x2'/> </hostdev> I should also mention that I changed the Bus since before it was audio was 5, non-essentials device was 6, usb controller was 7 and usb controller was 8. With the audio moved to Bus 4 with the graphics card, I changed the other 3 devices to the now vacated Bus 5. That should about cover it. If you are trying to use a previous vm and it doesn’t start and no error comes up and it’s just nothing on the screen, try removing the vm and recreating it. (Thanks to @testdasi for that one.)

After having tried a bunch of different DVR options, Emby has become my favorite, but yes, it does require Emby Premier, but includes guide data. (Well worth the lifetime imo.) Easy to setup and use. There are other options like SageTV and TVHeadend. They all were much harder to setup and the interface's left something to be desired.

I have an AsRock X570 Taichi and what I had to do to get the onboard usb to passthrough was to go into the bios and switch all of the usb options from Auto to Enabled.

Yes, if you boot into GUI mode, then you need to add it there.

Thanks for this. It's starting to make sense. In all my searching I would keep finding these things that people say to do, but usually not with any explanations as to why. Yes, that was already done. Kind of like I was saying before, I saw it mentioned somewhere to do it, but no explanation of why. Thank you both for the help. So far everything is now working.

Ok so I just added the 0a:00.3 usb controller to the vm and now I can get it to start with vnc. The message that comes up is: Guest has not initialized the display (yet). @testdasi like you said, I deleted the template and started again. Now it's working. Thank you. I still need to install the nvidia driver and make sure that works.

There was no display and I did get the vbios from my card, I did not download it from techpowerup.

Ok, trying to start with just vnc instead of the graphics card I get this

Success! The vm now starts and Unraid doesn't lock up. I am still having the no video problem. @testdasi I assume what I need to do is to use vnc to setup the vm and then try to add the graphics card afterwards so that then I could RDP into it? Mine is the same (except that it's 0a:00.0 and oa:00.1). What about 03:08.0 [RESET] 1022:57a4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 since that is in the same iommu group? I think what I typically read is that everything in a group needs to be passed through? So far it seems to be working without that passed through.

I did my best to map out which usb ports go to which motherboard controller. USB devices connected to the controller I was trying to pass through no longer show up in System Devices, but the Unraid USB stick does. Yes, the USB controller can be reset. It looks like Skitals might have the answer for me on this one. I dumped the bios myself and removed the header as shown in SpacerInvaderOne's video. Being a fresh vm, I haven't been able to get in far enough to actually install Windows 10 so no RDP. That makes a lot of sense. I will try that out and report back.

I've recently switched hardware and so I've started to create a new Windows 10 vm from scratch to make sure that I can get passthrough working. It looks like there are 2 problems. If I try to start the vm with passed through usb, the vm doesn't start and then it locks up the whole Unraid server. If I try to start the vm with just the video card passed through and not the usb, I don't seem to be getting any video. Hardware specs in sig. This is the only video card in the system and I am trying to passthrough a motherboard usb controller. They are both in their own groups. IOMMU group 28: [10de:1b81] 0d:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) [10de:10f0] 0d:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) IOMMU group 33: [1022:149c] 10:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller I am using the vfio-pci.cfg file method. BIND=10:00.3 0d:00.0 0d:00.1 vm xml <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10 Test</name> <uuid>8fe401f7-9b18-2ef5-d565-8a916dc0a78c</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='4'/> <vcpupin vcpu='1' cpuset='12'/> <vcpupin vcpu='2' cpuset='5'/> <vcpupin vcpu='3' cpuset='13'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='14'/> <vcpupin vcpu='6' cpuset='7'/> <vcpupin vcpu='7' cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-4.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/8fe401f7-9b18-2ef5-d565-8a916dc0a78c_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10 Test/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Data/ISO/Windows 10/Windows1909.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Data/ISO/virtio-win-0.1.171.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x9'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0xa'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0xb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0xc'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/> </controller> <controller type='pci' index='8' model='pcie-to-pci-bridge'> <model name='pcie-pci-bridge'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:a3:f7:80'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/domains/vbios/GTX1070.rom'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0d' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x10' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain> Relevant Syslinux section. I'm not sure if the video=efifb:off is necessary, it was just something I was trying. label unRAID OS menu default kernel /bzimage append isolcpus=4-7,12-15 initrd=/bzroot video=efifb:off mitigations=off This was in the system log file. I'm not sure what's relevant besides the vfio-pci not ready messages. Jan 15 15:21:03 Turtle kernel: br0: port 2(vnet0) entered blocking state Jan 15 15:21:03 Turtle kernel: br0: port 2(vnet0) entered disabled state Jan 15 15:21:03 Turtle kernel: device vnet0 entered promiscuous mode Jan 15 15:21:03 Turtle kernel: br0: port 2(vnet0) entered blocking state Jan 15 15:21:03 Turtle kernel: br0: port 2(vnet0) entered forwarding state Jan 15 15:21:08 Turtle kernel: clocksource: timekeeping watchdog on CPU10: Marking clocksource 'tsc' as unstable because the skew is too large: Jan 15 15:21:08 Turtle kernel: clocksource: 'hpet' wd_now: 9f7787c3 wd_last: 9e931f84 mask: ffffffff Jan 15 15:21:08 Turtle kernel: clocksource: 'tsc' cs_now: a5d3c006266 cs_last: a5ccefb201b mask: ffffffffffffffff Jan 15 15:21:08 Turtle kernel: tsc: Marking TSC unstable due to clocksource watchdog Jan 15 15:21:08 Turtle kernel: TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. Jan 15 15:21:08 Turtle kernel: sched_clock: Marking unstable (2886087668830, -16976841)<-(2886210465096, -139776844) Jan 15 15:21:08 Turtle kernel: clocksource: Switched to clocksource hpet Jan 15 15:21:16 Turtle kernel: vfio-pci 0000:10:00.3: not ready 1023ms after FLR; waiting Jan 15 15:21:18 Turtle kernel: vfio-pci 0000:10:00.3: not ready 2047ms after FLR; waiting Jan 15 15:21:21 Turtle kernel: vfio-pci 0000:10:00.3: not ready 4095ms after FLR; waiting Jan 15 15:21:26 Turtle kernel: vfio-pci 0000:10:00.3: not ready 8191ms after FLR; waiting Jan 15 15:21:35 Turtle kernel: vfio-pci 0000:10:00.3: not ready 16383ms after FLR; waiting Jan 15 15:21:53 Turtle kernel: vfio-pci 0000:10:00.3: not ready 32767ms after FLR; waiting Anything I left out and need to add? I'm not sure what to try next.

I ran pfsense as a vm for a little while. It became a pain whenever the server needed rebooting, especially if it was rebooting for troubleshooting that might be several times. Once everything was up and running, it was just fine. It could also be a pain if there was a problem with the server that would prevent the pfsense vm from running because then there would be no internet. Ultimately I decided that the added complication wasn't worth it. I ended up getting a Qotom pc to run pfsense. That was 3 years ago and it's been running great since.

That's pretty much exactly what I did. Put my CSE-846 on it's side. However, I had no idea that anyone was making a little cart like that so I made my own that actually looks rather similar. (Except mine is made out of wood.)

If I have to choose only one thing that I like most, I'll go with what made me choose Unraid in the first place, and that's the ability to easily add drives of different sizes or brands or whatever. I'm really not sure what I would like to see added in 2020 because Unraid already does everything I need it to do. I guess I'll go with multiple arrays and/or multiple cache pools. To the LimeTech team and to all the forum members helping other members, keep up the great work!

Yes you can, that's what I did. You can use a SAS card and then connect that to an expander backplane. I used a Dell H310 flashed to IT mode and the backplane that came with my particular SM chassis is the BPN-SAS2-846EL1. Someone more knowledgeable than me can tell you what else works or doesn't. If you went this route, I don't see any reason why that wouldn't work.

I don't think I'll be trying that. I do think that I should probably add fans to the front of the case to see about getting more air moving across the hard drives. I measured the air coming out of the back of the case as best as I could and it's showing around 34 degrees.