FCruz2489

Hi! Just wanted to provide an update - since I added the line you suggested to the config AND changing all my PCI-e and NVME slots to disable aspm in the bios, the drives have been solid for a week. I'll continue to monitor it, but this is very promising and I hope it did the trick. Thank you again!

Sadly, one of the drives disappeared with this setting enabled while i was running the cach pool BTRFS scrub/repair. But they are still connected via the Highpoint card so maybe I need to put them back into the M.2 slots with this new setting.

Thank you so much for the response! I added that to my flash config and am rebooting the server now. My docker and VM images are jacked again, so I will reconfigure them with this setting enabled and see if all drives can finally be reliable. If I wanted to add this to the GUI boot option as well, would it be the text below or do I need a comma? append initrd=/bzroot,/bzroot-gui nvme_core.default_ps_max_latency_us=0 pcie_aspm=off

Hi All, Sorry for the long post, but I have been experiencing some very strange SSD issues since I upgraded my platform from an i9-10900 to a i7-12700. Its been driving me mad. I had a stable Z490/DDR4/i9-10900 server which I migrated to B660/DDR5/i7-12700 and then W680/DDR5/i7-12700. I know SSDs are technically not supported in the array, but I am not using parity so I think its ok for my purposes (also understanding that my data is unprotected). ARRAY: TWO Intel P4510 8TB U.2 NVME drives - NO PARITY CACHE: TWO WD SN850X 1TB M.2 NVME drives - BTRFS RAID 1 ARRAY DRIVE ISSUES: Upon moving to the new platform (ASUS ROG STRIX B660-I) I started experiencing issues where the NVME drives would randomly drop out. CACHE drives were connected direct to the motherboard M.2 slots and ARRAY drives were connected via a Highpoint SSD7104 PCI-e 3.0 x16 RAID card with U.2 to M.2 adapters because of ITX motherboard limitation. CACHE drives were solid/stable, but the ARRAY drives would drop off randomly - only one at a time, never both. The server would also lock up and need a hard restart every few days. The motherboard was experiencing some coil whine as well so the entire situation drove me to replace the motherboard. CACHE DRIVE ISSUES: Now I have everything in a SUPERMICRO MBD-X13SAE-F-O W680 setup. The CACHE drives were connected straight to the motherboard via the M.2 slots and the ARRAY drives are now connected via U.2 to PCI-e 3.0 x4 adapters - got rid of the highpoint pci-e card since I have additional pci-e slots now. Memory in this system is NON-ECC Team T-Force Vulcan 32GB (2 x 16GB) DDR5 FLBD532G5600HC36BDC01. Everything seemed to be running fine and stable until after a few days the CACHE drives started disappearing and I was getting nvme drive btrfs errors in my system log. Only one drive would disappear, not both. I tried setting the pci-e link speed on the m.2 drives to 4.0 manually instead of AUTO - even though the slots on the motherboard and the drives are both 4.0. Worked fine for a bit and then one drive would drop out. Did some digging on the forum and people mentioned this could be memory related issue so I set the memory to default settings. Single CACHE drive dropped and error logs came back. At this point I figured it was something with the M.2 slots on the motherboard so I reintroduced the Highpoint SSD7104 into the system and put the CACHE drives on there. Again, everything worked fine for a bit and then one CACHE drive dropped off. I tried setting the PCI-e 5.0 slot on the motherboard to 3.0 manually to match the Highpoint SSD7104 card. Worked fine for a bit and single drive gone again. At this point I assume the Highpoint SSD7104 card might be the culprit since originally the ARRAY drives were dropping off connected to it (via the U.2 to M.2 adapters) and now the CACHE drives are dropping off connected to it. However, this doesn't explain why the CACHE drives were dropping off while connected to the M.2 slots on the SUPERMICRO MBD-X13SAE-F-O directly. As it stands, I don't know if I need to replace the Team Group DDR5 memory with some ECC memory/different memory or if the WD SN850X drives are not good as CACHE drives on this motherboard. I have had to reconfigure all my dockers and my docker image numerous times in the past few weeks. All of these drives are super expensive and have worked reliably in the past on the older DDR4 platform. I am hoping someone can take a look at my diagnostics and give me an idea of where to start on fixing this. I have done some research on ASPM to see if that could be causing issues with the NVME drives but I don't know what settings to change, if any. Thank you in advance for reading and any input at all! fc-unraid-diagnostics-20230215-1022.zip

I purchased it from Newegg in the past month and it came with 2.0 installed. I didn’t install a 13th gen CPU, but assume it would have worked. also, supermicro has some compatible ecc memory on their US online store. 32gb sticks were in store for about $160 last I checked.

Awesome, thanks so much for your help! After I posted here I found your other posts on a different thread so I had confidence it would work. I got my 10700 yesterday and everything is up and running with no issues! Got the hardware transcoding working and it handles it without breaking a sweat. Now I can sell my RTX 4000 (which was way overkill) and it will pay for the new CPU, Motherboard and case I bought haha. Really hoping the 6.9 RC is out soon, I don't mind running the beta but I'm excited for all of the other goodies it should include. Also, glad the latest AMD CPUs are so great they forced intel to put hyper-threading back into CPUs lower than i9s. I was tempted to get a 10900 but its just not worth the money for my purposes.

Hi @lukeoslavia, thanks for posting your findings for everyone else to see! I'm getting my 10th gen i7 tomorrow and just wanted to check - did you load the same driver for the 10th gen as previous models? The "modprobe i915" line in the go file or was it something else? Thank you again

Of course! This is for the GIGABYTE GeForce RTX 2080 GAMING OC 8G GV-N2080GAMING OC-8GC. I did not have to modify anything with a hex editor, the only change I made was to rename the extension from .bin to .rom (not even sure if I needed to do that or not). GigabyteGamingOC_2080_BIOS.rom

Hey Guys, so I was able to boot directly into windows on my NVMe drive with the RTX2080 (bypassing the boot to unRAID from the flash drive). Once in there I dumped the card BIOS and copied it over to a flash drive - I also installed the NVIDIA drivers while the card was easily recognized. Once back in unRAID I added the card and audio device to my VM, also had to re-add the NVMe drive (via the XML) as it got removed for some reason. I wasn't able to boot at first, was getting some IOMMU group errors. Thinking back to @rinseaid's suggestion, I set the PCIe ACS override to 'both' and rebooted the server. Once back in I was able to boot successfully with GPU passthrough and my NVMe drive, although there was definitely an uncomfortably long black screen until the windows login screen finally popped up. At this point I am all set, I just need to figure out the best way to handle USB devices and for some reason my Logitech receiver's range is absolutely terrible. I have my server connected to my TV and it doesn't even come close to reaching the couch for keyboard and mouse usage (only about 7 feet away), I have to sit 1-2 feet away from the server to get reliable mouse and keyboard performance. I will get a USB extension cable and put the receiver closer to the front of the TV to see if that helps, if not I am also thinking about the possibility of passing through the built in Bluetooth card on my mobo and using BT for mouse and KB connectivity - but will have to do some research to see if that is possible. Just wanted to provide a conclusion to this adventure in case someone runs into the same issues and thank you all once again for the help, suggestions and responsiveness. This is an awesome community!

Just got home and in the 15 minutes before I had to leave for a class, I cracked open the case, installed the new card and booted it up, NO GPU in unRAID..... Utterly confused I got back into the BIOS and loaded the defaults in a last ditch effort to resolve a potentional problem setting - and something I hadn’t tried yet. Booted back into unRAID and voila the card appeared! I feel so dumb BUT I never changed any advanced settings in there so I’m not sure what option caused it. Thanks again to all for the help! Now on to the next step after “plugging it in” at some point tonight when I get home

That’s pretty sweet! I just got a replacement Gigabyte RTX 2080 today - the founder’s edition is on its way back to NVIDIA. Will try to get it going again at some point tonight if I have time. Will start with getting the card recognized first(!) and then move on to GPU and USB passthrough

100% confident no - but I'm 99% sure a 600W Corsair 80+ Gold rated PSU is more than enough for this card and my configuration, especially since I'm just booting it up. The card will only consume its peak 225W when at full load which should only happen when gaming or rendering. I am not even able to get the OS to recognize it so as you had previously mentioned its most likely some other hardware issue. Besides the card I have a 65W CPU at stock clocks and voltage, the mobo, 2 fans, 2 hard drives and 2 SSDs so its not like I'm stressing the PSU at all.

Sweet! Hope it works for you on the first attempt like @rinseaid experienced. I will definitely be interested in hearing about it, thanks!

So to update this, I reseated the card and the PSU cables on both end 3 separate times. I saw no change at all so I am just going to assume the card arrived DOA since I dont have another system to test the card in, or another card to test in my unRAID build. I have initiated an RMA with NVIDIA so when I get a replacement card, if i still have this issue then at least I will know it is not the card. Thanks again to everyone for the help and suggestions, Ill be sure to check back once there is a change, hope everyone is having a great weekend!

Thanks for the suggestions guys. I have a Corsair SF600 600W 80+ Gold certified PSU with both the 8pin and 6pin connected to the card - but I’m going to pull the card and check all the cables to make sure they are inserted properly (they’re modular) and give this another go. My system is only supposed to consume around 450w at peak usage without overclocking so the PSU should be more than enough, especially for just booting. I wasn’t able to get a second card to test with but once I recheck everything, grabbing some kind of second card to check is my next step. I’ll update on status or progress, appreciate all the helpful suggestions!

I just tried setting my PCIe ACS override to 'both' and rebooted for the heck of it and I got no change unfortunately.

Interesting, that is sort of the experience I was hoping for. So you didn't have to do anything special when you first popped it in right? It just showed up and then the IOMMU group issue is a different matter. Did you get an NVIDIA Founder's edition or one of the AIB models? Just curious if our cards match since our systems are so different. Thanks for the reply and enjoy your new toy!

UEFI

I get the bios splash and then a black screen after that. - When on iGPU as primary with hdmi connected to mobo = usually shows BIOS Logo > UNRAID Boot Menu > boots to the CLI and login prompt - With the RTX as primary and hdmi cable connected to it = BIOS Logo > blackscreen

I guess its possible the card is DOA - which would be my luck. The motherboard is recent and running the latest 3.1 bios which is dated 7/10/2018 but anything is possible from a hardware standpoint and there's not much out there since the card is only a day or two old. I have also never tried any other card in the pci-e x16 slot so i definitely want to try something else to see if that works. When i set the PCI-E RTX GPU as the primary GPU in the bios i just get a black screen but unraid eventually boots. In the web interface there is no change to the system devices or IOMMU groups. Dead card is a real possibility, will have to do some further testing to confirm. Thanks for the reply!

it was worth a shot!

Ok so PCI lanes are not the problem, I was actually just doing research and confirmed the GPU is typically connected directly to the CPU and NVME drives are connected to the chipset so thats exactly what you just mentioned. This is a mini-itx board with a single pci-e slot so i theoretically should never run out of PCI-e lanes but it was a valid concern to mention. Thanks @johnnie.black! I think trying another GPU will be the best bet for now but the only real test would be another RTX GPU because i assume an older NVIDIA card would be ok.

Hi @david279 thanks for the reply! Hmmm that is a very interesting factor that I didnt even consider. I will remove the NVME drive and try booting without it to see if that makes a difference, that should free up the 4 lanes which would hopefully allow the 16 needed for the GPU. Unfortunately I dont have another GPU to test with, maybe I can borrow one from work just to see if i get different results. PSU is a Corsiar SF600 - 600 Watt 80 PLUS Gold so I dont think that is a limiting factor, the GPU lights up and the fans spin, the PSU fan doesn't even kick in so that means the load is less than 50% if i am not mistaken.

Hi @testdasi thanks for the reply! Ok that makes sense, I did something similar to pass through my NVME drive, but how do I know the address of the RTX card if it doesn't show up anywhere within UNRAID? Is there a default address for the first PCI-E slot or something like that or should I be looking somewhere else to find the address tab besides the system devices?

Hey Guys, I just received my new NVIDIA RTX 2080 with which I was hoping to do GPU pass-through with to a Windows 10 VM and unfortunately it doesn't appear that UNRAID even recognizes it. I am running UNRAID 6.6 final on a Asrock - Z370 Gaming-itx/ac, with an Intel i7-8700, I have VT-d and IOMMU enabled. In the bios I set the integrated Intel GPU to "ON" and I also set that as the primary (although I also tried with PCI-e GPU as the primary and got the same results). I also tried with the integrated graphics set to auto. I don't know if its a matter of the card being too new for unraid to recognize it or if I am missing something else. From all the guides I have read, it seems that unraid should at least see the card and the trickery comes in when trying to do the passthrough but I am not seeing the card under system devices and it is not an option for the GPU within the edit VM settings. I have already tried the basic hardware troubleshooting of re-seating the card and power cables, and i dont have a second pci-e slot to test on so I came here. I realize the cards were just released yesterday but I am hoping my diagnostics can help someone with more knowledge explain the issue or that its a simple matter of the hardware is unsupported for now. Any help is greatly appreciated, thank you!!! fc-unraid-diagnostics-20180921-0827.zip