[SOLVED] Bizarre behavior on threadripper build

[TPNuts]

As a preface I posted my first post on another thread linked here 

Just didn't want to hijack OP's thread.

 

So I am having a multitude of kernel panic, unable to find rootfs errors along with sometimes not even booting altogether. Then sometimes it boots fine and stays great! I suspect this being an ongoing issue as my unraid server was unstable on my i7-4770k prior to upgrading anyhow I figured this would just be a hardware change and it did for the most part work till well 2 days after when I started having kernel panic crashes.

And we are back to having issues. 

specs are as follows:

Gigabyte Master Aorus TRX40

AMD Threadripper 3970x

 128gb (currently 96gb) Gskill DDR4-3600 16gbx2 F4-3600C16D-32GVKC (im at 96gb ram (waiting on a replacement 2 dimm set from gskill as 1 was doa)) on XMP profile of 3600 <- testing with 3600 and 3200

5 x 10tb Ironwolf Pro hdd (4 on the raid array, 1 as parity)

1 x 2tb Sabrent Rocket 4.0 NVME

1 x 1tb WD SN750 NVME

 

I ran memtest because it may have been a bad ram stick or dimm slot.

 

when the 6 dimm modules were installed I did test it with memtest, in the first test I got 1000s of errors so i went the single dimm way and ran each stick through 2 passes with no errors. So that was a bit weird. I then added each set back in 2 at a time and ran 2 passes. No errors

I am currently (Edit: testing) on the set of 6 again on the same dimm slots as well and will update accordingly.

(Edit2: no errors on all 6 sticks.... what is going on!

Is there any instance a bad usb could cause kernel panic issues to rootfs? Even after multiple formats. any instruction on rebuilding the array without using the existing config on a new usb? I also forgot to state I am running the system on a SAMSUNG 128GB BAR Plus (Metal) USB 3.1 Flash Drive MUF-128BE3/AM)

 

Another Question I had was that when i was switching out the modules and testing them one by one sometimes the USB would not be recognized and bios would load showing only the NVME drives since the array is running off of a LSI Sas 9300-8e board to an external hard drive storage rack. 

This would then be resolved as once I shut the machine down all I would have to do is pull the usb and reinsert it back in the same USB 2.0 port and it would recognize again and start the boot process on unraid... This is highly irregular and any advise on this would be appreciated.

Edited April 30, 2020 by TPNuts
clarification

[JorgeB]

6 hours ago, TPNuts said:

esting with 3600 and 3200

For the DIMMs installed both are overclocking and a known source of instability with Ryzen based servers, see here.

[TPNuts]

So I reconfigured bios and set the ram at 32x (3200mhz), unraid booted up with no issues except somehow the network was reconfigured, so i fixed that

I have run into another issue now. reenabling vitualization in the bios I had passed through my 980ti to my VM but now i receive this error

"nternal error: qemu unexpectedly closed the monitor: 2020-04-30T09:11:33.530626Z qemu-system-x86_64: -device vfio-pci,host=0000:4a:00.0,id=hostdev0,bus=pci.5,addr=0x0: vfio 0000:4a:00.0: failed to setup container for group 55: Failed to set iommu for container: Operation not permitted"

Any way to get it back up in the same vdisk. oddly enough nothing else has changed that i know of. Devices still shows the IOMMU Group 55 as the 980ti and the hd audio GM200.

 

[bastl]

@TPNuts Are you passing other devices to the VM besides of the GPU? Onboard audio maybe?

[TPNuts]

7 hours ago, bastl said:

@TPNuts Are you passing other devices to the VM besides of the GPU? Onboard audio maybe?

Just the gpu and audio but I did get it figured out. I ended up deleting the VM and recreating it and it just started working on its own so i'm not going to touch it. So far as for server stability goes at 3200 it is rock solid. Im just curious as to what will happen when it runs in quad channel. Will that make any diffrence? I wont keep it higher than 3200 now that i know that is the actual issue (aka im an idiot and didnt even think of that being an issue even though i knew it, old people stubborness).