SOLVED - Existing pfsense vm will not boot and new freebsd won't either

[DZMM]

I'm in a very weird situation.  My existing pfsense vm won't boot and even when I try creating a fresh instance, that won't boot either.

 

I added two new sticks of ram today and I'm running a memtest now.  But, surely even if one of the sticks is faulty it wouldn't only impact freebsd based VMs?????

 

Anyone have any ideas???  I'll post some diagnostics once the memtest finishes.

Edited February 26, 2020 by DZMM

[Chess]

1 hour ago, DZMM said:

I'm in a very weird situation.  My existing pfsense vm won't boot and even when I try creating a fresh instance, that won't boot either.

 

I added two new sticks of ram today and I'm running a memtest now.  But, surely even if one of the sticks is faulty it wouldn't only impact freebsd based VMs?????

 

Anyone have any ideas???  I'll post some diagnostics once the memtest finishes.

 

I doubt that adding ram would cause this, but is the VM vdisk damaged. I think from your post you have already tried to a fresh pfSense VM, using a new blank vdisk?

 

How many DIMMs do you have in that mainboard? If you have all 8 DIMM slots filled up, drop your ram speed to 1866.

 

DIMM     ConfigMemory Ranks     Official Supported Transfer Rate (MT/s)

4 of 4     Single                            DDR4-2933

4 of 8                                          DDR4-2667

8 of 8                                          DDR4-2133

4 of 4     Dual                              DDR4-2933

4 of 8                                          DDR4-2667

8 of 8                                          DDR4-1866

[DZMM]

Well, that was a wild goose chase for 6 hours.  Because I made so many hardware changes (new ram, new gpu, moved cards and disks) as well as removing most kit to clean, I assumed I'd done something wrong.

 

The issue was pfsense needs an xml edit to work post 6.8.2 - I'd lost my edit and because I was panicking, I forgot about the change I'd made a few months ago.  I think the edit must have dropped off because of the hardware changes.

 

@Chess thanks for trying to help.  What's the source for the Ram timings? 

[Chess]

1 hour ago, DZMM said:

thanks for trying to help.  What's the source for the Ram timings?

https://www.tomshardware.com/reviews/amd-ryzen-threadripper-2950x-2990wx-cpu,5797.html

 

That is where I got it from. AMD also says on their site that the ram speed is a max of 2933, but they don't go into details on how many sticks. How many DIMMS do you have in that system?

Edited February 26, 2020 by Chess
url

[DZMM]

I've got 8 X 16GB and previously 6 sticks.  I don't know much about ram settings as I gave up on overclocking about 15 years ago, so I always just go for the auto settings in the bios.  I'll monitor for a bit to see if things seem odd.

[Chess]

6 hours ago, DZMM said:

I've got 8 X 16GB and previously 6 sticks.  I don't know much about ram settings as I gave up on overclocking about 15 years ago, so I always just go for the auto settings in the bios.  I'll monitor for a bit to see if things seem odd.

 

I suspect @ 2666 your system is fine, but keep an eye on it for random lock ups or crashes or even kernel panics now that you have 8 DIMMS. If you get any of that out of the blue, drop the ram speed to 2133. 

 

Glad you got your issue resolved. Are you willing to share your edits to make pfSense work? Seems there are a few other users on the forums having issues with getting pfSense up and running. I also use pfSense, but have keep that BM for now.

[DZMM]

On 12/12/2019 at 10:45 AM, bastl said:

I found a workaround for this!

 

The culprit is the cpu-mode "host-passthrough". If I switch to "Emulated QEMU64" the VM boots up again. Switching it in the gui should work if you havn't setup any special CPU flags. Another way is to edit the xml like the following:

 

change

  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='2' threads='1'/>
  </cpu>

to

  <cpu>
    <topology sockets='1' cores='2' threads='1'/>
  </cpu>

also forces the CPU into emulated QEMU64 mode.

 

Another option is to emulate a Intel Skylake CPU for example with the following:

  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Skylake-Client</model>
    <topology sockets='1' cores='2' threads='1'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='pcid'/>
    <feature policy='disable' name='hle'/>
    <feature policy='disable' name='erms'/>
    <feature policy='disable' name='invpcid'/>
    <feature policy='disable' name='rtm'/>
    <feature policy='disable' name='mpx'/>
    <feature policy='disable' name='spec-ctrl'/>
  </cpu>

 

Edit:

"AES-NI CPU Crypto" isn't supported on "Emulated QEMU64" mode. For future Pfsense versions this is a requirement if I remember correctly.

I used the Skylake emulation above - somehow with the upgrade or my panicked tinkering, the edit was lost from my xml