Switching from GPU passthrough local to VNC in linux (Pop!)

[Waffleophagus]

Hey guys, I have a weird usecase that I wanted to run by you folks to see if what I want is even really possible. Right now I have 2 VMs setup that point at the same virtual disk, both are Pop! (obviously) but they have different hardware configurations, one has a GPU passthrough, more ram and CPU, and more sound card options, the other is setup with VNC only, no GPU, no sound, less ram and CPU cores as well. I want to know if its possible to switch between these two configurations without having to fully shut down or spin up the hard disk? I don't much care if we have to change the configuration on the whole, like if its settings within a single VM instead of two, but the goal is to be able to stop working on the disk remotely, then when I sit down to my computer, resume my workflow on the desktop with all the power of the full VM, or vice versa, with the ability to leave the desktop but pick up with the VNC deploy without having to lose my spot by fully shutting down.

 

I know that this may be a really weird ask, and might not be possible, I'm all ears on possible solutions to get the net result though!

 

Thanks!

Cheers,

Waffleophagus

[ghost82]

I think this is totally feasible.

I didn't try it directly but I know that:

1. several users have 2 versions of a vm pointing to the same vdisk, one with gpu passthrough, the other with vnc; for example with mac os vms, users use the vnc version for mac os updates (otherwise during automatic reboots the mac os vm will kernel panic) and the gpu passthrough version for everyday use.

2. I changed the number of cpu cores for a vm and I didn't have any issue in starting the vm from the same vdisk.

 

Moreover, if you think about a bare metal system:

1. you can change the gpu with another without reinstalling the os

2. you can add/remove ram modules without reinstalling the os

3. you can change the cpu without reinstalling the os

 

It happens that simply drivers will be installed in the system and they will be used or not depending on the attached hardware.

 

By spinning down the hard disk you mean the physical disk, because obviously you need to shutdown the vm n.1 and start vm n.2.

 

Another thing you may want to consider is to have only the passthrough version of the vm and access the vm with a vnc client to a vnc server installed inside the vm (leaving the "unraid" novnc).

Edited October 26, 2022 by ghost82

[Waffleophagus]

Thanks for the breakdown! To clarify a bit though, the goal isn't to have the 2 VMs, I already do have them! Its more a question of being able to stop one and resume on the other without a reboot. 

13 hours ago, ghost82 said:

Another thing you may want to consider is to have only the passthrough version of the vm and access the vm with a vnc client to a vnc server installed inside the vm (leaving the "unraid" novnc).

 

This is a very reasonable idea that I could definitely check out! The one reason why this won't be the soul fix is resource distribution, as the GPU passthrough VM takes up way more of the system resources, my hope is to swap between them without having to "lose my spot" (reboot) while, when not using it, the system resources are freed up. 

 

 

13 hours ago, ghost82 said:

By spinning down the hard disk you mean the physical disk, because obviously you need to shutdown the vm n.1 and start vm n.2.

Yes, definitely using colloquialisms.

 

But yea, the goal is very much "suspend VM, free up system resources, but save state and be able to resume state on other VM if possible"

[ghost82]

Ok, now it's more clear what you want to do.

I really have some doubts that it could work and other doubts about how to set this thing..You should suspend the vm (identified by a domain) and wake it up (from another domain), and I really don't see how you can do this.

If I were you I would go with the route of gpu passthrough, vnc server inside the vm (so, no vnc set in xml, nor qxl or other virtual vga settings), and suspend/resume commands from the host.

The idea is to have a vm always with gpu passthrough when it runs, but free the resources when the vm is suspended (to disk).

 

About suspend/resume commands (you can implement scripts)

1. enable in the guest xml suspend to disk and suspend to mem:

<pm>
  <suspend-to-disk enabled='yes'/>
  <suspend-to-mem enabled='yes'/>
</pm>

 

2. suspend the guest to disk via unraid terminal with command:

virsh dompmsuspend domain disk

replacing "domain" with the name of the vm domain.

 

When the guest is suspended to disk libvirt should report the vm as shutdown, so the attached gpu should be released and resources released.

3. resume the vm via unraid terminal with command:

virsh dompmwakeup domain

replacing "domain" with the name of the vm domain.

Edited November 13, 2022 by ghost82