VMs Tab causes webGUI to Freeze: Unraid 6.10.3

[AtomZ]

Hi everyone -

 

I am running into an issue where every time I try to access my VMs tab in the webGUI, the webGUI freezes and can't access the VMs tab. Looking at the console logs I keep seeing a warning that states: 

[pool www] server reached max_children setting (50), consider raising it

 

Essentially this hangs up the webGUI and then it eventually loads. When trying to access one of the VMs nothing happens. However it is still running fine. 

 

Checked if the hypervisor was running via this command 

virsh list --all

 

Everything returned normal instantly with the following with no issue:

 

What's interesting this has happened before when I was on 6.9.3 and then I put off my VM project until this week and I still have the same issue. Would it help increasing the max children field? If so how should I go about this and to what value?

 

I do not think this is a resource issue but just in case I have a 16 core processor, 64GB RAM, TBs left in disk space etc. and I have tried multiple browsers.

 

Please see attached diagnostics as well if needed.

 

Any ideas on how to address this? I appreciate any help. Thanks!

diagnostics.zip

[AtomZ]

Update:

 

I killed the VM via cli in the console. Instantly VM tab is responsive. Restarted VM and everything is running smooth. It seem this becomes an issue when the VM is on for awhile. 

 

Any ideas that may help address this if this happens again will be helpful if possible.  

Edited August 24, 2022 by AtomZ

[KermitJr]

Has anyone found an update to this?  I mean, there are plenty of places where people mention this as an issue.

 

For me, it first happened when I tried to shutdown the server. VM tab hangs.

 

Rebooted a couple of times (after hard reset) and it magically worked. So I decide to take server down.

 

I have tried:
- Goto VM tab, click, STOP - VM tab hangs

Hard reboot a couple of times (Cause 1 reboot usually doesn't fix it)

- Remote into VM and shutdown from there ... works 90% of the time. Except today it hangs.

 

I've gone into settings and disable VMs and tab disappears. Re-enable - and tab hangs.

BTW - Settings - VM Manager -- Disable VM Server - takes the tab away but the VM is still running (I can remote into it)

 

Sometimes Libvirt fails to start. A couple of reboots and it'll magically start working again. But I shouldn't have to restart a bunch of times.

 

So what logs or diagnostics do we need to get at this?

 

Right now I'm running a Parity rebuild (added second drive). 30 hrs until its available.

 

EDIT: Libvert Log

 

2023-10-15 15:59:55.054+0000: 32055: warning : qemuDomainObjBeginJobInternal:858 : Cannot start job (query, none, none) in API remoteDispatchDomainGetBlockInfo for domain HomeAssistant; current job is (modify, none, none) owned by (32054 remoteDispatchDomainAttachDevice, 0 <null>, 0 <null> (flags=0x0)) for (10966s, 0s, 0s)
2023-10-15 15:59:55.054+0000: 32055: error : qemuDomainObjBeginJobInternal:893 : Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainAttachDevice)
2023-10-15 16:00:25.056+0000: 32057: warning : qemuDomainObjBeginJobInternal:858 : Cannot start job (query, none, none) in API remoteDispatchDomainGetBlockInfo for domain HomeAssistant; current job is (modify, none, none) owned by (32054 remoteDispatchDomainAttachDevice, 0 <null>, 0 <null> (flags=0x0)) for (10996s, 0s, 0s)
2023-10-15 16:00:25.056+0000: 32057: error : qemuDomainObjBeginJobInternal:893 : Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainAttachDevice)
2023-10-15 16:01:00.720+0000: 32487: error : qemuMonitorIORead:423 : Unable to read from monitor: Connection reset by peer

 

Tools -- diagnostics attached.

 

I'm in an out with my tech level in UnRAID, but I'll do my best to help.

 

Thanks.

 

 

skynet.local-diagnostics-20231015-1110.zip

Edited October 15, 2023 by KermitJr

[KermitJr]

AtomZ - what is the command for killing the VM via cli?

 

Thanks,

KermitJr