After Update to UNRAID 6.12.0, Windows VMs are not working

[eltonk]

After update Unraid to latest 6.12.0, my MS Windows VM is not starting anymore.

I checked the disk permissions, but seems ok. I tried to change the ownership to "nobody" and got the same error (as you can see below).
Any help on how to fix this?

 

root@draco:/mnt/cache/domains/Windows 11 VM# ls -la
total 50609848
drwxr-xr-x 1 nobody users          20 Mar 30 15:09 ./
drwxrwxrwx 1 nobody users          46 Mar 30 15:09 ../
-rwxrwxrwx 1 root   users 68719476736 Jun 20 01:08 vdisk1.img*
root@draco:/mnt/cache/domains/Windows 11 VM# chown nobody:users vdisk1.img 
chown: changing ownership of 'vdisk1.img': Read-only file system
root@draco:/mnt/cache/domains/Windows 11 VM# ls -la
total 50609848
drwxr-xr-x 1 nobody users          20 Mar 30 15:09 ./
drwxrwxrwx 1 nobody users          46 Mar 30 15:09 ../
-rwxrwxrwx 1 root   users 68719476736 Jun 20 01:08 vdisk1.img*
root@draco:/mnt/cache/domains/Windows 11 VM# 

 

[JorgeB]

Please post the diagnostics.

[eltonk]

draco-diagnostics-20230620-1309.zip
Hi @JorgeB
Here it is. Thanks

[JorgeB]

Your cache drive dropped offline:

 

Jun 20 01:08:16 draco kernel: ata5: hard resetting link
Jun 20 01:08:22 draco kernel: ata5: link is slow to respond, please be patient (ready=0)
Jun 20 01:08:51 draco kernel: ata5: COMRESET failed (errno=-16)
Jun 20 01:08:51 draco kernel: ata5: limiting SATA link speed to 3.0 Gbps
Jun 20 01:08:51 draco kernel: ata5: hard resetting link
Jun 20 01:08:57 draco kernel: ata5: COMRESET failed (errno=-16)
Jun 20 01:08:57 draco kernel: ata5: reset failed, giving up
Jun 20 01:08:57 draco kernel: ata5.00: disable device
Jun 20 01:08:57 draco kernel: ata5: EH complete

 

Check/replace cables and post new diags after array start.

[eltonk]

Ouch... so, the update process destroyed all my cache data?! 
I'm pretty sure that the issue is not cables as you suggested... It was working perfectly before the update to version 6.12

 

 

Edited June 20, 2023 by eltonk

[trurl]

3 minutes ago, eltonk said:

so, the update process destroyed all my cache data?

No, you have a hardware problem which caused the cache drive to disconnect. No evidence yet there is any data loss.

 

1 hour ago, JorgeB said:

Check/replace cables and post new diags after array start.

 

[eltonk]

As I said, there are no hardware issues.

 

Rolling back the installation to the previous 6.11.5 version fixed the issue. So, if anyone faces this issue, I recommend rolling back the update.

 

Thanks for the support, but I'll wait for the next bug-fixes release before update it again.

 

 

[JorgeB]

That issue has nothing to do with the update, may be a flaky cable, that sometimes works, sometimes not.

[Archonite]

I'm piggy backing on this thread as I have a similar issue. Upgraded to 6.12.2 from 6.11.5 yesterday. My CentOS 7 VM no longer works.

 

I'm uploading the diagnostics, but what I see this in the libvirt logs:

 

Quote

2023-07-03 13:11:35.004+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 13:21:36.638+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 13:31:38.242+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 13:41:39.792+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 13:51:41.459+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 14:01:43.090+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 14:11:44.672+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 14:21:46.334+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error
2023-07-03 14:31:47.988+0000: 6516: error : virNetSocketReadWire:1791 : End of file while reading data: Input/output error

 

I'm currently rolling back to 6.11.5 and will post again to say if the restore brought back the VM.

dragoon-diagnostics-20230703-1038.zip

[Archonite]

Rollback completed, and VM works.

[Archonite]

If anyone comes to this thread with a similar issue:

 

The issue boiled to down virtio not working with the kernel version I had on CentOS 7 (3.10).

 

Updating to current mainline kernel fixed the issue. To do so, I had to switch the VDISK BUS to SATA, update the kernel, then swap back to VIRTIO and everything was working.

[srfnmnk]

@Archonite -- how did you update the kernel?

[ghost82]

9 hours ago, srfnmnk said:

how did you update the kernel?

he is speaking about the kernel of the linux virtual machine, not that of unraid.

I think he simply updated the kernel of the virtual machine: centos should use yum command, so, 'sudo yum -y update' should update the system, including the kernel.

 

[srfnmnk]

Thanks. The issue with even attempting to update the kernel inside the VM is that the network also uses virtio it seems and I can't get that to work either...so no internet. More details on this thread. Appreciate the response.

Edited November 2, 2023 by srfnmnk

[ghost82]

33 minutes ago, srfnmnk said:

Thanks. The issue with even attempting to update the kernel inside the VM is that the network also uses virtio it seems and I can't get that to work either...so no internet. More details on this thread. Appreciate the response.

Change network from virtio (or virtio-net) to e1000-82545em.

Change disks from virtio to sata.

[srfnmnk]

virtio to sata for the disks does allow the VM to start up

 

I believe I tried all the different network options and none of them actually gave me a network.

[ghost82]

e1000-82545em takes advantage of alternative built-in linux drivers other than virtio, so it should work. Make sure it's detected inside the vm and configured properly to have internet access, use ip addr command for example to see if it's detected.

Edited November 2, 2023 by ghost82

[srfnmnk]

yeah it was working fine before the upgrade.

[srfnmnk]

these are the options I have

 

[srfnmnk]

seems like no matter which option I use, this is all i get back. It doesn't have any visible eth interfaces.

 

And sorry for the spam.

 

Edited November 2, 2023 by srfnmnk

[ghost82]

6 hours ago, srfnmnk said:

It doesn't have any visible eth interfaces.

Post diagnostics please

[srfnmnk]

pumbaa-diagnostics-20231101-1636.zip

[ghost82]

None of the vms you have have the e1000 type network emulated card.

Open the vm in advanced mode (xml view), find the network block, for example:

    <interface type='bridge'>
      <mac address='52:54:00:ce:dc:cb'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

 

and change the model type line by hand, like this:

      <model type='e1000-8254em'/>

 

Save and boot the vm.

If it still doesn't work reattach diagnostics and let us know which vm you are working on of the 10 vms you have.

Paste also the output of this terminal command, from inside the virtual machine:

lspci

 

[srfnmnk]

Apologies @ghost82 -- I gave you a diagnostics from a bit earlier.

 

I only have the e1000 as an option from the drop down. I can start it with the following but no network:

<interface type='bridge'>
  <mac address='52:54:00:9e:3b:8c'/>
  <source bridge='br0'/>
  <model type='e1000'/>
  <address type='pci' domain='0x0000' bus='0x07' slot='0x01' function='0x0'/>
</interface>

 

but with the following it fails to start showing the error in the screenshot below.

<interface type='bridge'>
  <mac address='52:54:00:9e:3b:8c'/>
  <source bridge='br0'/>
  <model type='e1000-8254em'/>
  <address type='pci' domain='0x0000' bus='0x07' slot='0x01' function='0x0'/>
</interface>

 

 

I started it with it set to e1000 and posted the lspci you requested -- the screenshot is below.

 

Thanks again for your review.

pumbaa-diagnostics-20231103-1151.zip

Edited November 3, 2023 by srfnmnk
hide mac address

[srfnmnk]

I was able to get it to start with 82540em (with a 0)

<model type='e1000-82540em'/>

 

but the same issue is present. No eth0 only lo: and no interenet