While installing Ubuntu on my Unraid server (LincStation N1), it hangs forever with 100% CPU use no matter how many vCPU are allocated to the VM, i.e., 1, 2 or even 3.

 

Switching to another console (before the install freezes) shows that the Ubiquity script takes all CPU cycles. It seems that the script hangs randomly in the install: if I hit all default options, I can get to the part where it says "curtin command in-target /" —though the forward slash "rotates" very slowly— but if I'm not fast enough, i.e., if I modify some of the defaut options, the script hangs earlier, which leads me to believe that this issue is not related to the script itself. I tried to let it run through the night thinking it would eventually get through the install, but it did not.

 

Installing with Ubuntu 22.04.4 or 22.04.5 ends up the same way. The VM is configured with all default options (though I have tried different variations), and with 8Gb RAM.

 

Could you let me know how to debug this? Have you seen this before?

 

Running Unraid 7.0.0-beta 4 (was the same with beta 3) on a LincStation N1.

Hi, from your post, it seems you are using Unraid bare metal and Ubuntu in a VM.

This section is for installing Unraid as a VM in another hypervisor.

 

If I am right, we will move this thread to a more appropriate section.

Indeed, that is right. Please help me move my post to the appropriate section.

9 hours ago, kevin.denver said:

Indeed, that is right. Please help me move my post to the appropriate section.

Can you post diagnostics

9 hours ago, SimonF said:

Can you post diagnostics

Here you go, Simon. Thanks for looking into it.

unraid-diagnostics-20241024-1026.zip

Looks like I'm having I/O errors on one of the NVMe's, but that started yesterday, when I started copying files to a share.

Edited October 24, 20241 yr by kevin.denver

On 10/24/2024 at 3:36 AM, kevin.denver said:

Looks like I'm having I/O errors on one of the NVMe's, but that started yesterday, when I started copying files to a share.

Cannot see anything in the logs, I have an N1 so will try to install Ubuntu VMs and see how it goes. I have run Debian and other test VMs before with no issue.

 

But to confirm it is not going in to the UEFI boot page a CPU normally shows running at 100% at this stage.

This was my XML, Installed start ok.

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='1'>
  <name>Ubuntu</name>
  <uuid>a789f74f-18c0-aa79-bb41-6d0e0640a29f</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Ubuntu" icon="ubuntu.png" os="ubuntu" webui="" storage="default"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-9.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/a789f74f-18c0-aa79-bb41-6d0e0640a29f_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' clusters='1' cores='2' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='utc'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='no'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='rtc' tickpolicy='catchup'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source file='/mnt/user/domains/Ubuntu/vdisk1.img' index='2'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/ubuntu-24.10-desktop-amd64.iso' index='1'/>
      <backingStore/>
      <target dev='hda' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <alias name='sata0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='sata0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:db:5a:ef'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/4'>
      <source path='/dev/pts/4'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/run/libvirt/qemu/channel/1-Ubuntu/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' sharePolicy='ignore'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

Hi Simon, let me try that. I've been having NVMe issues lately and with George's help I think it is fixed with some power-saving kernel params. I will try again setting up a Ubuntu VM in the next day or two. Hopefull the NVMe was the culprit behind it. (And I almost forgot: thanks a lot for taking your own time to try this. It is much appreciated).

Edited October 25, 20241 yr by kevin.denver

Hi Simon. I tried again after fixing the NVMe issue. It did the same thing, i.e., hung duting the install. I realized that it does hand on the network, somehow. I'm not sure whether it is related, but each time it does that when it downloads packages. It goes through 1, 2 ... 14, then nothing. When that happens, I can't even open a new tty as it hangs forever.

 

Next, I disabled ens3 and started the script again. This time, it looks like it went further but crashed with the output as can be seen in the screenshot. I could not get the crash file without network. Not sure this has anything interesting, though.

 

Then, I tried another VM with all default network settings and again, it hung forever at "Featured server snaps". While Unraid was showing the CPU at 100%, a top on another console in the VM shows that all was pretty normal. See screenshots.

 

I then tried again with modified params (see enclosed). It ended the same way: hang. I opened another tty before starting the install so I could run top. It shows that kwapd was taking 60-90% cpu, but sometimes less. free took about 5 minutes to execute (with 3 vCPUs). The script went as far as "Get:30 ... universe amd64 Packages [1133 kB]" then nothing.

 

Is there a better way to look at what's happening?

I think I'm seeing what's happening here, hopefully. I created a new VM and, before the install started, I shutdown the ethernet connection, which somehow allowed the install to go further down. While creating encrypted partitions, it crashed similarly as it did before —but can't confirm 100% as It didn't keep the previous log— but while going further down the log, I saw a OOM-killer executed on cryptsetup (see screenshot). Then, further down in the install script, there was an "unexpected error while running cryptsetup" (see screenshot). I did configure the VM with 1Gb min and 8Gb max, but running free -m on another tty showed around 780Mb. Is the min RAM var during the install a fixed vs. dynamic value?

 

I'm going to try it again with 4Gb min RAM and see what happens.

That's it... It works. I was under the wrong assumption that the RAM min/max was a dymanic value.

I'm not sure how to close this topic, though. Can you help out with this? Thanks, Simon.

Edited October 28, 20241 yr by kevin.denver