Disk sizing issues after transfering vdisk to NVMe SSD in MacOS kvm.

[frodr]

I have moved the startup disk fram vdisk to unassigned NVMe ssd on a MacOS HS kvm, following SpaceinvaderOne`s guide. It works fine, only I am not able to increase the size to ssd size, or create a new partition. 

 

What can be wrong?

 

 

// Frode

 

 

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='18' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>202_MacxOS HS</name>
  <uuid>xxxxxxxxxxxxxxxxxxxx</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Linux" icon="linux.png" os="linux"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>24</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='12'/>
    <vcpupin vcpu='1' cpuset='36'/>
    <vcpupin vcpu='2' cpuset='13'/>
    <vcpupin vcpu='3' cpuset='37'/>
    <vcpupin vcpu='4' cpuset='14'/>
    <vcpupin vcpu='5' cpuset='38'/>
    <vcpupin vcpu='6' cpuset='15'/>
    <vcpupin vcpu='7' cpuset='39'/>
    <vcpupin vcpu='8' cpuset='16'/>
    <vcpupin vcpu='9' cpuset='40'/>
    <vcpupin vcpu='10' cpuset='17'/>
    <vcpupin vcpu='11' cpuset='41'/>
    <vcpupin vcpu='12' cpuset='18'/>
    <vcpupin vcpu='13' cpuset='42'/>
    <vcpupin vcpu='14' cpuset='19'/>
    <vcpupin vcpu='15' cpuset='43'/>
    <vcpupin vcpu='16' cpuset='20'/>
    <vcpupin vcpu='17' cpuset='44'/>
    <vcpupin vcpu='18' cpuset='21'/>
    <vcpupin vcpu='19' cpuset='45'/>
    <vcpupin vcpu='20' cpuset='22'/>
    <vcpupin vcpu='21' cpuset='46'/>
    <vcpupin vcpu='22' cpuset='23'/>
    <vcpupin vcpu='23' cpuset='47'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/b6d807ac-76fd-78a5-3710-fe431bb5832b_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source dev='/dev/disk/by-id/nvme-Samsung_SSD_970_PRO_1TB_S462NF0KB07816X'/>
      <backingStore/>
      <target dev='hdc' bus='sata'/>
      <boot order='1'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b0:ef:81'/>
      <source bridge='br0'/>
      <target dev='vnet1'/>
      <model type='e1000-82545em'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/4'>
      <source path='/dev/pts/4'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-18-202_MacxOS HS/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
  <qemu:commandline>
    <qemu:arg value='-usb'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-mouse,bus=usb-bus.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-kbd,bus=usb-bus.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='isa-applesmc,osk=<REDACTED>'/>
    <qemu:arg value='-smbios'/>
    <qemu:arg value='type=2'/>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='Penryn,vendor=GenuineIntel,kvm=on,+invtsc,+avx,+avx2,+aes,+xsave,+xsaveopt,vmware-cpuid-freq=on,'/>
  </qemu:commandline>
</domain>

Edited September 20, 2019 by trurl
redact osk

[david279]

Passthough the nvme disk as a PCIe device. SpaceInvader has a video to do it. Doing it that way will allow you to partition it. 

 

 

[frodr]

Thanks. 

 

A question regarding the xml mod. How do I translate the "1a", [144d:a808] 1a:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981, into:

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

[david279]

bus='0x1a' should do it. When you go to add it, if its not correct you will see a error at the moment.

[frodr]

44 minutes ago, david279 said:

bus='0x1a' should do it. When you go to add it, if its not correct you will see a error at the moment.

This error:

 

[david279]

Just now, frodr said:

This error:

 

Is the drive mounted using unassigned devices?

[frodr]

I left it unmounted, unformatted. 

 

Its the Samsung.

 

Edited September 20, 2019 by frodr

[david279]

Is it in a group by itself? Check the system devices in the Tools Tab.

[frodr]

3 minutes ago, david279 said:

Is it in a group by itself? Check the system devices in the Tools Tab.

Yes, its alone.

[david279]

I would stub it from the system so you can just add it in the VM edit page.

 

vfio-pci.ids=144d:a808

 

Add that to the syslinux config. Click the flash device in the Main tab and scroll to the bottom. Add that line in there and reboot. 

 

 

[frodr]

I added "vfio-pci.ids=144d:a808" at the bottom of Syslinux config. Same error message. Hummm...

 

 

[david279]

Rebooted and everything?

[david279]

12 minutes ago, frodr said:

I added "vfio-pci.ids=144d:a808" at the bottom of Syslinux config. Same error message. Hummm...

 

 

Thats the wrong spot. It needs to go into the spot for unRAID OS. Put it in before the initrd=/bzroot . 

[frodr]

By before, do you mean literally? Or above?

 

I tried:

kernel /bzimage

vfio-pci.ids=144d:a808

append initrd=/bzroot

 

Same error message.

 

Then:

kernel /bzimage

append vfio-pci.ids=144d:a808 initrd=/bzroot

 

The Unraid would not start. 

 

[david279]

This is how mine looks in the unRAID OS GUI mode slot

 

kernel /bzimage
append  pcie_acs_override=downstream,multifunction kvm.ignore_msrs=1 vfio-pci.ids=1002:67ef,1002:aae0,1b21:1242,1b21:1343,8086:10bc initrd=/bzroot,/bzroot-gui

[frodr]

2 hours ago, frodr said:

By before, do you mean literally? Or above?

 

I tried:

kernel /bzimage

vfio-pci.ids=144d:a808

append initrd=/bzroot

 

Same error message.

 

Then:

kernel /bzimage

append vfio-pci.ids=144d:a808 initrd=/bzroot

 

The Unraid would not start. 

 

kernel /bzimage

append initrd=/bzroot vfio-pci.ids=144d:a808

 

As SIO video, also the same error message. 

[frodr]

2 hours ago, david279 said:

This is how mine looks in the unRAID OS GUI mode slot

 

kernel /bzimage
append  pcie_acs_override=downstream,multifunction kvm.ignore_msrs=1 vfio-pci.ids=1002:67ef,1002:aae0,1b21:1242,1b21:1343,8086:10bc initrd=/bzroot,/bzroot-gui

Tried yours modified: 

 

label Unraid OS
  menu default
  kernel /bzimage
   append pcie_acs_override=downstream,multifunction kvm.ignore_msrs=1 vfio-pci.ids=144d:a808 initrd=/bzroot

 

but stille the same error message:

 

internal error: Device 0000:1a:00.0 is already in use

 

What does this message actually mean. That the another device is currently using this address?

 

 

[david279]

1 hour ago, frodr said:

Tried yours modified: 

 

label Unraid OS
  menu default
  kernel /bzimage
   append pcie_acs_override=downstream,multifunction kvm.ignore_msrs=1 vfio-pci.ids=144d:a808 initrd=/bzroot

 

but stille the same error message:

 

internal error: Device 0000:1a:00.0 is already in use

 

What does this message actually mean. That the another device is currently using this address?

 

 

Post the new xml with the nvme stuff added?

[frodr]

What should the ACS override be set at? Since the Samsung NVMe controller is in a separate group when disabled, it no point in splitting groups by enabling ACS override??

Edited September 21, 2019 by frodr

[frodr]

22 hours ago, david279 said:

Post the new xml with the nvme stuff added?

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>202_MacxOS HS</name>
  <uuid>xxxxxxxxxxxxxxx</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Linux" icon="linux.png" os="linux"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>24</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='12'/>
    <vcpupin vcpu='1' cpuset='36'/>
    <vcpupin vcpu='2' cpuset='13'/>
    <vcpupin vcpu='3' cpuset='37'/>
    <vcpupin vcpu='4' cpuset='14'/>
    <vcpupin vcpu='5' cpuset='38'/>
    <vcpupin vcpu='6' cpuset='15'/>
    <vcpupin vcpu='7' cpuset='39'/>
    <vcpupin vcpu='8' cpuset='16'/>
    <vcpupin vcpu='9' cpuset='40'/>
    <vcpupin vcpu='10' cpuset='17'/>
    <vcpupin vcpu='11' cpuset='41'/>
    <vcpupin vcpu='12' cpuset='18'/>
    <vcpupin vcpu='13' cpuset='42'/>
    <vcpupin vcpu='14' cpuset='19'/>
    <vcpupin vcpu='15' cpuset='43'/>
    <vcpupin vcpu='16' cpuset='20'/>
    <vcpupin vcpu='17' cpuset='44'/>
    <vcpupin vcpu='18' cpuset='21'/>
    <vcpupin vcpu='19' cpuset='45'/>
    <vcpupin vcpu='20' cpuset='22'/>
    <vcpupin vcpu='21' cpuset='46'/>
    <vcpupin vcpu='22' cpuset='23'/>
    <vcpupin vcpu='23' cpuset='47'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/b6d807ac-76fd-78a5-3710-fe431bb5832b_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/502_MacxOS/vdisk3.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b0:ef:81'/>
      <source bridge='br0'/>
      <model type='e1000-82545em'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x1a' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x1a' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-usb'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-mouse,bus=usb-bus.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-kbd,bus=usb-bus.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='isa-applesmc,osk=<REDACTED>'/>
    <qemu:arg value='-smbios'/>
    <qemu:arg value='type=2'/>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='Penryn,vendor=GenuineIntel,kvm=on,+invtsc,+avx,+avx2,+aes,+xsave,+xsaveopt,vmware-cpuid-freq=on,'/>
  </qemu:commandline>
</domain>

Edited September 22, 2019 by trurl
redact osk

[frodr]

I am able to start the kvm. I had placed the hostdev xml insert at the wrong place. Working without any ACS override. Now its cloning the vdisk, and see if I can get the kvm to start from the NVMe ssd. 

 

Thanks.

[trurl]

@frodr In future, please save us the trouble and don't post OSK.

 

6 hours ago, frodr said:

<qemu:arg value='isa-applesmc,osk=<REDACTED>'/>

 

[frodr]

14 minutes ago, trurl said:

@frodr In future, please save us the trouble and don't post OSK.

 

 

I'm sorry.