Passthrough of ConBee II Zigbee USB Gateway to Home Assistant Virtual Machine

[Reptar]

On 9/3/2021 at 1:21 AM, danioj said:

Posting this here in the hope that it assists someone in the future.

 

I host my instance of HomeAssistant in a VM on unRAID. I have recently purchased a ConBee II USB-Gateway so I can add Zigbee devices.

 

I added the USB using the unRAID VM GUI, like I imagine most would, by just checking the tick box next to the device. This didn't work. While Home Assistant found the device, the integration would not add (there were communication errors).

 

The trick was to add the device as a serial-usb device. AFAIK you cannot do this via the GUI. So I added the following code to my VM config:

 

    <serial type='dev'>
      <source path='/dev/serial/by-id/<yourusbid>'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial1'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

 

I was then able to add the integration easily. Interestingly, it didn't auto discover, but that's just an aside.

 

Note, <yourusbid> can be found via the command line - it contains the device serial so its not to be posted.

 

Can you tell me where exactly to add this? Within the <hostdev mode=subsystem section? Or should I add it at the very end of the file?

Thanks!

 

EDIT: I figured it out, this replaces the existing USB config for the device in the xml... now I'm seeing an error of

zigbee-herdsman:deconz:driver no free slots. Delay sending of APS Request

 

Looks like I'm still unable to access the webui for zigbee2mqtt though... not sure what the deal is

Edited June 8, 2022 by Reptar

[MrSir]

I tried the solutions here, constant restarts, unmounts, remounts none of it worked.

 

The solution was to Rollback to Unraid 6.9.2 and now my Conbee II is working with my Home Assistant VM again, weird bug hope it gets solved, until then I'm sticking with 6.9.2

[Grsh]

Looks like after updating to 6.10.2 today I got the same issue. My perfectly working HA install got no Conbee II stick anymore and I am literally sitting in the dark.

 

I think I got my serial number from the stick in the form of:
iSerial                 3 DE24##### (numbers blocked out)

Then I posted this into the XML form right above domain but it would not work.

<serial type='dev'>
      <source path='/dev/serial/by-id/3 DE24#####'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial1'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

 

Did I do the code right and pastet it in the right section?

I tried to start with a clean VM as well but that did not improve the situation.

 

Thanks

 

[jgosnell]

28 minutes ago, Grsh said:

Looks like after updating to 6.10.2 today I got the same issue. My perfectly working HA install got no Conbee II stick anymore and I am literally sitting in the dark.

 

I think I got my serial number from the stick in the form of:
iSerial                 3 DE24##### (numbers blocked out)

Then I posted this into the XML form right above domain but it would not work.

<serial type='dev'>
      <source path='/dev/serial/by-id/3 DE24#####'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial1'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

 

Did I do the code right and pastet it in the right section?

I tried to start with a clean VM as well but that did not improve the situation.

 

Thanks

 

Your pasted code show be inside the domain area.  Mine for instance is between the "interface" section and the "console" section.  Here is the code I am successfully using:

<serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2XXXXXX-XXXX'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial0'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

 

Edited June 8, 2022 by jgosnell

[Reptar]

2 hours ago, jgosnell said:

Your pasted code show be inside the domain area.  Mine for instance is between the "interface" section and the "console" section.  Here is the code I am successfully using:

<serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2XXXXXX-XXXX'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial0'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

 

 

Question here, does this bit need to match the bus/port in unraid?

 

<address type='usb' bus='0' port='4'/>

 

My HA VM keeps disconnecting from the conbee2 (like it disappears from the hardware list and the only way to fix is to reboot the host)

[jgosnell]

No it doesn't need to match.  My Conbee is physically connect to Bus 6 port 1 as reported by UnRaid.  I think line that is specifying what bus/port to attach to within the HA VM so the Zigbee integration know where to look for it.

[Reptar]

1 hour ago, Reptar said:

 

Question here, does this bit need to match the bus/port in unraid?

 

<address type='usb' bus='0' port='4'/>

 

My HA VM keeps disconnecting from the conbee2 (like it disappears from the hardware list and the only way to fix is to reboot the host)

I figured out the issue with the device getting disconnecting and also showing up in HA but not being able to pick up devices in pairing mode.

Looks like it doesnt like the front USB ports on my rosewill 4U chasis...

 

Thanks for all your help!!

[Grsh]

Thanks so much for everyones help.
I still can't seem to get it right.

It either won;t show up in HA or will give me an error.

 

I grabbed the serial like this:




That gives me 3 DE24##### as my serial number for the stick.

Using the info in this thread I copied this in my xml:
 

<serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_3DE24#####'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial0'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

Which results in my xml:

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Hassio</name>
  <uuid>c84f0292-195b-60b3-3e17-a3b2b5dd3f98</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Linux" icon="linux.png" os="linux"/>
  </metadata>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='3'/>
    <vcpupin vcpu='1' cpuset='7'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/c84f0292-195b-60b3-3e17-a3b2b5dd3f98_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='1' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='vmdk' cache='writeback'/>
      <source file='/mnt/user/domains/Hassio/haos_ova-7.6.vmdk'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
	<serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_3DE24#####'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial0'/>
      <address type='usb' bus='0' port='4'/>
    </serial>
    <interface type='bridge'>
      <mac address='52:54:00:49:e3:7a'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='2'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1cf1'/>
        <product id='0x0030'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'/>
</domain>

I tried a couple of combinations with the serial to take off usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_ and do the serial with the 3 at the start or not but the VM wont start unfortunately.

Error message:

I also saw that it looks like the Bus of the stick stays the same but the device number is changing. Not sure if that's something to do with this.
I left Q35 at 4.2 as I had problems in the past getting the VM to start if I go higher.

 

Thanks again for all the help so far.

 

[ghost82]

On 6/9/2022 at 6:12 AM, Grsh said:

3DE24#####

that 3 in front of the string is wrong.

Double check within the unraid terminal to see if the device is there, cd /dev/serial/by-id/ nad perform an ls command.

[shulgach]

On 6/8/2022 at 7:27 AM, MrSir said:

I tried the solutions here, constant restarts, unmounts, remounts none of it worked.

 

The solution was to Rollback to Unraid 6.9.2 and now my Conbee II is working with my Home Assistant VM again, weird bug hope it gets solved, until then I'm sticking with 6.9.2

 

Just adding on - this was the only thing that worked for me. As soon as I rolled back, everything worked flawlessly. Looks like a regression with 6.10.x.

[jgosnell]

1 hour ago, shulgach said:

 

Just adding on - this was the only thing that worked for me. As soon as I rolled back, everything worked flawlessly. Looks like a regression with 6.10.x.

I just upgraded to 6.10.3 and my ConBee II is still working using the original steps above.  I am running the latest version of Home Assistant as well.

[shulgach]

14 minutes ago, jgosnell said:

I just upgraded to 6.10.3 and my ConBee II is still working using the original steps above.  I am running the latest version of Home Assistant as well.

Strange.. just tried an upgrade to 6.10.3 from 6.9.2, and still broken for me - I must be doing something wrong or my NUC has some special features that are messing with the USB bus.

[chris_netsmart]

ok I got it to work in the end,  after following the steps here I ended up reverting to a older version of Unraid. 6.9 - which then allows me to plug and play my Zigbee device and my VM Home Assistant saw it and configured it straight away

 

NOTE: I had to recreate my Home Assistant VM and also some of my dockers like PLEX had to be created.

[jgosnell]

On 6/16/2022 at 3:26 PM, shulgach said:

Strange.. just tried an upgrade to 6.10.3 from 6.9.2, and still broken for me - I must be doing something wrong or my NUC has some special features that are messing with the USB bus.

Check your VM xml.  Mine had a double entry for the ConBee after upgrade where one of the entries was wrapped in a <console> tag.  I removed that entry and only left the <serial> tagged entry and everything started working.  I have no idea what is adding the second <console> entry though.

[shulgach]

Figured it out - I didn't realize that the path in Home Assistant changed - it should be the serial path instead of the "by-id" path. This link helped me figure that out (specifically point 8 in the post): https://community.home-assistant.io/t/solved-conbee-2-fails-to-connect-to-zha-ha-in-unraid-vm/431276/2

Edited June 19, 2022 by shulgach

[Grsh]

Thanks for all the suggestions guys. I rolled Unraid a version back to get the lights back online. Might give it another try a little bit later.
Only annoying thing was I had to reconnect all Zigbee devices after the downgrade.

[Mirano]

Changing to Q35-4.2 and  adding the following to the config succesfully added the device:

 

<serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE******-if00'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='serial0'/>
      <address type='usb' bus='0' port='4'/>
    </serial>

[LP_LP_LP]

I post here hoping that it will be helpful in the future. As of July 2022 (Unraid v6.10.3), I still encountered the USB passthrough issue described above with the Home Assistant VM. Adding the lines in the XML file did not solve the issue alone: I also had to revert back to Q35-4.2. 

[sir_storealot]

I also encountered this after upgrading from 6.9 to 6.10.3

Following the link in this thread (https://community.home-assistant.io/t/solved-conbee-2-fails-to-connect-to-zha-ha-in-unraid-vm/431276/2) worked perfectly.

In my case, no furhter actions like qemu downgrades etc. were necessary.

[SimonF]

The latest release of my USB Manager plugin now supports Serial USB mappings when the VM Starts. I am still working on hotplug when the device is connected.

 

See this post for screen shots.

 

Edited July 15, 2022 by SimonF

[jgosnell]

2 hours ago, SimonF said:

The latest release of my USB Manager plugin now supports Serial USB mappings when the VM Starts. I am still working on hotplug when the device is connected.

 

See this post for screen shots.

 

Is there a setting that we would need to change to specify the serial connection option or is it automatically determined somehow?

 

Nevermind....

After updating the plugin, I now see the toggle for "Connect as Serial Only".  Thank you.

Edited July 15, 2022 by jgosnell

[SimonF]

4 minutes ago, jgosnell said:

Is there a setting that we would need to change to specify the serial connection option or is it automatically determined somehow?

We you set the VM Mapping you enable connect as serial only.

 

For my ZWAVE Stick I can connect just as a normal device.

 

Edited July 15, 2022 by SimonF

[jgosnell]

Thank you very much.  I will test with my ConBee II and let you know the results.

[jgosnell]

28 minutes ago, SimonF said:

We you set the VM Mapping you enable connect as serial only.

 

For my ZWAVE Stick I can connect just as a normal device.

 

I shut down my VM and edited the XML file to remove all references of a serial connection and all references of my ConBee II.  I then enabled the automatic attachment of the device in USB Manager and selected to attach as serial only.  Started up the VM and all seems good.  Home Assistant sees the device and my integration and all devices show properly.  Great Job!!

One question for you though.  When I go back to view the VM XML, I see a serial entry and a console entry for the ConBee like in this snippet.  Is that normal?

    <serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DEXXXXXXX-if00'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='ua-serial006002'/>
      <address type='usb' bus='0' port='4'/>
    </serial>
    <console type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DEXXXXXXX-if00'/>
      <target type='serial' port='1'/>
      <alias name='ua-serial006002'/>
      <address type='usb' bus='0' port='4'/>
    </console>

Also, for anyone else seeing this.  I did not have to edit the "core.device_registry" file under the .storage directory within HASS.  What I detailed above is the only changes I had to make. 

Edited July 15, 2022 by jgosnell

[SimonF]

6 minutes ago, jgosnell said:

I shut down my VM and edited the XML file to remove all references of a serial connection and all references of my ConBee II.  I then enabled the automatic attachment of the device in USB Manager and selected to attach as serial only.  Started up the VM and all seems good.  Home Assistant sees the device and my integration and all devices show properly.  Great Job!!

One question for you though.  When I go back to view the VM XML, I see a serial entry and a console entry for the ConBee like in this snippet.  Is that normal?

    <serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DEXXXXXXX-if00'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='ua-serial006002'/>
      <address type='usb' bus='0' port='4'/>
    </serial>
    <console type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DEXXXXXXX-if00'/>
      <target type='serial' port='1'/>
      <alias name='ua-serial006002'/>
      <address type='usb' bus='0' port='4'/>
    </console>

 

Not seen that on my system

 

 

   <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <serial type='dev'>
      <source path='/dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DEXX-if00'/>
      <target type='usb-serial' port='1'>
        <model name='usb-serial'/>
      </target>
      <alias name='ua-serial001015'/>
      <address type='usb' bus='0' port='4'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>

 

Edited July 15, 2022 by SimonF