• 6.9.0-beta22 again unexpected GSO type


    NewDisplayName
    • Minor

    Hi,

    ive posted this issue around 6.5 unraid and it got "fixed"? by 6.8.3, now its back... it seems to happen when i start the Windows 10 VM. I already tried to edit the template and click update, but didnt changed anything.

     

    While its spamming the log with



    Jul 7 20:16:18 Unraid-Server kernel: tun: unexpected GSO type: 0x0, gso_size 1384, hdr_len 1450
    Jul 7 20:16:18 Unraid-Server kernel: tun: c0 7a f0 21 a5 fd 21 2a 72 d6 16 97 13 5f df 47 .z.!..!*r...._.G
    Jul 7 20:16:18 Unraid-Server kernel: tun: 24 44 8b 16 37 ef 25 74 da 6f f6 3b 4a ba 2e 19 $D..7.%t.o.;J...
    Jul 7 20:16:18 Unraid-Server kernel: tun: 8f e8 8f 9c 27 25 c4 e0 ad 39 61 70 15 c3 34 7a ....'%...9ap..4z
    Jul 7 20:16:18 Unraid-Server kernel: tun: 8d c5 6b 9b ac cf 45 bc e2 28 2e 75 56 58 00 00 ..k...E..(.uVX..

    the server is kinda unresponsive.

     

    Is this supposed to work or what i am doing wrong?


    Or is this bug still unresolved?

     




    User Feedback

    Recommended Comments



    Can you tell me exactly what i have to change in the XML?

     



        <interface type='bridge'>
          <mac address='52:54:00:50:c9:66'/>
          <source bridge='br0'/>
          <target dev='vnet0'/>
          <model type='virtio-net'/>
          <alias name='net0'/>
          <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
        </interface>
        <interface type='bridge'>
          <mac address='52:54:00:93:e2:57'/>
          <source bridge='br1'/>
          <target dev='vnet1'/>
          <model type='virtio-net'/>
          <alias name='net1'/>
          <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
        </interface>

    Edited by nuhll
    Link to comment

    Okay, NVM. Now it seem to have changed after i checked if it had changed it, I guess you need to "update" NOT in XML view...?

     

    No error so far, ill report back if it starts again, besides that it seems to be fixxed.

    Link to comment
    35 minutes ago, nuhll said:

    I guess you need to "update" NOT in XML view...?

    correct

     

    From OP:

    Quote

    In most cases this can be accomplished simply by clicking Update in "Form View" on the VM Edit page.

     

    Link to comment
    45 minutes ago, nuhll said:

    Can you tell me exactly what i have to change in the XML?

    Note that I found a performance penalty with virtio-net. I can't get above 200MB/s over the network between VM and disk share. That's ok with SATA SSD but is rather slow compared to the usual GB/s I get with my NVMe.

     

    Instead of using virtio-net, I keep the type as virtio but change the machine type to Q35-5.0 (which is available with 6.9.0 beta). Haven't had those GSO error since while still doing GB/s to cache. 

     

    • Thanks 1
    Link to comment
    4 minutes ago, testdasi said:

    Note that I found a performance penalty with virtio-net. I can't get above 200MB/s over the network between VM and disk share. That's ok with SATA SSD but is rather slow compared to the usual GB/s I get with my NVMe.

     

    Instead of using virtio-net, I keep the type as virtio but change the machine type to Q35-5.0 (which is available with 6.9.0 beta). Haven't had those GSO error since while still doing GB/s to cache. 

     

    What do you think?  Better to revert code to set model back from 'virtio-net' to 'virtio' and then tell everyone to update their machine type?  (I don't think we should "auto convert" their machine type).

    Link to comment
    57 minutes ago, testdasi said:

    Instead of using virtio-net, I keep the type as virtio but change the machine type to Q35-5.0 (which is available with 6.9.0 beta).

    Do you know if using machine type i44fx-5.0 also works?

    Link to comment

    My test results

     

    1. Q35-5.0 + virtio-net

    # iperf3 -c 10.0.101.28
    Connecting to host 10.0.101.28, port 5201
    [  5] local 10.0.101.11 port 37044 connected to 10.0.101.28 port 5201
    [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    [  5]   0.00-1.00   sec   412 MBytes  3.46 Gbits/sec    0    390 KBytes
    [  5]   1.00-2.00   sec   438 MBytes  3.67 Gbits/sec    0    370 KBytes
    [  5]   2.00-3.00   sec   419 MBytes  3.51 Gbits/sec    0    376 KBytes
    [  5]   3.00-4.00   sec   401 MBytes  3.37 Gbits/sec    0    379 KBytes
    [  5]   4.00-5.00   sec   400 MBytes  3.36 Gbits/sec    0    379 KBytes
    [  5]   5.00-6.00   sec   439 MBytes  3.68 Gbits/sec    0    430 KBytes
    [  5]   6.00-7.00   sec   438 MBytes  3.67 Gbits/sec    0    393 KBytes
    [  5]   7.00-8.00   sec   441 MBytes  3.70 Gbits/sec    0    427 KBytes
    [  5]   8.00-9.00   sec   415 MBytes  3.48 Gbits/sec    0    362 KBytes
    [  5]   9.00-10.00  sec   429 MBytes  3.60 Gbits/sec    0   5.66 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.00  sec  4.13 GBytes  3.55 Gbits/sec    0             sender
    [  5]   0.00-10.00  sec  4.13 GBytes  3.54 Gbits/sec                  receiver

     

    2. Q35-5.0 + virtio

    # iperf3 -c 10.0.101.28
    Connecting to host 10.0.101.28, port 5201
    [  5] local 10.0.101.11 port 37092 connected to 10.0.101.28 port 5201
    [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    [  5]   0.00-1.00   sec  1.11 GBytes  9.55 Gbits/sec    1    766 KBytes
    [  5]   1.00-2.00   sec  1.03 GBytes  8.87 Gbits/sec  204    602 KBytes
    [  5]   2.00-3.00   sec  1.03 GBytes  8.88 Gbits/sec    2    532 KBytes
    [  5]   3.00-4.00   sec  1.06 GBytes  9.14 Gbits/sec    0    543 KBytes
    [  5]   4.00-5.00   sec  1.10 GBytes  9.45 Gbits/sec    0    447 KBytes
    [  5]   5.00-6.00   sec  1.10 GBytes  9.42 Gbits/sec    0    529 KBytes
    [  5]   6.00-7.00   sec  1.07 GBytes  9.17 Gbits/sec    1    540 KBytes
    [  5]   7.00-8.00   sec  1.03 GBytes  8.86 Gbits/sec  778    447 KBytes
    [  5]   8.00-9.00   sec  1.05 GBytes  9.06 Gbits/sec  116    522 KBytes
    [  5]   9.00-10.00  sec  1.06 GBytes  9.13 Gbits/sec  174    520 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.00  sec  10.7 GBytes  9.15 Gbits/sec  1276             sender
    [  5]   0.00-10.00  sec  10.7 GBytes  9.15 Gbits/sec                  receiver

     

    There is a performance improvement when the virtio driver is used. BUT .... still GSO type errors

     

    Also restarting the system with virtio driver (VM autostart) results in a call trace.

    Jul 7 22:12:40 vesta kernel: tun: unexpected GSO type: 0x0, gso_size 1376, hdr_len 1442
    Jul 7 22:12:40 vesta kernel: tun: 02 42 0a 00 65 65 18 e8 29 bd 80 c7 08 00 45 00 .B..ee..).....E.
    Jul 7 22:12:40 vesta kernel: tun: 05 94 43 d8 40 00 40 06 13 26 0a 00 65 01 0a 00 ..C.@.@..&..e...
    Jul 7 22:12:40 vesta kernel: tun: 65 65 88 4c 1f 90 45 e1 7c 7b ab 96 c4 f5 80 18 ee.L..E.|{......
    Jul 7 22:12:40 vesta kernel: tun: 00 73 46 2f 00 00 01 01 08 0a 1a 7c 00 fd ca 17 .sF/.......|....
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: Multiple Corrected error received: 0000:00:03.2
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: PCIe Bus Error: severity=Corrected, type=Data Link Layer, (Transmitter ID)
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: device [8086:6f0a] error status/mask=00001100/00002000
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: [ 8] Rollover
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: [12] Timeout
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: Error of this Agent is reported first
    Jul 7 22:14:27 vesta kernel: nvme 0000:06:00.0: AER: PCIe Bus Error: severity=Corrected, type=Data Link Layer, (Receiver ID)
    Jul 7 22:14:27 vesta kernel: nvme 0000:06:00.0: AER: device [2646:2263] error status/mask=00000040/0000e000
    Jul 7 22:14:27 vesta kernel: nvme 0000:06:00.0: AER: [ 6] BadTLP
    Jul 7 22:14:27 vesta kernel: pcieport 0000:00:03.2: AER: Corrected error received: 0000:06:00.0
    Jul 7 22:15:03 vesta kernel: tun: unexpected GSO type: 0x0, gso_size 1397, hdr_len 1463
    Jul 7 22:15:03 vesta kernel: tun: 02 42 0a 00 65 65 18 e8 29 bd 80 c7 08 00 45 00 .B..ee..).....E.
    Jul 7 22:15:03 vesta kernel: tun: 05 a9 55 f3 40 00 40 06 00 f6 0a 00 65 01 0a 00 ..U.@[email protected]...
    Jul 7 22:15:03 vesta kernel: tun: 65 65 88 50 1f 90 dd 67 a6 7f e9 bc 7e 92 80 18 ee.P...g....~...
    Jul 7 22:15:03 vesta kernel: tun: 00 73 4f b0 00 00 01 01 08 0a 1a 7c 38 de ca 19 .sO........|8...

     

    3. i440fx-5.0 + virtio-net

    # iperf3 -c 10.0.101.29
    Connecting to host 10.0.101.29, port 5201
    [  5] local 10.0.101.11 port 47198 connected to 10.0.101.29 port 5201
    [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    [  5]   0.00-1.00   sec   428 MBytes  3.59 Gbits/sec    0    373 KBytes
    [  5]   1.00-2.00   sec   406 MBytes  3.41 Gbits/sec    0    294 KBytes
    [  5]   2.00-3.00   sec   406 MBytes  3.41 Gbits/sec    0    297 KBytes
    [  5]   3.00-4.00   sec   402 MBytes  3.38 Gbits/sec    0    305 KBytes
    [  5]   4.00-5.00   sec   412 MBytes  3.46 Gbits/sec    0    297 KBytes
    [  5]   5.00-6.00   sec   424 MBytes  3.55 Gbits/sec    0    308 KBytes
    [  5]   6.00-7.00   sec   420 MBytes  3.52 Gbits/sec    0    294 KBytes
    [  5]   7.00-8.00   sec   416 MBytes  3.49 Gbits/sec    0    303 KBytes
    [  5]   8.00-9.00   sec   385 MBytes  3.23 Gbits/sec    0    300 KBytes
    [  5]   9.00-10.00  sec   394 MBytes  3.30 Gbits/sec    0    308 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.00  sec  4.00 GBytes  3.43 Gbits/sec    0             sender
    [  5]   0.00-10.00  sec  3.99 GBytes  3.43 Gbits/sec                  receiver

     

    4. i440fx-5.0 + virtio

    # iperf3 -c 10.0.101.29
    Connecting to host 10.0.101.29, port 5201
    [  5] local 10.0.101.11 port 47294 connected to 10.0.101.29 port 5201
    [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    [  5]   0.00-1.00   sec   200 MBytes  1.68 Gbits/sec    3    266 KBytes
    [  5]   1.00-2.00   sec  73.8 MBytes   619 Mbits/sec    0    277 KBytes
    [  5]   2.00-3.00   sec  76.7 MBytes   643 Mbits/sec    0    274 KBytes
    [  5]   3.00-4.00   sec  75.7 MBytes   635 Mbits/sec    0    272 KBytes
    [  5]   4.00-5.00   sec  75.8 MBytes   635 Mbits/sec    0    274 KBytes
    [  5]   5.00-6.00   sec  77.7 MBytes   652 Mbits/sec    0    269 KBytes
    [  5]   6.00-7.00   sec  80.6 MBytes   676 Mbits/sec    0    274 KBytes
    [  5]   7.00-8.00   sec  76.7 MBytes   643 Mbits/sec    0    272 KBytes
    [  5]   8.00-9.00   sec  75.6 MBytes   634 Mbits/sec    0    269 KBytes
    [  5]   9.00-10.00  sec  76.6 MBytes   642 Mbits/sec    0    269 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.00  sec   889 MBytes   746 Mbits/sec    3             sender
    [  5]   0.00-10.00  sec   887 MBytes   744 Mbits/sec                  receiver

     

    With i440fx-5.0 we see the exact opposite. The virtio driver has much lower performance.

    Edited by bonienl
    Link to comment

    Oh.. thats bad. Bc i need that windows 10 VM for transcoding 4k files  LOL (using 10Gbt)

     

    edit:
    yea only around 180-200mb/s instead of 7-9 Gbits

    Edited by nuhll
    Link to comment

    When the virtio-net driver is used, there are no GSO type errors. For the moment this is the safest solution.

     

     

    Link to comment
    4 minutes ago, nuhll said:

    Should i try 

    4. i440fx-5.0 + virtio

    It gives worst results than virtio-net for me

    • Like 1
    Link to comment

     

    2 hours ago, testdasi said:

    Note that I found a performance penalty with virtio-net. I can't get above 200MB/s over the network between VM and disk share. That's ok with SATA SSD but is rather slow compared to the usual GB/s I get with my NVMe.

     

    Instead of using virtio-net, I keep the type as virtio but change the machine type to Q35-5.0 (which is available with 6.9.0 beta). Haven't had those GSO error since while still doing GB/s to cache. 

     

     

    Thanks for your input.

     

    I tried to set it to Q35 but it wont boot and show me some screen (via inbuild VNC) where it like want to know what to boot... how you converted?

    Edited by nuhll
    Link to comment
    1 minute ago, nuhll said:

     

     

    Thanks for your input.

     

    I tried to set it to Q35 but it wont boot and show me some screen (via inbuild VNC) where i cant press anything.. how you converted?

    Moving between i440fx and Q35 isn't quite simple if you use SeaBIOS.

    I would say try i440fx-5.0 + virtio and test to see if you get any performance improvement + whether there's any GSO.

     

    I don't have any GSO error with Q35-5.0 and i440fx-5.0 for days already; hence, worth a try.

    Link to comment

    XML error: The PCI controller with index='0' must be model='pci-root' for this machine type, but model='pcie-root' was found instead

     

    :(

    I changed that now its

     

     

    XML error: The device at PCI address 0000:00:02.2 cannot be plugged into the PCI controller with index='0'. It requires a controller that accepts a pcie-root-port.

     

    wtf.

    Edited by nuhll
    Link to comment
    4 minutes ago, testdasi said:

    I don't have any GSO error with Q35-5.0 and i440fx-5.0 for days already; hence, worth a try.

    Using OVMF bios for both of those machine types?

    Link to comment
    17 minutes ago, testdasi said:

    I don't have any GSO error with Q35-5.0 and i440fx-5.0 for days already

    Perhaps there is a hardware dependency, but in my case I get the GSO type errors and a call trace at system start up.

     

    Link to comment
    1 hour ago, limetech said:

    Using OVMF bios for both of those machine types?

    OVMF with Q35, SeaBIOS with i440fx.

    51 minutes ago, bonienl said:

    Perhaps there is a hardware dependency, but in my case I get the GSO type errors and a call trace at system start up.

    Haven't thought of that but could very well be.

    Link to comment
    1 hour ago, nuhll said:

    Okay arounmd 24h later i got around 3 GSO after "link becomes ready". But i think that doesnt interfer with anything!?

     

    Around 12:05 08.07.20

    unraid-server-diagnostics-20200708-1548.zip 162.27 kB · 0 downloads

    I don't think those GSO relate to the VM.

    Those errors started after the appdata backup job kicked off, which presumably stopped the dockers. I also see Restarting PiHole / Hydra.

    Could be related to the docker custom network issue?

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.