• [6.8-RC1] kernel: tun: unexpected GSO


    Outcasst
    • Retest Minor
    Oct 12 03:35:17 Storage kernel: tun: unexpected GSO type: 0x0, gso_size 1357, hdr_len 1411
    Oct 12 03:35:17 Storage kernel: tun: 13 e4 3d f7 10 86 b8 9e 87 b1 5f 81 d9 7a 98 c9 ..=......._..z..
    Oct 12 03:35:17 Storage kernel: tun: 26 fa 2d 78 50 03 f2 b2 22 55 bc 68 29 75 83 46 &.-xP..."U.h)u.F
    Oct 12 03:35:17 Storage kernel: tun: 04 35 d4 e4 71 d8 5c 04 e3 e2 a2 6d 4e 1f 22 9d .5..q.\....mN.".
    Oct 12 03:35:17 Storage kernel: tun: 6f 97 72 60 c9 63 2b dc f4 ec c7 4f 68 60 66 9e o.r`.c+....Oh`f.

    Getting the above message repeated over and over again in the log whenever a docker tries to access the NIC.

    storage-diagnostics-20191012-0237.zip

    • Thanks 3



    User Feedback

    Recommended Comments



    I do not have good info regarding when this started - I first noticed after upgrading to RC6, but I just rolled back to RC5 and it's still there, and due to really light VM usage, it's possible I just didn't notice. So count me out as far as useful datapoints. :/

    Link to comment

    to be honest, I don't think it is unraid issue, my best guess is it is the docker issue with how the docker manage the network, it is better to report to docker team

    Link to comment
    13 hours ago, limetech said:

    For those reporting this issue: what version of Unraid OS were you running that did not exhibit this issue or similar issue reported here:

     

    I cant understand the question. So im just reporting how it was for me, it startet with some RC, i dont know 2 or 3, never had such problems before. I didnt notice it in any 6.7.

     

    I run dockers in br0 and vms also. And since bridge cant talk to bridge and or host, i need to use br0.

     

    All dockers just run fine together, bridge, br0, host, what ever. As soon as i start the windows 10 VM (or any other) at some point i get these spam in the logs which only disappear when i stop the vm. Sometimes its instant, sometimes it takes minutes.

     

     

     

    unraid-server-diagnostics-20191127-1136.zip

    Edited by nuhll
    Link to comment

    I'm on RC7. For me the errors only appear 4 times and stop when I start a VM (I have 4 nic ports bonded, don't know if it's a coincidence).

    Maybe i'ts because I only have 2 dockers with custom IP but so far I have no log spamming as reported by other people here.

    Link to comment
    28 minutes ago, Nephilgrim said:

    I'm on RC7. For me the errors only appear 4 times and stop when I start a VM (I have 4 nic ports bonded, don't know if it's a coincidence).

    Maybe i'ts because I only have 2 dockers with custom IP but so far I have no log spamming as reported by other people here.

    Do you have a lot of traffic going through it? When I start up my VM's, I get a couple of log entries, but if I don't really use it nothing happens. It's when I try to put some bandwidth through it (going online to do a speedtest for example) the logged errors start flowin'.

    Link to comment

    Tried to force it: Launched a speedtest + fast.com + downloading a game from steam to stress the connection (300/300 fiber) and no new log entries since I started the server + mainVM 2 hours ago.

    Edit: Also tried to launch a speedtest from the unraid plugin (out of the VM) but no more logs neither:

    image.png.0049a587d81f3fc05dfadc48e9258c78.png

    So far I only see those logs messages once (or once per nic port) when a VM is booted. I suppose that this could be a expected behaviour since they stop.

    Edited by Nephilgrim
    Another test.
    Link to comment

    Just curious, did you all check wireguard?  That was the issue on my machine, soon as I disabled it all the errors stopped.

    Link to comment

    Was disabling Wireguard enough for you? I just gave that a shot - disabled my wireguard tunnels and then directly started up my VM (on br0.xx vlan) and it started erroring all over the place as soon as I logged in and started using it.

    Link to comment
    Quote

    Tower kernel: traps: lsof[18796] general protection fault ip:14bce164fb8e sp:a2713be92af2c4fa error:0 in libc-2.30.so[14bce1630000+16b000]

    And it's back RC 7 Unraid Nvidia build 6.8.0, the funny thing is it didn't happen at docker and 1 VM start like before, it happened at 4am Dec 4th when nothing much was happening. Also deleted the inactive Wireguard Tun I had set up will monitor after removal. I'm also noticing something called general protection fault that I've not noticed before. Log attached

    tower-diagnostics-20191204-2021.zip

    Edited by Fiservedpi
    Wire guard Tun removed
    Link to comment

    After upgrading to RC8 - My dockers are all running, my VM is up on my VLAN interface, no errors since last night when I upgraded. Looks like I'll be sticking with RC8 for a while! Thank you.

    Link to comment

    I would like the report the same as Nephilgrim on 6.8.0-rc5.  I get between 1 and 4 "unexpected GSO type" log messages when I start my docker on br0, and then nothing else. That is with one pihole docker on br0 w/ static ip and one win10 vm also on br0. I have an additional 9 docker containers on autostart in bridge network mode.

     

    Edit: Just fired up another (ubuntu) vm and hit speedtest on both VMs while pihole is running and still no more unexpected GSO messages.

    Edited by Skitals
    Link to comment

    i found an easy workaround it seams .... just don't use the virtio driver in the vm

     

    from:

        <interface type='bridge'>
          <mac address=**:**:**:**:**:**/>
          <source bridge='br0'/>
          <model type='virtio'/>
          <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
        </interface>

    to:

        <interface type='bridge'>
          <mac address=**:**:**:**:**:**/>
          <source bridge='br0'/>
          <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
        </interface>

     

    removing the model will revert back to default driver downside is the little higher cpu usage and lower throughput  

    • Like 1
    • Thanks 1
    Link to comment
        <interface type='bridge'>
          <mac address='52:54:00:39:64:90'/>
          <source bridge='br0'/>
          <model type='virtio-net'/>
          <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
        </interface>

    ok this seams to fix the issues for me no more dmesg spam

    changed from virtio to virtio-net

    will do more testing

    • Like 1
    Link to comment

    I've changed my VM's NIC also from 'virtio' to

     

    <model type='virtio-net'/>

     

    and since then no more "unexpected GSO type" in my logs.

     

    I've 3 VM's (2 Win + 1 CentOS) running and 3 dockers on br0 with 5.5 kernel.

    • Like 4
    • Thanks 1
    Link to comment

    For the upcoming version the default driver is changed to "virtio-net".

     

    This means newly created VMs will use the revised driver, but existing VMs need manual adjustment.

    • Like 4
    Link to comment

    I'm getting the KERNEL: TUN: UNEXPECTED GSO log spam on 6.9 beta 29. Fills log all the way until it hit's memory limit.

    Seems to stop if shutdown a particular win10 vm, i have 3 win10 vm's all running on virtio-net doesn't seem to do it with the other 2 vm's.

    Link to comment

    Same here - 2 linux Vms and the log filled quickly

    Going to update to beta 30 and see if that helps

     

    Saw you comment under the beta-30 release and one of my Vms had reverted to virto also.. .Strange, but that should fix it :)

    Edited by steini84
    Link to comment

    Version: 6.9.0-rc2 

    Same Issues. Array crash overnight

    1 x Windows VM on br0 (only VM)
    4 x Dockers on br0

    Just moved the VM to br1 on another physical NIC and monitoring. So far no issues.

    Link to comment

    Hi, I've been on the same issue. An 'easy way' to fix this (temporarely, but without crazy always runnings scripts) is by adding something in the blacklist of rsyslogd. In my case I added the following:

     

    Quote

    /etc/rsyslog.d/01-blocklist.conf

    :msg,contains,"tun:"


    KEEP IN MIND THAT I ADDED, NOT REPLACED. There were 2 other lines in the file for me:

    :msg,contains,"Error: Nothing to do" stop
    :msg,contains,"user \"logout\" was not found in \"/etc/nginx/htpasswd\", client" stop

     

    Hope someone can use this info as workaround. I really couldn't change anything like said before, it would ruin the dockers functions due to local network scans.

    Link to comment
    On 3/1/2021 at 8:46 PM, maxstevens2 said:

    Hi, I've been on the same issue. An 'easy way' to fix this (temporarely, but without crazy always runnings scripts) is by adding something in the blacklist of rsyslogd. In my case I added the following:

     

    
    :msg,contains,"tun:"


    KEEP IN MIND THAT I ADDED, NOT REPLACED. There were 2 other lines in the file for me:

    :msg,contains,"Error: Nothing to do" stop
    :msg,contains,"user \"logout\" was not found in \"/etc/nginx/htpasswd\", client" stop

     

    Hope someone can use this info as workaround. I really couldn't change anything like said before, it would ruin the dockers functions due to local network scans.

    Just found out this doesn't seem to work..

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.