• [6.9.0-beta25] "Host access to custom networks" option in Docker changes MAC address of Unraid Server


    cybrnook
    • Closed

    Been scratching my head all afternoon on this one, but I think I finally narrowed it down. What I found is that when you have "Host access to custom networks" enabled in the Docker options, it's fudging the MAC address of the Unraid server as a whole.

     

    image.thumb.png.26263c7490cc9bac08165e278e78cf8e.png

     

    I found this out because I was attempting to debug the "macvlan" Call Trace issue when we have custom IP's assigned to Docker Containers. To me, having custom IP's (Custom br0) is a requirement so I can implement firewall rules on a per docker container basis. As well, just taking the approach of creating another VLAN on my network is not a suitable workaround because that just adds additional complexity when trying to route to those containers from my primary network. Anyways.....

     

    I thought what the hell, let me swap in a different network card to see if it makes a difference (Intel X550 vs X540 vs 82599ES etc..) since the assumption is some people have this issue and others don't, perhaps it's a hardware or driver issue?

     

    So, I installed my different network card, and proceeded to do the usual, set static IP in unraid, then in pfsense make sure I tag the proper MAC address (the new one) to make sure I also assign the same static IP. This is where it started getting weird....

     

    The actual MAC of my adapter begins with D0 and ends with 9C:

    image.thumb.png.a71a67d8dfc0b3ddcfc7754cdc2f8249.png

     

    So when I went into pfsense, I was ready to tag D0..9C, but didn't see it. So I checked the ARP table, and low and behold I saw a MAC of 36...52:

    image.thumb.png.ba69735f6b70489c75cf90705318a8df.png

     

    If I disable "Host access to custom networks", then the real MAC comes back:

    image.thumb.png.8da690409f8c3d9983f3dc5233b5138b.png

     

    image.thumb.png.8d44061488f43211d29fe89630f79c3a.png

     

    At least for me, this is an issue as this means that when enabling this option, any static IP assignment I have in my router would be ignored since the MAC is randomized each time with macvlan. As well, I am not sure of the additional macvlan ramifications of this when the MAC for an IP is changing.

     

    Perhaps this is working as designed, but I wanted to bring this to light as it may be related to the calltrace for macvlan, and for sure surely breaks my static IP rule that I define in my router.

     

    And yes, I know of course that I could set a static IP in unraid (which I do). But I also like to make sure that IP is static in my router as well, as I have had occasions where I needed to nuke the network.cfg file, and still want to come up on the same IP.

     

     

    @limetech I am tagging as "other" for now since I can't make out if this is Urgent, Annoyance, Other or by design.




    User Feedback

    Recommended Comments

    Just to add, I ended up breaking down and building out another VLAN for dockers in the meantime. However, I think this is a bug as the MAC of unraid should not be changing. Way too much of a pain in the butt though, when the check box should have avoided this all together 🙂

    image.thumb.png.dd597517d19c442cd77a9122b75f4508.png

    image.thumb.png.ea2da6d6f8fbd292057ad9ee5cf6c56a.png

    image.png.f09245d9dd847a9bdc475bc140b7e1c7.png

    image.png.e437afa7b25f07e9a5828177ad61ef52.png

    image.thumb.png.7ee9c29708e623a1bafc64d90cbb4c71.png

    image.thumb.png.0ab8d78a7830ddedd5bb7da2338f629c.png

    image.thumb.png.e9f67aee29f27599f5587bcf537aa670.png

    Edited by cybrnook
    Link to comment

    When you enable "Host access to custom networks" it will create a shim network interface, e.g. shim-br0.

    This shim interface has the same IP address as the main interface (br0), but it is using a different MAC address (because it is a different interface/network).

     

    This shim interface is used to make Docker think it is accessing a different system, in other words it makes access possible between docker and the host system.

     

    Your pfsense firewall gets confused by this, it sees the same IP address with two different MAC addresses (and two different networks).

    I am not using pfsense and don't know how to tell it to understand "this situation". In worst case when pfsense can not cope with this, you need to disable the host access function, which is rarely needed anyway.

     

    This is not a bug but an implementation by design.

    • Thanks 1
    Link to comment
    On 8/8/2020 at 3:05 AM, bonienl said:

    In worst case when pfsense can not cope with this, you need to disable the host access function, which is rarely needed anyway.

    Thanks for the response @bonienl that helps paint of picture of what's going on.

     

    To your line I quoted above. I would say this option is needed for those who want to assign a static IP to your docker containers, but also don't want to have to go through the trouble of setting up a separate VLAN for those docker containers because either it's too hard, or their equipment does not support it.

     

    What I found in the past was that when you assign static IP's to your docker containers, Unraid by default blocks those containers from talking to one another. So if you have a setup like mine (and I assume others run something similar for automation) Sonarr and Radarr need to be able to talk to Jackett, they also need to be able to talk to your torrent client to pass information back and forth. Without this option enabled, this won't work. Or, creating a separate network for these dockers, hence the need for a VLAN.

    Edited by cybrnook
    Link to comment
    18 hours ago, cybrnook said:

    Without this option enabled, this won't work.

    Docker containers which are in the "same" network can always communicate with each other.

    So either your containers are ALL bridge/host network or are ALL custom network.

     

    Link to comment
    5 hours ago, bonienl said:

    So either your containers are ALL bridge/host network or are ALL custom network.

    This may explain it then. Is this a hard requirement that in order to use some docker containers with static IP's, then ALL my docker need to be set to use a static IP? I can't have one using host, a few using bridge, then a handful using custom br0, they would all need to be custom br0?

    Link to comment

    I guess I will set this to closed as everything I pointed out so far seems to be either specific to me or expected. Thanks @bonienl for explaining in detail my concerns.

     

    Perhaps it was timing, or something else on my side, but without having Host Access enabled, my custom br0 dockers were not able to talk to one another, at least at the time. Which is what led me down this path anyways.

     

    regardless, so far using a VLAN has removed the call traces I was seeing, and with the rules in the firewall , everyone can talk to everyone, so I guess that's good enough 🙂

     

    Edited by cybrnook
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.