• Upgrade to 6.9 failed: no network


    Helmonder
    • Solved Urgent

    The upgrade from 6.8.3 60 6.9 failed in my case.

     

    The update itself succeeded, reboot worked but the system did not become active on the network. No GUI, no share, no telnet.

     

    - I was able to IPMI into the system and get on console;

    - The system could ping its own IP address from console but not the default gateway. GUI mode also did not work from console; "page not found";

    - Network setup appeared good, correct default gateway and such (ROUTE -N)

    - The port was shown as "linkdown" (IP ROUTE SHOW);

    - I have attached a diagnostics file (note: this is for the current situation, so with 6.8.3).

    - I also attach a syslog that I was able to capture on the flashdrive for the 6.9 config. Hopefully that will give someone some insight.

    - I reverted to 6.8.3 by restoring the flashdrive from the backup, that seems to have worked 100%.

     

    I am suspecting the issue lies with my ethernet controller:

    IOMMU group 12:[15b3:6750] 03:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)

     

    Also: 

    Tower kernel: emhttpd[17268]: segfault at 0 ip 00001489487cff36 sp 0000148943dfde80 error 4 in libc-2.30.so[1489487ab000+16b000]

     

    A segfault in emhttpd would also cause the GUI to be not reachable but I figure that would make the GUI unreachable but not telnet/shares.. So probably emhttpd crashed because of the closed eth0 ?

     

    tower-diagnostics-20210304-1414.zip syslog.bad

    image.png

    image.png



    User Feedback

    Recommended Comments

    Just tried to do it again while having a bond active on my br0 that also had the native ports of my motherboards in it... 

     

    that resulted in a "boot error" 

     

    I think I'l wait for the next version ;-)

    Link to comment
    Share on other sites

    Something very odd happening which should be "impossible".  I suggest you restore the flash device from backup using the Flash Creator tool.  Then try the upgrade again but boot in 'safe mode'.

    Link to comment
    Share on other sites

    Also so you're aware, the recommended way to install extra packages are via NerdPack / DevPack.  If you use /boot/extra, it is your responsibility to make sure that everything is compatible.

    Link to comment
    Share on other sites

    @Squid Thanks for pointing that out.... I wasn't even aware I still had that.. I will remove those first as I do not need them anymore and reboot within 6.8.3. Just to be sure. Hope that helps the issue since then its my fault !

     

    @limetech I remade my USB by creating it anew with the usb drive creator and then copying over the contents of my existing.. (with that boot/extra stuff removed). Maybe I need to make a fresh usb in all and only copy the config files that are needed for the basic array.. Have been using unraid for a great while so maybe there is more old sh*t in there..

     

    After that I will try again and post syslog 

     

     

    Edited by Helmonder
    Link to comment
    Share on other sites

    This is the diagnostics with the reboot after removing /extra, so still on 6.8.3.

     

    Now rebooting with 6.9.. 

     

    I'll get back in an hour with the results !

    Link to comment
    Share on other sites

    Ok this is weird... booting seems to not start at all with a "boot error" message... When I recreate my usb stick and copy 6.8.3 over it, then it works again... Tried several times...

     

    I will now recreate my usb stick (with the image tool) and copy over the 6.9 files... curious if that works.. This does appear to be something totally different then the networking that this started out with..

    Link to comment
    Share on other sites

    Ok now I am totally puzzled..

     

    I just reimaged my flashdrive with the imaging tool, I put 6.9 on it and just booted with that.. Made no changes to it..

     

    So then there is no boot error (I even tried a couple of times, eveyr time the system boots)

     

    I also get the blue screen asking for how to boot (GUI / No Gui / etc), I then started with the regular unraid, that boots, I even get network (so that card now seems to be working), but when logging on to the system I can enter with "root" only (no password is asked) and my file system is not looking the way it should...

     

    I was able to grab the syslog (syslog_sat.bad), maybe somebody can give me a pointer here ? No idea where to start now..

    syslog_sat.bad

     

    Just to be sure I also copied my copy op the upgraded usb stick (so in 6.9) over it and rebooted. 

     

    That rebooted but the network is gone again (so back where we started).

     

    Next reboot I did was in  SAFE mode (so new stick with 6.9, contents of 6.9 copied over, but booted in safe mode). This results in the same situation, no network. syslog is attached as syslog_notok.

     

     

    syslog_notok

    Edited by Helmonder
    Link to comment
    Share on other sites
    28 minutes ago, Helmonder said:

    and my file system is not looking the way it should...

    What do you mean by that ?

    Link to comment
    Share on other sites
    On 3/5/2021 at 8:50 PM, limetech said:

    Something very odd happening which should be "impossible".  I suggest you restore the flash device from backup using the Flash Creator tool.  Then try the upgrade again but boot in 'safe mode'.

     

    @limetech

     

    Did this, same result (see below for my journey here but syslog with this "safe mode" herewith also attached..)

     

    Kind of baffled..

     

    I recopied the 6.8.3 files to the stick and rebooted: all working again..

     

    There is something in my existing flash drive config that breaks this...  and Safe Mode does not solve it...

     

     

    Let me know what you want me to test, not a problem whatsoever !

     

    syslog_notok

    Edited by Helmonder
    Link to comment
    Share on other sites

    I did some extra checks, the above might be a bit garbled with info:

     

    1) I have a working, running 6.8.3, I have rebooted just to be sure the usb drive works: it does

    2) I run an upgrade to 6.9, this results in a "BOOT ERROR" , before stuff gets loaded so it looks to me like the bootloader is not recognised or something

    3) I copy back all of my 6.8.3 files over the 6.9 migrated usb stick, the stick continues to not work and give me a "BOOT ERROR" when booting, I think that shows its not the files., dockers, plugins (also: the issue also occurs with a safe boot)

     

    If I want to make it work again I have to recreate the thumbstick and copy back my 6.8.3 files, then I have a perfectly working system.

     

    This triggered me, if it is to due with the bootloader/bootpartition.. Maybe its linked to my way of booting ?  I am using legacy boot..

     

    I changed to UEFI and now I have been able to boot !

     

    However...  No network... No GUI, no shares. I was able to use console: 

     

    Route-N and IP Route Show output attached.

    route-n.JPG.ebd174cb50af53acfc830cfec817cb0d.JPG

     

    iprouteshow.JPG.510ddb8bf35b1959b35e646e2e0f6bbe.JPG

     

    Both look fine to me...

     

    I grabbed the syslog and also attached that to the post:

     

    syslog.0803211812

     

    I also started the system in safe mode. This does not make a difference, also no network. I also grabbed the syslog fro mthe safe mode:

    syslog-safe.08031211824

     

    I am no expert here but I do notice that there -is- a message "eth4: link up", but there is no message for "eth0: link up". Should that not be there ?

     

    Edited by Helmonder
    Link to comment
    Share on other sites
    47 minutes ago, Helmonder said:

    If I want to make it work again I have to recreate the thumbstick and copy back my 6.8.3 files, then I have a perfectly working system.

     

    Instead of recreating your USB device from scratch, what happens if you run the appropriate "make-bootable" script for your OS?

     

    Link to comment
    Share on other sites
    2 minutes ago, John_M said:

     

    Instead of recreating your USB device from scratch, what happens if you run the appropriate "make-bootable" script for your OS?

     

     

    I do not have an issue getting the system to boot anymore... Switching to UEFI boot solved that.

    Link to comment
    Share on other sites


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.