Settings lost after reboot

[jpalmer]

I'm continuing to struggle with network configuration not persisting after a reboot.  This is a licensed fresh install of 6.1.9 (not an upgrade) onto a new USB stick.

 

I have set a static IP address and checked the box turning off DHCP.  Everything appears to function correctly and the system remains stable for weeks without issue.

 

However, once rebooted, the IP address changes to a dynamically assigned value and I lose my DNS settings.  Also I have a couple VMs that cannot start because the bridge setting are either lost or otherwise invalid.

 

If I visit the settings menus and restore all of these settings I note the various cfg files in /boot/config have current timestamps from my edits.  Functionality will be restored and stable right up to the next reboot when things go south again.

 

Has anybody experienced similar symptoms?  Any thoughts on what I should try or even what to look for in the logs for a clue?

 

Any thoughts you my have would be most welcome...  This has become quite frustrating as I cannot deploy this to production until the system is stable and re-bootable.

 

[JonathanM]

http://lime-technology.com/forum/index.php?topic=39257.0

[jpalmer]

Thanks jonathanm...

 

Yes, I read that post and in part that's why I'm asking about logs.  Since my system is fully functional and stable for literally weeks and it's a boot event that introduces the issue, wouldn't a post-boot log be the place to look?

 

I can certainly provide a pre or post log (or both) as required this afternoon when I have access to the server.

 

[RobJ]

Boot events are mostly logged to the syslog, which is included in the diagnostics.  PLEASE provide your diagnostics

 

It does sound like there's something wrong with your flash drive, but don't know what.  All of those settings are saved to the flash, and used on next boot.  The Fix Common Problems plugin might identify the issue too.

[jpalmer]

Thanks RobJ for clearing up the issue of the logs...

 

I started by installing and running the Fix Common Problems plugin.  That only returned warnings about the WebGUI not being set to auto-update, not having the Shutdown plugin installed and a share having both some include and exclude (mutually exclusive) disks defined.  I fixed those issues and installed Shutdown.  Fix Common Problems now returns no warnings.

 

I then stopped the array and performed a shutdown.  After a few moments I restarted and found I had the correct, supposedly static, IP address assigned.  However, I believe this was simply assigned by DHCP as the previously assigned IP from sixty seconds earlier still with a valid lease.  I believe this because my DNS was changed and my bridge was bad with my VM's refusing to start, complaining that "Cannot get interface MTU on 'br0' : No such device"

 

To test this I performed another shutdown and this time bounced my router, too before restarting in an attempt to clear any IP caching by my DHCP.  Suspicions confirmed - after a restart I had a totally foreign IP and DNS and the Bridge was scrambled.

 

I ran diagnostics before restoring network settings.  Diagnostic file is attached...

 

Scanning syslog at about 7 or 8 seconds into the boot process I can see DHCP configuring networking, but I can't discern why.

 

I also notice the system referring to itself as Tower even though the identification config should have set it to rainier.

 

Any thoughts you may have on this would be greatly appreciated.

 

-John

 

rainier-diagnostics-20160510-2201.zip

[itimpi]

As a check can you issue a

df

command from a console/telnet session to check that the USB stick has been mounted as /boot.    Your symptoms sound as if this might not be the case so that any changes you make are only happening in RAM.

[Squid]

Its weird on a quick glance... Obviously you can write to the flash as you managed to install the plugins, and fix common problems didn't complain about it

 

the diagnostics are clearly labelled rainier, but the syslog states Tower

 

ident.cfg clearly states rainier

 

ifconfig is returning a completely different ip address than whats set in network.cfg

 

I see no "bread" errors in the logs so the flash doesn't appear to have mounted / unmounted.

 

I'm almost thinking that its booting off of a plop disk or something and then its mounting the flash drive, but that doesn't make any sense either.

 

 

[jpalmer]

Thanks guys for all your thoughts, I really do appreciate it...

 

Normally I like a good challenge, but this one has been especially frustrating.  As you noted, it just doesn't seem to add up.

 

And it gets worse.  I believe I'm losing my root password at boot as well.  I realized this morning that my SSH session was connecting without password entry.  I was slow to spot it because I have exchanged keys between all of my VMs for ease of connecting for maintenance.  As a result password-less connection seemed "normal" - but not for this connection to the unRaid box.

 

Root password would rely on the passwd/shadow files in /boot/config, correct?  It's almost as if the files in config are ignored or are inaccessible / unusable.

 

Do you think there might be any benefit to rebuilding the flash from scratch?  I could pick up a new/different drive and start over.  Not knowing what's wrong makes it hard for me to visualize this helping, but I would try it in a heartbeat if you think it has the possibility of helping.

 

If I go this route I have two questions:

 

[*]Is it possible to reassign my existing key to a new flash GUID?

[*]I would want to be sure that I could build a new array on the new system while retaining all array content.

 

[Squid]

Won't hurt.  First thing to try would be to just download from lt website the appropriate zip file for your version and just copy the files on to the stick.

 

Sent from my LG-D852 using Tapatalk

 

[jpalmer]

OK, unless a better idea comes along from another quarter, I'll proceed with a flash rebuild this afternoon.

 

I found the page telling how to replace my key so, since I'm doing this at all, I might as well go all in and replace the flash drive, too.

[jpalmer]

Well, I have no idea what was wrong in the first place, but a fresh install onto a new flash drive seems to have solved the issue.

 

I reconfigured the system, added the drives back into the array (with the known good parity box checked) and then reestablished my VMs using their existing disk images.

 

All applications were functional, so I crossed my fingers and rebooted.  Everything came back up with the correct configuration, the array was online and all VMs were running.

 

Played around a little more getting everything back in order, then tried another reboot.  Again everything came back up as it should.

 

I don't know what was wrong, but I'm going to call this closed if not outright solved.

 

Thanks to everyone for their suggestions to work through this...

 

-John

 

 

 

[nextgenpotato]

Bump.

I am having this problem as well. I guess it doesn't happen after every reboot and since I don't reboot often it's not a big problem at the moment but it's scary and annoying when it happens.

This is what happened, or the problems I observed (configuration lost after reboot)

- ssh keys were regenerated (root password is OK). I'd have to delete known_hosts files on all my clients

- All Identification settings were set to default

- Date/time setting are gone

 

So if the only solution is to to start with a new config, is there a guide to help me through it?

I have bunch off docker apps/plugins that I would have to restore their configuration and data.

It just seems a lot of work.