Cannot login anymore, all state ini file gone!


Recommended Posts

Hello everyone,

 

When I get to the unraid web interface, I have this message on top:

 

Quote

Warning: parse_ini_file(state/var.ini): failed to open stream: No such file or directory in /usr/local/emhttp/login.php on line 36

 

If I try to login, I get to a white page with these:

Quote


Warning: parse_ini_file(state/var.ini): failed to open stream: No such file or directory in /usr/local/emhttp/login.php on line 36

Warning: session_start(): Cannot start session when headers already sent in /usr/local/emhttp/login.php on line 92

Warning: session_regenerate_id(): Cannot regenerate session id - session is not active in /usr/local/emhttp/login.php on line 95

Warning: Cannot modify header information - headers already sent by (output started at /usr/local/emhttp/login.php:36) in /usr/local/emhttp/login.php on line 98

 

On my syslog, I see that my backup occured at 3am like supposed then after, it's saying core overheat and other stuff

 

Quote

Jun 12 03:46:10 ServRaid CA Backup/Restore: Backup / Restore Completed
Jun 12 03:46:27 ServRaid root: error: /webGui/include/DashUpdate.php: uninitialized csrf_token
Jun 12 03:46:58 ServRaid kernel: mce: CPU26: Core temperature is above threshold, cpu clock is throttled (total events = 2)
Jun 12 03:46:59 ServRaid kernel: mce: CPU2: Core temperature is above threshold, cpu clock is throttled (total events = 24)
Jun 12 03:47:05 ServRaid kernel: mce: CPU12: Core temperature is above threshold, cpu clock is throttled (total events = 2)
Jun 12 03:47:06 ServRaid kernel: mce: CPU23: Core temperature is above threshold, cpu clock is throttled (total events = 2)
Jun 12 03:47:16 ServRaid kernel: mce: CPU9: Core temperature is above threshold, cpu clock is throttled (total events = 2)
Jun 12 03:47:27 ServRaid root: error: /webGui/include/DashUpdate.php: uninitialized csrf_token
Jun 12 03:48:27 ServRaid root: error: /webGui/include/DashUpdate.php: uninitialized csrf_token

 

The csrf_token error and temperature throttled are repeated non stop, and that's all I have in that log file

 

Thanks

Edited by Nodiaque
Link to comment

I think you might have been hit by a possibly issue with CA Backup plugin. 

 

maybe @Squid has an idea about what you can try and do to help them troubleshoot the issue? I say this because from your snippet it looked like errors happened after CA Backup Plugin ran.

 

At the very least post your sylslog to your NEXT post. Ideally for troubleshooting what is desired is your Diagnostics zip file to your next post by typing in "diagnosics" at the terminal widow. But I think you might have issues with that as the command might not be found, which is why I mentioned at least the syslog file. If you can't get the file now, try to copy it to a drive in cache or array before rebooting.

 

Also before rebooting try looking around at the fiesystems of /usr/ or other standard locations to see if files are still there or seem to be missing from the system in memory.

 

Edit: I see you were able to post diagnostics. But still have a look around the various standard locations of the OS and see what's there and what isnt.

 

 

Edited by BRiT
Link to comment

Ah, I think you replied while I was posting my diagnostics files, it's right above your post.

 

I manage to login by doing a touch var.ini and touch network.ini. This lead me to a weird GUI with mostly no information (like my docker are running but no information on them, array seems offline but it's not). At least, that let me run some stuff.

 

I seem to hit some problem with CA Backup, sometime it'S the nvidia card that are crashed with the plugin (I stop using my card because of that, never found what was the problem).

 

I found out something was hogging my cpu, thus why the heat issue (had 1 core out of 24 always at 100%). Just closed some docker to resolve that heat issue, but the csrf issue remain.

 

Here's the syslog file

 

I've saw on other thread people saying it was ram full. I've checked with grafana, I only used about 32gb of ram, I have 128gb.

syslog-192.168.0.4.log

Edited by Nodiaque
  • Thanks 1
Link to comment

The Last time I ran CA Backup on a 6.10.x system I had system errors, it looked like certain in-memory directories were entirely wiped out. It couldn't find the "newperms" script which is built in and almost nothing in the UI worked as it couldn't find any of the files. When I looked at UI Plugins it could not determine the version of anything. I think that was because somehow entire directories or mountpoints somehow got wiped out or reset.

 

I think you might be in the same state, so see what if anything exists in /usr/ and /usr/local/emhttp/.

Link to comment

seems to have some stuff:

 

Quote

/usr# ls -l
total 0
lrwxrwxrwx  1 root root     8 Apr  7  2021 adm -> /var/adm/
drwxr-xr-x  2 root root 18860 May 11 16:11 bin/
drwxr-xr-x  6 root root   120 Apr 26  2020 doc/
drwxr-xr-x  3 root root    80 May 11 16:11 include/
drwxr-xr-x  6 root root   260 May 27 07:19 lib/
drwxr-xr-x 40 root root 19780 May 27 07:19 lib64/
drwxr-xr-x  9 root root   500 Aug 21  2020 libexec/
drwxr-xr-x 10 root root   200 May 30 17:25 local/
drwxr-xr-x  6 root root   120 Apr 26  2020 man/
drwxr-xr-x  2 root root  5460 Apr 26  2020 sbin/
drwxr-xr-x 44 root root   880 May 11 16:11 share/
lrwxrwxrwx  1 root root    10 Apr  7  2021 spool -> /var/spool/
drwxr-xr-x  3 root root    60 Apr  7  2021 src/
drwxr-xr-x  3 root root    60 Sep 30  2020 systemtap/
lrwxrwxrwx  1 root root     8 Apr  7  2021 tmp -> /var/tmp/

 

Quote

/usr/local/emhttp# ls -la
total 44
drwxr-xr-x  5 root root   320 Jun 12 00:10 ./
drwxr-xr-x 10 root root   200 May 30 17:25 ../
-rw-rw-rw-  1 root root     0 Jun 12 09:01 0
-rw-r--r--  1 root root  1611 Jan 17  2020 auth_request.php
lrwxrwxrwx  1 root root     5 Jun 19  2016 boot -> /boot/
drwxr-xr-x  3 root root    60 May 12  2020 languages/
lrwxrwxrwx  1 root root     8 Jun 19  2016 log -> /var/log/
-rw-r--r--  1 root root  7484 Jul 15  2020 logging.htm
-rw-r--r--  1 root root 15474 Mar 25  2021 login.php
lrwxrwxrwx  1 root root     4 Jun 19  2016 mnt -> /mnt/
drwxr-xr-x 37 root root   740 May 30 17:25 plugins/
-rw-r--r--  1 root root    26 Sep  6  2018 robots.txt
drwxr-xr-x  3 root root   100 Jun 12 09:30 state/
-rw-r--r--  1 root root  3934 May 28  2020 update.htm
-rw-r--r--  1 root root  4343 Mar 13  2018 update.php
lrwxrwxrwx  1 root root    15 Jun 19  2016 webGui -> plugins/dynamix/

 

 

Link to comment

It's very bizarre that those ini files would not be there as they are part of every running system, and what you described is very close to what I and another experienced after CA Backup ran.

 

Have a deeper look inside there, like can you look at a recursive listing [ find /usr/local/emhttp -print ] to get a manifest of what is or isn't there?

 

It will prob be pretty big so might need to redirect output to a file.

Link to comment

There's also log files from the CA Backup process located in /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log or /tmp/ca.backup2/tempFiles/  that you should look at to see if you can find anything in there.

 

I see a line in the syslog that was at the end run of CA Backup at 03:46:10, not sure if that's normal to have it echoed in the syslog.

Link to comment

Well I can't think of other commands to run to troubleshoot things as to the cause or what really happened. I was hoping someone else would jump in with ideas to run.

 

If you want to resolve the situation now, you can reboot the system. However, that will prevent additional troubleshooting steps, but I think to trigger it you might only have to run CA Backup.

 

If you do reboot now, comparing the earlier posted emHttp.txt file to one generated after the reboot and you verify the system is functional again might be interesting to see what exactly disappeared.

Link to comment

ok, so rebooting will recreate all those file. That's a relief, I was affraid to be stuck with a dead configuration.

 

I'll do some tweaking on the ca backup, like not stopping container anymore and not using verify, just to see if things get better.

 

Thanks for your help

Link to comment

No meaningful difference between the two lists.

 

If you have time and are in a troubleshooting mood, you could see if anything happens after running CA Backup before it's next scheduled time. If you see similar or odd behavior, at an open shell, type "sync" before you try troubleshooting again and then see what's what.

Link to comment
  • Nodiaque changed the title to Cannot login anymore, all state ini file gone!
  • 2 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.