My server was in a strange state when I got home.  I logged into the webgui and it showed no disks - data, parity, cache, boot etc.  The Main/Array Operation tab just had two button, reboot and shutdown.  I could SSH in and look around, everything seemed mounted.

 

I tried to reboot from the gui and nothing happened but a blank page.  First reboot from command line did nothing and returned a prompt.  Second one seemed to do something, wait, something else, then as I was tailing the syslog, it seemed to go through it's startup sequence by installing plugins and what not.  It finished the startup sequence and kicked into a parity check.

 

From the webgui everything seems normal now.  I'll let the parity run.  Attached are the diagnostics and a manual copy of the syslog when it was in this weird place. Any ideas?

 

tdm-diagnostics-20190306-2110.zip

syslog

Gabbing that syslog before rebooting was a good thing to do. It seems to go wrong around here:

Mar  6 05:06:03 tdm CA Backup/Restore: tar verify returned errors.  Not deleting old backup sets of appdata
Mar  6 05:06:03 tdm CA Backup/Restore: tar verify errors occurred
Mar  6 05:06:03 tdm CA Backup/Restore: Backup / Restore Completed
Mar  6 05:07:01 tdm crond[1804]: exit status 2 from user root /usr/local/emhttp/plugins/dynamix.system.stats/scripts/sa1 1 1 &> /dev/null
Mar  6 05:08:01 tdm crond[1804]: exit status 2 from user root /usr/local/emhttp/plugins/dynamix.system.stats/scripts/sa1 1 1 &> /dev/null

CA Backup/Restore has just finished a job but it hasn't managed to build the tar file properly.

And this appears later:

Mar  6 20:55:36 tdm nginx: 2019/03/06 20:55:36 [crit] 9271#9271: *449374 pwritev() "/var/lib/nginx/fastcgi/1/00/0000000001" failed (28: No space left on device) while reading upstream, client: 192.168.1.108, server: , request: "GET /Tools/Syslog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "tdm", referrer: "http://tdm/Tools"

It's the "No space left on device" that I'm concerned about. It looks as though you've run out of storage space somewhere. I'm thinking maybe the root file system (which is in RAM) has become full. You have CA Backup/Restore configured to save the backups to an unassigned disk. If that disk is not mounted or becomes disconnected then writes to it will go to the root file system and start to fill it up. So that's where I'd look. Is the 6th March backup missing from the backup disk, for example?

Thanks John.  You're right.  I had some issues with a drive so I moved in my hot spare that I normally use as an unassigned drive for backups.  CA Backup/Restore ran, couldn't find the path and I guess as you say backed up to memory.  I didn't realise that was the behaviour of CA B/R, something to be aware off in the future.

8 hours ago, dalben said:

I didn't realise that was the behaviour of CA B/R, something to be aware off in the future.

Not so much the behaviour of the plugin - all it does is backup the files where you tell it to. It's simply a consequence of the destination disk not being mounted.