Setting up syslog to capture server crashes


jj_uk

Recommended Posts

My server keeps crashing; it was every 6 months, now its every few weeks. 

I've set up syslog server to capture server logs, but they are written to a share.  When the server crashes, the syslog shows no logging around the time of the crash, it shows the community apps backup run at 3am, then nothing until the hard reset at 1:16am.  The server crashed at about 12:45am.

 

Quote

Jan 26 03:51:47 tower1 CA Backup/Restore: #######################
Jan 26 03:51:47 tower1 CA Backup/Restore: appData Backup complete
Jan 26 03:51:47 tower1 CA Backup/Restore: #######################
Jan 26 03:51:47 tower1 CA Backup/Restore: Deleting /mnt/user/backup/CommunityApplicationsAppdataBackup/appdata/[email protected]
Jan 26 03:51:47 tower1 CA Backup/Restore: Backup / Restore Completed
Jan 27 01:16:12 tower1 kernel: mdcmd (36): set md_write_method 1
Jan 27 01:16:12 tower1 kernel: 
Jan 27 01:16:12 tower1 cache_dirs: Arguments=
Jan 27 01:16:12 tower1 cache_dirs: Max Scan Secs=10, Min Scan Secs=1
Jan 27 01:16:12 tower1 cache_dirs: Scan Type=adaptive
Jan 27 01:16:12 tower1 cache_dirs: Min Scan Depth=4
Jan 27 01:16:12 tower1 cache_dirs: Max Scan Depth=none
Jan 27 01:16:12 tower1 cache_dirs: Use Command='find -noleaf'
 

 

 

So my question is how can I capture system logs to a directory on the array rather than a share? Maybe the crash isn't capturing logs because the crash is killing the shares?

 

image.thumb.png.7f17d91f6b789ede2daccbd3a1cd8bd6.png

 

 

image.thumb.png.3ec14f4d7ce1812177e0a57a11c2f145.png

Edited by jj_uk
Link to comment
2 hours ago, jj_uk said:

So my question is how can I capture system logs to a directory on the array rather than a share?

That's why mirror to flash is an option.  Your flash drive can be removed and read in another computer even when the array won't start or is not accessible.  This should not be left on long-term due to wear and tear on the boot flash drive but it is there for temporary troubleshooting.

 

Only shares or <custom> can be selected as destinations in the syslog config GUI.  <custom> let's you edit the syslog config and point to another location.  I have mine going to an unassigned device called SYSLOG which is a removable NTFS formatted flash disk.  This is what I did:

 

Modify the syslog config files on the boot flash drive to point to the UD location:

 

Change the server_folder line in /boot/config/rsyslog.cfg to point to the UD location:

      server_folder="/mnt/disks/SYSLOG/"

In rsyslog.conf, change the $template remote variable to point to the UD location:

  $template remote,"/mnt/disks/SYSLOG//syslog-%FROMHOST-IP%.log"

 

The entire process for my use case (very specific and not generally applicable) is documented here.  It can be modified for other similar situations.

Edited by Hoopster
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.