Strange syslog problem since upgrading to 6.10.0


Recommended Posts

My syslog is written to a  flash drive configured as an unassigned device.  For some reason it says all space is used when it has very little actually stored on the card.  This is a Micro SD card without a lock switch.

 

image.thumb.png.7bb085a00132148d1bfbd0ffddc8394a.png

Since the upgrade to 6.10.0 there have been lots of issues with the syslog.

 

Currently, the syslog in RAM is filling up with the following errors:

May 25 04:50:01 MediaNAS rsyslogd: [origin software="rsyslogd" swVersion="8.2102.0" x-pid="26279" x-info="https://www.rsyslog.com"] rsyslogd was HUPed
May 25 04:50:01 MediaNAS rsyslogd: rsyslogd[internal_messages]: 552 messages lost due to rate-limiting (500 allowed within 5 seconds)
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: file '/mnt/disks/SYSLOG/syslog-192.168.1.10.log'[9] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2102.0 try https://www.rsyslog.com/e/2027 ]
May 25 04:50:01 MediaNAS rsyslogd: action 'action-3-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2102.0 try https://www.rsyslog.com/e/2027 ]

 

The flash drive for the syslog is definitely not out of space (it's 32GB) but rsyslogd is not writing anything to the syslog file on the flash drive.  I even reformatted the flash drive.  Immediately after this, the syslog file was written but it stopped after a few lines and has not been updated in a couple of days.

 

My appdata backup also goes to the same flash UD flash drive and there are no issues with that process. [UPDATE: apparently this was the problem as the last appdata backup went crazy and filled the disk]

 

While running unRAID 6.9.2 the syslog on the flash drive was written to without issue from Jan to May 24 when it suddenly stopped after the 6.10.0 upgrade. [It was the first appdata backup that happened shortly after upgrading that caused the problem so I though it was the upgrade that was the issue.]

 

What could be causing rsyslogd to think the drive is full when it is not and failing to write to the syslog?

 

Permissions look OK?

image.png.c55a9c934e5d920f27463ac3e09a0269.png

 

 

 

medianas-diagnostics-20220527-0852.zip

Edited by Hoopster
Link to comment

Problem resolved.  Something happened during the last appdata backup that resulted in it filling the disk (size was reported incorrectly).  Previous backups were much smaller. 

 

I have moved appdata backup to a different location with lots of space for troubleshooting and removed it from the SYSLOG location.  Syslog is functioning properly now and again writing to the syslog on the Micro SD card.  The "no space left on device" errors have ceased.  :)

Edited by Hoopster
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.