rysnc logging to own log?

[TheWombat]

I've successfully set up rsync to backup my Synology NAS to my UnRaid server (5.0RC5) on a nightly basis using information in this thread.

 

However rsync on my UnRaid server is outputting to my system log (/var/log/syslog) one line for each file being backed up - which means my syslog is spammed with thousands of lines.

 

What I'd really like would be:

• rsync outputs to its own log in /var/log/rsync/[date]-log
  

i.e. there is a new rsync logfile written each night the backup occurs.

 

Is this possible, and how would I make this occur?

 

thanks

 

Alex

 

[WeeboTech]

Depends where you are running the rsync.

 

 

on the client you can use the --log-file

http://linux.die.net/man/1/rsync

 

 

On the server side you can set a statically named logfile in the rsyncd.conf file.

Then use logrotate to rename it.

[ftp=ftp://samba.org/pub/rsync/rsyncd.conf.html]ftp://samba.org/pub/rsync/rsyncd.conf.html[/ftp]

 

 

[TheWombat]

 

thanks - is on the server.

 

Will try out the suggestion in the rsyncd link you provided this evening.

 

Alex

 

[TheWombat]

I'm a linux novice so while I've made progress on setting this up it is not quite 100% working. Any chance you could take a look below and provide some guidance.

 

I created a directory "custom/etc" on my flash drive

I created a directory "custom/etc/rc.d" on my flash drive

 

I placed rsyncd.conf in "custom/etc/"

I placed S20-init.rsyncd in "custom/etc/rc.d"

I placed rsyncdlog in "custom/etc/rc.d"

 

rsyncd.conf

uid             = root
gid             = root
use chroot      = no
max connections = 4
pid file        = /var/run/rsyncd.pid
timeout         = 600
log file		= /var/log/rsyncd.log

[dailybackups]
   path = /mnt/user/Backup/Daily
   comment = Backups
   read only = FALSE

[weeklybackups]
   path = /mnt/user/Backup/Weekly
   comment = Backups
   read only = FALSE

[testbackups]
   path = /mnt/user/Backup/Test
   comment = Backups
   read only = FALSE

 

S20-init.rsyncd

#!/bin/bash

if ! grep ^rsync /etc/inetd.conf > /dev/null ; then 
cat <<-EOF >> /etc/inetd.conf
rsync   stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/bin/rsync --daemon 
EOF
read PID < /var/run/inetd.pid
kill -1 ${PID}
fi

cp /boot/custom/etc/rsyncd.conf /etc/rsyncd.conf
cp /boot/custom/etc/rsyncdlog /etc/logrotate.d/rsyncdlog

 

rsyncdlog

# Rotate /var/log/rsyncd.log

/var/log/rsyncd.log {
missingok
daily
maxage 31
dateext

}

 

The idea being that every night when the backup occurs the current rysncd.log file will be renamed with a 'date' extension and a new rsyncd.log file will be created.

 

I'm not sure how to test this other than waiting for each day's backup. However I am fairly certain I've probably made an error somewhere or broken a best practice rule.

 

As a first question - does logrotate.conf automatically look in /etc/logrotate.d for custom configuration files or do I need to tell it to look at my "/etc/logrotate.d/rsyncdlog" that was copied by S20-init.syncd?

 

Alex

 

[WeeboTech]

Looks OK to me.

 

 

As a first question - does logrotate.conf automatically look in /etc/logrotate.d for custom configuration files or do I need to tell it to look at my "/etc/logrotate.d/rsyncdlog" that was copied by S20-init.syncd?

 

 

Logrotate knows to process the directory /etc/logrotate.d

Remember that you have to modify your go script to run the custom startup script.

 

 

What I would change is put the rsyncdlog configuration file in

/boot/custom/etc/logrotate.d/rsyncdlog 

 

 

 

then cp /boot/custom/etc/logrotate.d/rsyncdlog /etc/logrotate.d/rsyncdlog

 

 

 

 

[TheWombat]

Looks OK to me.

 

 

As a first question - does logrotate.conf automatically look in /etc/logrotate.d for custom configuration files or do I need to tell it to look at my "/etc/logrotate.d/rsyncdlog" that was copied by S20-init.syncd?

 

 

Logrotate knows to process the directory /etc/logrotate.d

Remember that you have to modify your go script to run the custom startup script.

 

 

What I would change is put the rsyncdlog configuration file in

/boot/custom/etc/logrotate.d/rsyncdlog 

 

 

 

then cp /boot/custom/etc/logrotate.d/rsyncdlog /etc/logrotate.d/rsyncdlog

 

Thanks - I have the following line in my "go"

/boot/custom/etc/rc.d/S20-init.rsyncd

 

I'm still getting to grips with unRaid. If I reboot the server do I lose all my rotated log files as they would be in /var/log ?

 

Alex

 

[WeeboTech]

I'm still getting to grips with unRaid. If I reboot the server do I lose all my rotated log files as they would be in /var/log ?

 

 

 

Yes, you will have to copy or rsync them somewhere on your array.

You can run commands inside the logrotate.d/rsyncdlog to do this.

 

 

[TheWombat]

Another question.. Looking at logrotate suggests I could use either prerotate or postrotate commands. I'm guessing it could look like this:

 

prerotate

# Rotate /var/log/rsyncd.log

/var/log/rsyncd.log {
missingok
daily
maxage 31
dateext

    prerotate
         mv /var/log/rsyncd.log /boot/logs/rsyncd.log
    endscript

}

 

In the above, is logrotate still going to be able to rotate the logfile to be logfile-date/time considering that I've moved it to /boot/logs/rsyncd.log ?

 

postrotate

# Rotate /var/log/rsyncd.log

/var/log/rsyncd.log {
missingok
daily
maxage 31
dateext

    postrotate
         mv /var/log/rsyncd.log /boot/logs/rsyncd.log
    endscript

}

 

In the postrotate example above, would logrotate know that the file I want to move is the one it is rotating to have the date/time extension?

 

I'm guessing neither would work (although the concept of using prerotate or postrotate is correct)

 

Googling suggests that the filename is returned as $1, does this mean the following would work?

 

# Rotate /var/log/rsyncd.log

/var/log/rsyncd.log {
missingok
daily
maxage 31
dateext

    postrotate
         mv /var/log/$1 /boot/logs/$1
    endscript

}

 

 

Alex

 

[WeeboTech]

 

 

 

I would probably use rsync to copy the rotated rsync logs.

 

 

first do mkdir -p /boot/logs/rsyncd

 

 

# Rotate /var/log/rsyncd.log


/var/log/rsyncd.log {
   missingok
   daily
   maxage 31
   dateext


    postrotate
         rsync /var/log/rsyncd*log* /boot/logs/rsyncd/
    endscript


}

 

 

If you wanted to move them you could use --remove-sent-files

 

 

However, I probably would not use my boot flash.

 

 

I would probably use /mnt/user/Backup/logs

 

 

There is also the issue of aging the log files.  How long do you want to keep them?

You might need to put a

find /mnt/user/Backup/logs -mtime +30 -exec /bin/gzip -9 {}\;

find /mnt/user/Backup/logs -mtime +90 -exec /bin/rm -f {}\;

 

 

 

[TheWombat]

Other than the postrotate that I've not tried yet everything seems to be working fine. thanks!

 

Alex