I got syslog-ng working, but the issue that I'm running into is that unraid syslogs seem to report only as "unraid", so multiple server syslogs get combined together.  I have a fairly good tailscale setup with local dns and subnet routing, so getting the logs isnt hard, but collecting them all and displaying them in any useable fashion has proved difficult.

 

What are you guys using to display remote syslogs?  I also would like to display logs for my pfsense boxes as well.

 

I'm sure this gets asked alot, but syslog is a pretty common term and pulls up a ton of results.  Sorry if this is already covered.

I asume pfsense--syslog-ng

It sounds like you're trying to centralize and better organize your logs from multiple servers (including Unraid and pfSense) and display them in a meaningful way. Here are some strategies and tools that can help solve your problem:

 

Organizing Syslogs with Host-Specific Tags

The issue you're encountering with all logs being tagged as "unraid" stems from the way logs are configured to be sent to your syslog server. To resolve this:

Modify the syslog-ng configuration (syslog-ng.conf) on your Unraid servers to include host-specific tags. For example:

 

source s_network {
    syslog(ip(0.0.0.0) port(514));
};

destination d_file {
    file("/var/log/$HOST/$YEAR$MONTH$DAY-$HOST.log");
};

log {
    source(s_network);
    destination(d_file);
};

 

This creates separate log files for each host based on their hostname.

Ensure each Unraid server has a unique hostname for proper identification.

 

2. Using a Centralized Log Management Tool

A dedicated log management tool can make it much easier to aggregate, parse, and visualize logs from multiple sources. Popular choices include:

a. Graylog

Graylog is a powerful open-source log management platform.

It supports inputs from syslog-ng, and you can set up streams and dashboards for each device or service.

You can install it as a Docker container or on a dedicated server.

To configure:

Set up a Graylog server.

Point your syslog-ng instances (on Unraid and pfSense) to the Graylog server.

Use filters and streams to segregate logs by hostname or device type.

b. ELK Stack (Elasticsearch, Logstash, Kibana)

The ELK Stack is another popular choice for log aggregation and visualization.

Logstash can parse logs from syslog-ng, and Kibana provides a user-friendly UI for searching and visualizing logs.

Configuration is more complex than Graylog but offers excellent scalability and customization.

c. Grafana with Loki

Grafana Loki is designed for lightweight log aggregation and works well with syslog-ng.

Combine with Grafana dashboards to visualize logs.

Loki is easier to set up compared to ELK and is a great option if you already use Grafana.

 

3. Setting Up pfSense Logging

To include pfSense logs in your centralized setup:

In pfSense, go to Status > System Logs > Settings.

Configure Remote Logging Options:

Enable "Send log messages to remote syslog server."

Point it to your syslog-ng or centralized log server's IP and port (e.g., 192.168.1.100:514).

Organize pfSense logs by hostname or create a separate destination in your syslog-ng configuration for these logs.

 

4. Displaying Logs

Once centralized, you can use any of the tools above to create dashboards, search, and visualize logs. Here's a breakdown of tools:

Graylog or Kibana (from ELK): Excellent for creating dashboards to view system metrics, identify errors, and correlate events.

Grafana + Loki: Ideal if you’re already using Grafana for monitoring.

LogViewPlus: A Windows application for viewing and analyzing log files locally if you prefer a simpler GUI tool.

 

5. Troubleshooting Tips

Ensure Unique Hostnames: Without unique hostnames or tags, logs from different servers will merge.

Check Time Synchronization: Sync all devices with an NTP server to ensure logs are timestamped consistently.

Enable Filtering: Use syslog-ng's filtering capabilities to segregate logs by hostname, service, or message content.

Use Subdirectories for Logs: If you're storing logs on disk, use directory structures (/logs/unraid/server1, /logs/pfsense/router1) for organization.

 

By implementing one of the above tools and properly tagging/logging your servers, you'll have a centralized and organized system for viewing and managing logs from Unraid, pfSense, and other devices.

Here's a clearer and more structured example of a syslog-ng.conf configuration file that centralizes logs and organizes them by hostname, making it easier to manage logs from multiple devices like Unraid servers and pfSense boxes.

 

Example syslog-ng.conf for a Centralized Log Server

 

@version: 3.35
@include "scl.conf"

# Define a network source to receive logs from remote servers
source s_network {
    syslog(ip(0.0.0.0) port(514)); # Listen on all interfaces, UDP port 514
};

# Define a file destination that organizes logs by hostname and date
destination d_host_logs {
    file("/var/log/syslog-ng/$HOST/$YEAR-$MONTH-$DAY.log" 
        create-dirs(yes) # Automatically create directories for hostnames
        perm(0644) # Set permissions for log files
    );
};

# Log messages from the network source to the destination
log {
    source(s_network);
    destination(d_host_logs);
};

# Optional: Filter for specific devices or services (e.g., pfSense logs)
filter f_pfsense {
    host("pfsense-router"); # Replace with the actual hostname of your pfSense box
};

destination d_pfsense {
    file("/var/log/syslog-ng/pfsense/$YEAR-$MONTH-$DAY.log" 
        create-dirs(yes)
        perm(0644)
    );
};

log {
    source(s_network);
    filter(f_pfsense);
    destination(d_pfsense);
};

# Optional: Filter for Unraid servers
filter f_unraid {
    host("unraid-server1"); # Replace with the actual hostname of your Unraid server
};

destination d_unraid {
    file("/var/log/syslog-ng/unraid/$YEAR-$MONTH-$DAY.log"
        create-dirs(yes)
        perm(0644)
    );
};

log {
    source(s_network);
    filter(f_unraid);
    destination(d_unraid);
};

 

Explanation of Key Sections

Source (s_network):

Listens on all interfaces (0.0.0.0) on UDP port 514.

You can change the port or specify a particular interface IP if needed.

Destination (d_host_logs):

Logs are saved under /var/log/syslog-ng/ organized by the hostname of the device sending the logs.

The create-dirs(yes) option ensures directories are created automatically for each hostname.

Log Path:

Routes logs from the network source (s_network) to the organized destination (d_host_logs).

Filters (f_pfsense, f_unraid):

Filters logs from specific devices based on their hostname.

Example: Logs from a device named pfsense-router are saved in a separate directory (/var/log/syslog-ng/pfsense/).

Custom Destinations (d_pfsense, d_unraid):

Allows you to store logs for specific devices or services in their own directories.

 

Steps to Deploy

Place the configuration file in the appropriate location:
 

/etc/syslog-ng/syslog-ng.conf

Restart the syslog-ng service to apply changes:

sudo systemctl restart syslog-ng

Ensure devices sending logs are configured correctly:

For Unraid: Set the remote syslog server to the IP of the centralized log server (port 514).

For pfSense: Configure Status > System Logs > Settings > Remote Logging Options to point to the same IP and port.

Verify logs:

Check the /var/log/syslog-ng/ directory to see if logs are being properly organized.

 

Testing the Setup

Send a test log message using logger

logger -n <log_server_ip> -P 514 "Test message from <hostname>"

 

Confirm that the log appears in the appropriate directory under /var/log/syslog-ng/.

 

 

Holy crap!  Thanks for the detailed response.  I'll get to work on making these changes and let you know how it goes!