System Logging Protocol (or syslog) is a standard protocol used to send system log or event messages to a specific server, called a syslog server. It’s primarily used to collect various device logs from several machines in a central location for monitoring and review. Engineering teams have been using the syslog protocol for decades to transport messages. Due to its longevity and popularity, most major operating systems, including macOS, Linux, and Unix, support the syslog protocol.

Syslog-ng is a free and open source implementation of the syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options, and adds important features to syslog, such as using TCP for transport. Syslog-ng is a very popular tool with many contributors. It’s widely used because of its flexibility and its simplicity to set up and configure.

Read on to learn more about how to use syslog-ng with New Relic.

Setup and configuration

Syslog-ng can be straightforward to set up and is a reliable solution to get your syslog data into New Relic. In fact, I had it installed and configured in five minutes. A few seconds later, I was receiving all my system logs.

Depending on your Linux distribution and the version of the syslog-ng package, you may or may not need to install the syslog HTTP package. This is because HTTP destination is usually not part of the core syslog-ng package. On CentOS, it is in a sub-package called syslog-ng-http. The name may vary in other distributions. I used the latest stable version of syslog-ng in a Ubuntu 16.04 distro. I didn’t need to install the HTTP module since it was already there.

Once syslog-ng is installed, perform the following steps:

  1. Open the config file:
    sudo vim /etc/syslog-ng/syslog-ng.conf
  2. Make sure the source is properly defined:
    source s_src{
    system();
    internal();
    };

    You can monitor files as well by adding these lines for example:

    source s_files{
    file("path/to/your/file1.log",flags(no-parse),follow_freq(1),program_override("<program_name_file1>"));
    file("path/to/your/file2.log",flags(no-parse),follow_freq(1),program_override("<program_name_file2>"));
    };
  3. Define the destination (change the URL if you are in the EU region):
    destination d_newrelic {
    http(url("https://log-api.newrelic.com/log/v1")
    method("POST")
    headers("Content-Type: application/json", "X-Insert-Key: <Your Key>")
    body("$(format-json --scope all-nv-pairs)")
    );
    };

    Note: Make sure to use your Insights Insert key, not your API key or License key. Learn how to generate an Insert key, if you don’t have one already.

  4. Define the output:
    log {
    source(s_src);
    destination(d_newrelic);
    };
  5. Restart syslog-ng:
    sudo systemctl restart syslog-ng

Once you are done with the config, check New Relic One and you should find your logs:

screenshot of New Relic logs

Use New Relic log analytics to explore your syslog logs.

 

log details screenshot

Check log details by clicking on any log entry.

You can also reduce the number of attributes, rename them, or add tags by changing the body attribute in the HTTP destination in the config file. Here’s an example showing how we send only specific attributes from the list of all available attributes and adding a team tag to each entry:

destination d_newrelic{
http(url("https://log-api.newrelic.com/log/v1")
method("POST")
headers("Content-Type: application/json", "X-Insert-Key: <Your Key>")
body("$(format-json date=$ISODATE priority=$LEVEL host=$HOST program=$PROGRAM pid=$PID message=$MSG messageId=$MSGID team=expert-services)")
); };

The example below provides easier to read logs in New Relic One:

screenshot of easy to read logs in New Relic One

I hope this can be helpful if you’re using syslog-ng or are looking for ways to send syslog data to New Relic.

And if you haven’t already, sign up for our free tier today—you’ll get 100 GB free every month.

Amine Benzaied is a Senior Solution Architect in Expert Services at New Relic working on key accounts. He has over 10 years of experience in software architecture, development, and management, with strong knowledge of SDLC and leading practices across multiple verticals. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!