Delivering modern software applications requires complex assemblages of web servers, databases, CDNs, ephemeral containers, load balancers, and so much more. Tools like New Relic provide instrumentation that gives you deep visibility into your applications and their underlying infrastructure, but sometimes you need to look at the logs created by these systems.
Logs hold chronological lists of events about performance, security, activity, and more that IT admins, DevOps teams, site reliability engineers, and software developers use every day to troubleshoot and audit systems. Logs need to be parsed, normalized, and combined; log data needs to be exposed so dev and ops teams can perform live ad-hoc searches, set up log-level alerts for proactive troubleshooting, and secure specific historical reports in order to gain insight into trends in system performance, usage, and security.
Logs contain vast numbers of events that teams need to combine, sift, sort, analyze, and be alerted on. Thankfully, there are many tools designed to help modern software teams get the most out of their logs. We looked at 15 lead logging tools—including full-service solutions and single-purpose tools—to help you choose the right ones to meet your logging needs.
Full-service logging solutions
The tools in this category come with a wide range of functionality designed to cover the majority of logging use cases, including advanced searching, archiving, and alerting.
Short for Data Evolved, Devo aims to provide the operational insights that security and DevOps teams need to make real-time decisions. It gathers and centralizes all the machine-generated data from across your business and creates a unified contextual view across your systems.
Devo can be deployed in the cloud, on-premise, or in a hybrid configuration. It’s built to explore and visualize data using a sophisticated GUI without the need for coding or specialized query languages. Non-technical business users can view data in the GUI dashboard, and seasoned dev and ops pros can drill down to the raw data captured in your system logs.
Cost: Request a demo for pricing.
Built on the venerable open-source ELK stack (Elasticsearch for search, Logstash for log ingestion, and Kibana for visualization—plus Beats for tailing log files), this general-purpose log search and visualization stack forms the basis of many commercial log tools (such as Logz.io ) and is a favorite of enterprises looking to create a custom, real-time log management solution.
You can use the ELK stack to build anything related to logging, analytics, metrics, search, and visualization. You can run it on-premise or have it hosted as a service. AWS also offers a managed Elasticsearch service that you can use for log analytics.
Elastic’s active community creates plenty of additional integrations for automating self-healing environments and pushing alerts to popular DevOps tools.
This open-source log management platform searches, analyzes, and alerts across all of your log files. Graylog’s flexible processing engine parses and enriches logs from any data source. The interface includes a powerful drill-down tool for exploring logs and pinpointing issues.
Cost: Free for up to 5 GB a day (available for download on GitHub). For enterprise pricing, contact sales.
Part of Rapid7, this cloud-based log management service focuses on eliminating the complexity of managing applications and systems across distributed environments by providing both business- and technical-oriented reports.
InsightOps emphasizes making it easy for IT and DevOps teams to gain greater visibility into their applications and infrastructure. It centralizes logs in real time from across your environment, and it monitors live logs and performance metrics with real-time alerts and troubleshooting dashboards. You can interact with logs via a mechanism that allows you to ask direct questions such as, “What software has been installed across all my assets?”
Cost: The Standard tier starts at $48 for 30 GB a month. For Enterprise tier pricing, contact sales.
Promising that anyone can get started in less than two minutes, LogDNA claims to be the fastest and easiest way to start working with your logs. Its cloud-based log management platform provides a simple dashboard for infrastructure operators, developers, and DevOps practitioners.
You can use the command line or a GUI to search, save, tail, and store all your logs in real time. LogDNA also includes machine-learning algorithms designed to identify and detect issues in your environment.
For teams delivering applications via Kubernetes, LogDNA features an easy-to-install Kubernetes integration.
Cost: ree trials and pay-as-you-go tiers start at $1.50 per GB per month.
This cloud-based log management and analytics service focuses on simplifying log management for DevOps, SysOps, and other engineering teams. Loggly includes sophisticated analysis features for log exploration that enable you to start with a high-level view, and drill down to specifics and filter on actions or time ranges using full-text searches, ranges, and Boolean operators.
Cost: Loggly comes in four tiers: Lite (free), Standard (starting at $7.99 per month), Pro (starting at $199 per month), and Enterprise (starting at $349 per month).
LogRhythm focuses on turning the morass of enterprise log data into actionable business, governance, and security insights. From data collection to analysis, LogRhythm can ingest virtually any kind of data into the Elasticsearch backend and then quickly run sophisticated queries.
Built on patented TrueTime record processing technology that records the actual time of event occurrences, LogRhythm automatically corrects time zone, device-clock offsets, and collection offsets. This makes it well suited for security functions such as enforcing continuous compliance via automated LogRhythm modules and attack forensics.
Papertrail is a cloud-hosted log management service that aggregates all your logs in one place so they can be tailed and searched in real time using a browser, the command line, or an API. Papertrail also automatically detects trends and issues instant alerts, and DevOps teams will love the REST API.
Cost: Default plans start at $7 for 1 GB per month and run up to $1,945 for 1,500 GB per month.
Splunk provides a rich set of tools for collecting, storing, indexing, searching, correlating, visualizing, analyzing, and reporting on any machine-generated data to identify and resolve operations and security issues. Though often thought of as a security tool, it can also help dev and ops teams.
Splunk’s dashboards and trend charts are designed to help users visualize trends at a glance, and explore logs using drill-downs, time filters, and ad-hoc queries. It comes in both on-premise and hosted versions.
Cost: Splunk offers a free trial, a light edition ($75 per GB per month), and an enterprise edition ($150 per GB per month)
Sumo Logic is a unified platform for analyzing, correlating, and monitoring machine data in its native format for real-time insights into applications. Sumo Logic’s hosted platform tracks infrastructure host metrics (CPU utilization, memory, and disk I/O), Amazon Web Services metrics via a Amazon Cloudwatch integration, and any application using Graphite. The service automatically performs anomaly detection on all log data it ingests, and then it displays those anomalies in a unified logs and metrics dashboard that is designed to help you quickly establish and address root cause when troubleshooting.
Cost: Sumo Logic offers a free edition, a professional edition ($90 per GB per month), and an enterprise edition ($150 per GB per month).
Built by former Google infrastructure engineers, this server-monitoring tool combines log data, system metrics, website monitoring, and alerting. Everything is stored and analyzed from a central location. Scalyr is known for its fast web interface, which is designed to search through terabytes of log files and return results in less than a second.
Cost: Plans start at $35 a month (1 GB per day); contact them to learn about plans for large workloads.
WhatsUp is a modular log management solution that includes a set of apps designed to analyze, collect, report, alert, and store log data in real time. Once configured, the suite monitors log files from your environment, with significant events triggering alerts.
WhatsUp uses EventTracker for indexing and Elasticsearch for searching. WhatsUp includes pre-built queries and reports for more than 1,500 compliance and security audits, including PCI-DSS, HIPAA, and GDPR.
Integrations include: Contact sales.
Single-purpose logging tools
These specialized tools are great for collecting and managing log data, but you’ll need to integrate them with other solutions for use cases like graph-based analysis or advanced alerting.
Another open-source entry, Fluentd is a data collector for building unified logging platforms that combine multiple sources and destinations. Fluentd ingests, filters, buffers, and routes log data from sources such as app logs, system logs, and databases to other solutions that handle alerting, analysis, or archiving.
Fluentd relies on a flexible plugin architecture to extend functionality to other sources and outputs. For example, if you want to query your log data, you’ll need to integrate it with a tool like Elasticsearch. For implementations with tight memory requirements, a common constraint on edge devices, Fluent Bit is a lightweight forwarder for Fluentd.
Cost: Open source under the Apache 2.0 License.
This centralized, on-premise software manages and monitors application logs, event logs, service logs, operating systems logs, and syslog data; and it issues alerts when patterns are detected. It’s used frequently by enterprises for network security monitoring, alerting, reporting, and audits.
Nagios Log Server integrates with other solutions via API. A customizable dashboard enables you to quickly view and share important data. Infrastructure teams can use its fast visual search, query, and filtering capabilities to help find the exact cause of a particular problem.
Integrations include: Splunk and VMWare
Cost: In addition to an open source version, commercial prices range from $3,995 (for 1 instance) to $14,994 (for 10 instances).
This open-source log management solution helps infrastructure engineers and developers collect, ingest, aggregate, and process log data from a variety of sources and send it to a log analysis or alerting tool. Syslog-ng is designed to classify, rewrite, and correlate logs in real time. It relies on a plugin architecture to extend functionality, and you can write your own plugins in C, Python, Java, Lua, or Perl.
Syslog-ng supports the Advanced Message Queuing Protocol (AMQP) and the Simple Text Oriented Messaging Protocol (STOMP) for full message queue support, which makes it perfect for teams working with log data using sophisticated messaging pipelines.
Integrations include: Splunk and Elasticsearch
Logs contain critical insight into the performance of your applications, databases, and infrastructure. Every logging tool has its own strengths, weaknesses, and learning curves, so the goal is to find the one that best matches the needs of organization.
In fact, it’s likely that you’ll want more than one—think Fluentd + Elasticsearch. Or consider how to complement your logging tool with New Relic for full-stack visibility into your applications, infrastructure, and—most importantly—your customer experience.