The ELK (Elasticsearch, Logstash, Kibana) stack is a popular open source log analysis and management platform. The collection, processing, normalization, enhancement, and storage of log data from various sources are grouped under the term “log management.” It is a necessary component in any Security Information and Event Management (SIEM) solution, but insufficient by itself. The ELK stack can be configured to perform all these functions, but it involves a fair bit of work to set it up and an entirely separate challenge to monitor its output. And so, since open source has the best possible price, some are initially captivated by the potential of great power at zero cost.

Building Blocks for SIEM

There are many necessary components to build a SIEM solution and remember the saying: It’s not what you have but what you do with it, that counts.

Log collection: Aggregate data from multiple data sources including applications, the infrastructure (e.g., servers, databases), security controls (e.g., firewalls, VPNs), network infrastructure (e.g., routers, DNS), and external security databases (e.g. threat feeds). Using a combination of Beats and Logstash, you can build a logging architecture consisting of multiple data pipelines. Not for the faint of heart, but it can be done.

Log processing: All those data source types generate data in different formats. Logs must be normalized to search and analyze the data. This normalization process involves breaking down the various log messages into meaningful field names, mapping the field types correctly, and enriching specific fields where necessary. Without log parsing, there is no meaning and actionable insights. You can do log parsing with Logstash that supports many different filter plugins. Logstash can also break up your logs, enrich specific fields with geographic information, drop fields, and add fields, for instance. Just contribute expertise and lots of time to tweak and get correct.

Storage and retention: Index for fast search, retain for forensic and compliance purposes to allow you to process larger volumes of data over time. Think of scalability, fault resilience, handle disconnects, and data bursts. Just add expertise and time, lots of time. ELK doesn’t perform log archiving, so that responsibility falls on your shoulders. Note that compliance frameworks such as PCI DSS (Payment Card Industry Data Security Standard) and HIPAA (Health Insurance Portability and Accountability Act) often require 365 days or more of log archiving.
 
Querying: Once your data is collected, parsed, and indexed in Elasticsearch, the next step is querying the data. Log queries enable you to conduct a forensic investigation into previous security incidents. You can do this with Kibana using the Lucene syntax. You do know Lucene, right?

Dashboards: Kibana supports a wide array of different visualization types and allowing users to slice and dice their data in any way they like. You can create pie charts, graphs, geographical maps, single metrics, data tables, and more, to be useful. Of course, all this is complicated and requires intimate knowledge of your data and the different fields constructing the log messages. Since ELK has no role-based access control (RBAC), every user sees everything. You are faced with learning and using X-pack if you want RBAC.

Correlation: The connection of signals coming in from different data sources in a pattern could indicate a security breach. A correlation rule defines the specific sequence of events that forms this pattern. No such thing exists within the ELK stack. It is now up to your security analysts to use Kibana queries, based on the parsing and processing performed using Logstash, to correlate between events. Finding and retaining staff can be a challenge given today’s cybersecurity resource and skills shortage.

Alerts: The ELK Stack, in its open source form, does not ship with a built-in mechanism for alerting on suspicious activity. To enable this capability, you will need to augment the ELK Stack with an alerting plugin or add-on. Other tools such as X-pack or ElastAlert are needed along with more time to set up, configure, and maintain.

Incident Management: How well you handle the problem detection and analyst alert function determines the actual Return on Investment (ROI) of SIEM. The ELK stack is useful when it comes to helping your in-house analysts identify a cybersecurity event. Without recommendations for remediating cybersecurity gaps, the ELK stack does not have many capabilities for incident management.

So, is the ELK stack a SIEM? Nope, just as lumber, nails, and a circular saw at the home improvement store isn't a backyard deck. ELK has capability for centralized logging; but in raw form, it isn’t a SIEM. It’s a Do-It-Yourself (DIY) tool for those with the staff, skills, and patience to create a solution on their own. Netsurion can assist organizations looking for rapid time to deployment and results with comprehensive SOC-as-a-Service (SOCaaS) from an award-winning SIEM provider.

How We Help

Organizations that have started on their SIEM journey often find themselves in one of two places:

1. Under-invested in their initial implementation, assuming they have the staff and skills needed for a DIY implementation, and not getting the hoped-for value, or
2. Over-licensed with a bloated SIEM solution and stuck paying higher maintenance costs for that unused capacity over many years

We filter out the “noise” of raw logs to enhance your operational effectiveness and enable you to focus on the highest priority incidents and suspicious activities.

EventTracker co-managed SIEM provides these robust capabilities out of the box:

• 24/7 Security Operations Center (SOC) that escalates to your team, using the SANS Incident Handler Guide
• Threat hunting and rapid searching using Elasticsearch within EventTracker SIEM
• Actionable information to escalate real attacks to you after false positives are eliminated
• Machine learning that accelerates correlation rules to identify suspicious activity when every minute matters
• 400 days of log archiving is standard with EventTracker

Lumifi enables visibility and correlation with straightforward implementation. And since it’s co-managed, we augment your staff to address the IT and Security staff and expertise shortages.