How To Install Intrusion Detection System

September 9, 2022: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

To help you lot secure your AWS resources, we recommend that you prefer a layered arroyo that includes the use of preventative and detective controls. For example, incorporating host-based controls for your Amazon EC2 instances can restrict access and provide appropriate levels of visibility into system behaviors and access patterns. These controls ofttimes include a host-based intrusion detection system (HIDS) that monitors and analyzes network traffic, log files, and file access on a host. A HIDS typically integrates with alerting and automated remediation solutions to find and address attacks, unauthorized or suspicious activities, and general errors in your environs.

In this web log mail, I show how you tin can use Amazon CloudWatch Logs to collect and amass alerts from an open-source security (OSSEC) HIDS. I use a CloudWatch Logs subscription to deliver the alerts to Amazon Elasticsearch Service (Amazon ES) for analysis and visualization with Kibana – a popular open-source visualization tool. To make it easier for you to run into this solution in action, I provide a CloudFormation template to handle nearly of the deployment piece of work. Y'all can use this solution to gain improved visibility and insights across your EC2 fleet and help drive security remediation activities. For example, if specific hosts are scanning your EC2 instances and triggering OSSEC alerts, you can implement a VPC network access control list (ACL) or AWS WAF rule to block those source IP addresses or CIDR blocks.

Solution overview

The following diagram depicts a high-level overview of this mail service's solution.

Diagram showing a high-level overview of this post's solution

Hither is how the solution works:

On the target EC2 instances, the OSSEC HIDS generates alerts that the CloudWatch Logs agent captures. The HIDS performs log assay, integrity checking, Windows registry monitoring, rootkit detection, real-time alerting, and active response. For more information, come across Getting started with OSSEC.
The CloudWatch Logs group receives the alerts every bit events.
A CloudWatch Logs subscription is applied to the target log group to forward the events through AWS Lambda to Amazon ES.
Amazon ES loads the logged alert data.
Kibana visualizes the alerts in near-real fourth dimension. Amazon ES provides a default installation of Kibana with every Amazon ES domain.

Deployment considerations

For the purposes of this post, the primary OSSEC HIDS deployment consists of a Linux-based installation for which the alerts are generated locally within each organisation. Note that this solution depends on Amazon ES and Lambda in the target region for deployment. You can notice the latest information about AWS service availability in the Region tabular array. You lot likewise must identify an Amazon Virtual Private Cloud (VPC) subnet that has Internet access and DNS resolution for your EC2 instances to provision the required components properly.

To simplify the deployment procedure, I created a examination environment AWS CloudFormation template. Y'all can use this template to provision a test environment stack automatically into an existing Amazon VPC subnet. You will employ CloudFormation to provision the core components of this solution and then configure Kibana for alert analysis. The source code for this solution is available on GitHub.

This postal service'south template performs the following high-level steps in the region you choose:

Creates ii EC2 instances running Amazon Linux with an AWS Identity and Admission Management (IAM) office for CloudWatch Logs access. Note: To provide sample HIDS alarm information, the 2 EC2 instances are configured automatically to generate fake HIDS alerts locally.
Installs and configures OSSEC, the CloudWatch Logs agent, and additional packages used for the test environs.
Creates the target HIDS Amazon ES domain.
Creates the target HIDS CloudWatch Logs grouping.
Creates the Lambda function and CloudWatch Logs subscription to send HIDS alerts to Amazon ES.

After the CloudFormation stack has been deployed, y'all can access the Kibana instance on the Amazon ES domain to consummate the last steps of the setup for the exam environs, which I show later in the postal service.

Although out of scope for this weblog postal service, when deploying OSSEC into your existing EC2 environment, yous should determine the desired configuration, including target log files for monitoring, directories for integrity checking, and active response. This typically also requires time for testing and tuning of the system to optimize it for your environment. The OSSEC documentation is a good place to starting time to familiarize yourself with this process. You could take another arroyo to OSSEC deployment, which involves an amanuensis installation and a separate OSSEC manager to procedure events centrally before exporting them to CloudWatch Logs. This deployment requires an additional server component and network advice betwixt the agent and the manager. Note that although Windows Server is supported by OSSEC, it requires an agent-based installation and therefore requires an OSSEC manager to be present. Review OSSEC Architecture for additional data well-nigh OSSEC architecture and deployment options.

Deploy the solution

This solution's high-level steps are:

Launch the CloudFormation stack.
Configure a Kibana index pattern and begin exploring alerts.
Configure a Kibana HIDS dashboard and visualize alerts.

one. Launch the CloudFormation stack

You will launch your test environment past using a CloudFormation template that automates the provisioning process. For the following input parameters, you must place a target VPC and subnet (which requires Internet access) for deployment. If the target subnet uses an Net gateway, prepare the AssignPublicIP parameter to true. If the target subnet uses a NAT gateway, you can leave the default setting of AssignPublicIP as imitation.

First, you volition need to phase the Lambda function deployment package in an S3 bucket located in the region into which you are deploying. To do this, download the zipped deployment parcel and upload it to your in-region bucket. For boosted information near uploading objects to S3, encounter Uploading Object into Amazon S3.

You likewise must provide a trusted source IP address or CIDR block for access to the surround following the creation of the stack and an EC2 central pair to associate with the instances. For information about creating an EC2 central pair, see Creating a Key Pair Using Amazon EC2. Notation that the trusted IP address or CIDR block too is used to create the Amazon ES admission policy automatically for Kibana access. We recommend that you use a specific IP address or CIDR range rather than using 0.0.0.0/0, which would allow all IPv4 addresses to access your instances. For more information about authorizing inbound traffic to your instances, see Authorizing Inbound Traffic for Your Linux Instances.

After you have confirmed the input parameters (encounter the following screenshot and tabular array for more details), create the CloudFormation stack.

Numbered screenshot showing input parameters

Input parameter	Input parameter description
1. HIDSInstanceSize	EC2 example size for test server
2. ESInstanceSize	Amazon ES instance size
3. MyKeyPair	A public/private primal pair that allows yous to connect securely to your instance after it launches
4. MyS3Bucket	In-region S3 bucket with the zipped deployment package
5. MyS3Key	In-region S3 key for the zipped deployment package
6. VPCId	An Amazon VPC into which to deploy the solution
7. SubnetId	A SubnetId with outbound connectivity inside the VPC you selected (requires Internet admission)
8. AssignPublicIP	Gear up to true if your subnet is configured to connect through an Internet gateway; set up to imitation if your subnet is configured to connect through a NAT gateway
ix. MyTrustedNetwork	Your trusted source IP or CIDR cake that is used to whitelist access to the EC2 instances and the Amazon ES endpoint

To finish creating the CloudFormation stack:

Enter the input parameters and choose Next.
On the Options page, accept the defaults and choose Next.
On the Review folio, ostend the details, select the I acknowledge that AWS CloudFormation might create IAM resources check box, so cull Create. (The stack volition be created in approximately x minutes.)

After the stack has been created, note the HIDSESKibanaURL on the CloudFormation Outputs tab. Then, proceed to the Kibana configuration instructions in the next section.

ii. Configure a Kibana index pattern and begin exploring alerts

In this section, you perform the initial setup of Kibana. To access Kibana, find the HIDSESKibanaURL in the CloudFormation stack outputs (see the previous section) and choose it. This will bring you to the Kibana instance, which is automatically provisioned to your Amazon ES example. The source IP you provided in the CloudFormation input parameters is used to automatically populate the Amazon ES access policy. If yous receive an fault like to the following error, you must confirm that your Amazon ES admission policy is correct.

For additional information about securing access to your Amazon ES domain, encounter How to Command Admission to Your Amazon Elasticsearch Service Domain.

The OSSEC HIDS alerts now are beingness processed into Amazon ES. To utilise Kibana to analyze the warning information interactively, you must configure an index pattern that identifies the data you wish to analyze in Amazon ES. You can read additional information about alphabetize patterns in the Kibana documentation.

In the Index name or pattern box, type cwl-2017.*. The index pattern is generated inside the Lambda function as cwl-YYYY.MM.DD, so you tin can use a wildcard graphic symbol for the calendar month and day to match data from 2022. From the Time-field proper noun drop-down list, cull @timestamp, and then cull Create.

Screenshot of the "Configure an index pattern" screen

In Kibana, you lot should now be able to choose the Observe pane and see alerts being populated. To set the refresh charge per unit for the display of almost-real-time alerts, choose your desired time range in the top right (such as Last fifteen minutes).

Screenshot of setting the refresh rate of near-real-time alerts

Choose Automobile-refresh, so choose an interval, such as five seconds.

Screenshot of auto-refresh of 5 seconds

Kibana should now be configured to auto-refresh at a 5-second interval within the timeframe you lot configured. You should now come across your alerts updating forth with a count graph, as shown in the post-obit screenshot.

Screenshot of the alerts updating with a count graph

The EC2 instances are automatically configured by CloudFormation to simulate activity to display several types of alerts, including:

Successful sudo to ROOT executed – The Linux sudo command was successfully executed.
Web server 400 error code – The server cannot process the asking due to an apparent client error (such as malformed request syntax, likewise large size, invalid asking message framing, or deceptive request routing).
SSH insecure connexion attempt (scan) – Invalid connection endeavour to the SSH listener.
Login session opened – Opened login session on the system.
Login session closed – Airtight login session on the system.
New Yum bundle installed – Package installed on the organisation.
Yum packet deleted – Package deleted from the system.

Permit's take a closer look at some of the alert fields, as shown in the following screenshot.

Screenshot highlighting some of the alert fields

The numbered alert fields in the preceding screenshot are divers equally follows:

@log_group – The source CloudWatch Logs group
@log_stream – The CloudWatch Logs stream name (InstanceID)
@message – The JSON payload from the source alerts.json OSSEC log
@owner – The AWS account ID where the alert originated
@timestamp – The time stamp applied by the consumer Lambda function
full_log – The log effect from the source file
location – The source log file path and file name
rule.comment – A brief clarification of the OSSEC dominion that was matched
dominion.level – The OSSEC rule nomenclature from 0 to 16 (come across Rules Classification for more data)
rule.sidid – The rule ID of the OSSEC rule that was matched
srcip – The source IP address that triggered the alarm; in this example, the false alerts comprise the local IP of the server

You can enter search criteria in the Kibana query bar to explore HIDS alert data interactively. For example, y'all can run the following query to run across all the rule.level 6 alerts for the EC2 InstanceID i-0e427a8594852eca2 where the source IP is 10.10.10.10.

You tin perform searches including simple text, Lucene query syntax, or utilize the total JSON-based Elasticsearch Query DSL. Yous can discover additional information on searching your information in the Elasticsearch documentation.

3. Configure a Kibana HIDS dashboard and visualize alerts

To analyze alert trends and patterns over time, it tin can exist helpful to apply charts and graphs to represent the alert data. I take configured a basic dashboard template that y'all tin import into your Kibana instance.

To add the template of a sample HIDS dashboard to your Kibana instance:

Save the template locally and then choose Management in the Kibana navigation pane.
Choose Saved Objects, Import, and the HIDS dashboard template.
Choose the eye icon to the right of the HIDS Alerts dashboard entry. This will take you to the imported dashboard.

After importing the Kibana dashboard template and selecting it, yous will meet the HIDS dashboard, equally shown in the following screenshot. This sample HIDS dashboard includes Alerts Over Time, Top 20 Alert Types, Dominion Level Breakdown, Pinnacle 10 Rule Source ID, and Superlative x Source IPs.

Screenshot of the HIDS dashboard

To explore the warning data in more detail, you can choose an warning blazon on which to filter, as shown in the post-obit two screenshots.

Alert showing SSH insecure connection attempts

Alert showing @timestamp per 30 seconds

Y'all can see more than details about the alerts based on criteria such every bit source IP address or time range. For more information well-nigh using Kibana to visualize alarm data, see the Kibana User Guide.

Summary

In this blog post, I showed how to use CloudWatch Logs to collect alerts in near-real time from an OSSEC HIDS and use a CloudWatch Logs subscription to pass the alerts into Amazon ES for analysis and visualization with Kibana. The dashboard deployed by this solution can help you improve the security monitoring of your EC2 fleet every bit part of a defense-in-depth security strategy in your AWS environment.

You lot can use this solution to help notice attacks, anomalous activities, and fault trends beyond your EC2 fleet. You can also use it to help prioritize remediation efforts for your systems or help make up one's mind where to innovate additional security controls such as VPC security group rules, VPC network ACLs, or AWS WAF rules.

If you have comments about this mail service, add together them to the "Comments" section beneath. If you take questions about or problems implementing this solution, start a new thread on the CloudWatch or Amazon ES forum. The source lawmaking for this solution is bachelor on GitHub. If you need OSSEC-specific back up, see OSSEC Support Options.

– Cameron

Want more AWS Security how-to content, news, and characteristic announcements? Follow us on Twitter.