Lab 1 - Setting up Elastic in AWS

Base Setup:

Setup Instance:

Update Security Group

Port 5601: Kibana

Port 9200: Elasticsearch

Building ELK:

Install Elasticsearch:

SSH into Instance:

Commands to install Elastic Search

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

sudo apt-get update
sudo apt-get install apt-transport-https

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

sudo apt-get update
sudo apt-get install elasticsearch

Now that we have installed Elasticsearch we must change some configs

It is a good best practice to bind Elasticsearch to the private IP (like 172.31.something):

sudo nano /etc/elasticsearch/elasticsearch.yml

Now to start Elasticsearch

sudo service elasticsearch start

Test Elasticsearch

curl http://<Private IP>:9200

Install Logstash:

sudo apt-get install default-jre

Verify the java version with the command below

java -version

Now install logstash

sudo apt-get install logstash

Create data pipeline:

mkdir /logstash
cd /logstash
# pull down sample data
sudo chown -R logstash /logstash
sudo chgrp -R logstash /logstash

Create Logstash Configuration File:

sudo nano /etc/logstash/conf.d/apache-01.conf

To test config

/usr/share/logstash/bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/apache-01.conf

It can take a minute or so to run - but should get a configuration OK message

Start Logstash

sudo service logstash start

Test if it's working

curl http://<Private IP>:9200/_cat/indices?v

Install Kibana:

sudo apt-get install kibana

Edit config

sudo nano /etc/kibana/kibana.yml

Change the lines below

server.port: 5601
server.host: '<YourPrivateIP>'
elasticsearch.hosts: ["http://<YourPrivateIP>:9200"]

Start Kibana

sudo service kibana start

Test Kibana

Add an Index Pattern to display to Logstash Index

Go to Stack Management → Kibana -> Index Patterns - select “Create Index Pattern”

Use Kibana to query data