Planning (and a little doing)

I'm starting to feel that blog may only be interesting to me. That's OK. If you're reading this, Hi and welcome. I'm just continuing to reevaluate what it is I like about technology and thought that writing it down where someone might stumble across it could motivate me somehow.

Lab/test environment

  • 192.168.130.101 - My workstation
  • 192.168.160.103 - The VM
  • 192.168.101.27 - Aruba switch

In a previous post on planning a SIEM implementation I realized that I haven't been using the right words. I now believe I should have been Googling "why isn't ELK indexing?".

So, I'm going to start over by once-again setting up the server mentioned in step 4 of that post. I'll be sure to create a checkpoint this time, and test this piece. Why wouldn't I start with step one? Well, I've only began to recognize the issues I'm having, not trying to solve them all at once.

One thing to note, I recognize how difficult it can be to learn when blogs and how-to videos use jargon. Why do people use abbreviations and slang when writing or answering technical questions? I see this a lot on Reddit, a little on YouTube tutorials, and constantly on forums. I know this shouldn't be revolutionary but I'm going to do my best to reference things in the way that I did when talking about VMs above but will skip it for the basics like when I reference IP addresses below. Mostly, I'll introduce it, then use the abbreviation going forward.

  1. Installed Ubuntu 20.04.1 LTS on a Microsoft Hyper-V VM and made sure to enable SSH during the setup wizard.

After a restart, I noted the IP address it received. There's a lot to be said about networking, firewalls, and overall communicating between this new VM and my workstation but I wont be addressing that here. I know they can "talk" to each other in my lab environment and I've reserved that IP address in my DHCP server.

  1. I always make sure to update a newly installed server.
  2. After a restart, I powered off the VM and created a checkpoint. This will allow me to roll back any changes (read: mistakes) I make. My uncertainty really ramps up after this point and I'd like to be able to undo changes.
  3. Using Putty, I connected to the new server.
  4. I'm going to piece together my approach following a few different tutorials. This time, I'm going to be sure to completely read through each before starting. :)

How To Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 20.04 by Erin Glass which was published in June of 2020. This uses Nginx. I hadn't planned on this for the time being. I really want to wrap my head around each piece individually and hadn't planned on adding reverse proxies to that list.

The Complete Guide to the ELK Stack by Dotan Horovits which was last updated in June 2020. This initially appears to be focused on a specific platform but I really like the introduction and explanation of the ELK stack. It should be a good reference.

One thing that will be tugging at me is whether any major changes have been made since this was published. I know I can go back and review the individual release notes for Elastic and the other components, but then I'd likely have the answers to my questions. :)

  1. I knew I still had my Aruba Instant On 1930 switch set to forward its logs to the new VM's IP address from my last test. In theory, I should be able to see that log traffic arriving at the VM buy using tcpdump.

I ran tcpdump host but didn't see anything at all. I logged in and out of the switch in hopes that would generate log traffic. It did within the switch's web-based interface but not in the tcpdump output on the VM.

I pinged the VM from the switch and finally saw some output. So, is the VM not listening on UDP port 514 (the switch's default send-to port)? I installed net-tools on the VM and ran netstat -anlpu which the internet said would "list your UDP listening ports". I didn't see 514 but, now that I'm writing this out, why did I expect a newly installed server would be listening on that without being configured to do so?Would installing the Elk stack resolve that? I suspect so.

So, let's install some things.

Elasticsearch

I started with How To Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 20.04 for the Elasticsearch install, backing up the elasticsearch.yml file before editing it.

Both articles recommend setting the network.host option in the elasticsearch.yml file to localhost, starting the service, and finally setting the service to start each time your server does.

A quick check using curl -X GET "localhost:9200" outputs text that implies we haven't messed anything up yet.

Kibana

This step was similar. However, this is where I had to move to The Complete Guide to the ELK Stack because I'm not yet interested in nginx. Ultimately, install, backup and configure the kibana.yml file, start kibana, make sure it starts later.

The thing I didn't understand before was the relationship between Elasticsearch and Kibana. Some tutorials assumed you had multiple servers (which I should have been able to recognize weren't for my use case). Next, in the Elasticsearch file, I didn't fully realize that my workstation would not be needing to connect directly. I'd be connecting instead (via HTTP) to the Kibana service on port 5601. Later, I'll revisit this whole thing to ensure I'm using HTTPS. Can we navigate to Elastic using a web browser on my workstation?

yesKibana.png

Yes! Now, I'd like to get some data flowing but at this point, I powered off the VM and took another snapshot. So far so good.

Next up, Logstash collects data and sends it to Elastic.