Using Grok and Geoip filters in Logstash

Photo by Antonio Grosz on Unsplash

In this post, we will see an example of using Logstash to read input from a file and write to stdout and apply two filter plugins, the Grok and Geoip.

This video will show the steps: (No Audio, just illustration)

We need to have a configuration file, in my case I put in: /etc/logstash/conf.d/grok_geoip.conf then started Logstash using: logstash -f /etc/logstash/conf.d/grok_geoip.conf

Here’s the conf

input {
file {
path => "/home/vermin/input.txt"
}
}
filter {
grok {
match => { "message" => "%{WORD:name} %{IP:ip} %{TIMESTAMP_ISO8601:date}" }
remove_field => [ "message", "path", "@version", "host" ]
}
geoip {
source => "ip"
}
}
output {
stdout {}
}

We can test the configuration by writing to a file/home/vermin/input.txt:

You can change the path of the input file according to your environment, or in case you use vermin to setup logstash, you can use the exact same configuration.

$ echo "sami 19.1.193.230 $(date --iso-8601=seconds)" >> ~/input.txt

Now you see the log from stdout of Logstash as follows:

{
"@timestamp" => 2020-08-14T14:55:28.515Z,
"date" => "2020-08-14T14:55:27+00:00",
"geoip" => {
"continent_code" => "NA",
"timezone" => "America/Chicago",
"country_code3" => "US",
"latitude" => 37.751,
"location" => {
"lon" => -97.822,
"lat" => 37.751
},
"country_name" => "United States",
"country_code2" => "US",
"longitude" => -97.822,
"ip" => "19.1.193.230"
},
"ip" => "19.1.193.230",
"name" => "sami"
}

Software Developer/Architect

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store