Making Peace with Logstash Part 1 – Input and Output
2February 21, 2018 by Mike Hillwig
Logstash is an incredibly powerful tool. If you can put data into a text file, Logstash can parse it. It works well with a lot of data, but I’m finding myself using it more to use it for event data. When I say event data, if it triggers a log event and it writes to a log, it’s an event. For the purposes of my demos, I’m using data from the Bureau of Transportation Statistics. They track flight performance data, which works perfectly for my uses. It’s a great example dataset without using anything related to my real job.
Logstash configuration files typically have three sections, INPUT, FILTER, and OUTPUT. However, FILTER is optional.
In this case, my configuration looks like this:
[code]input {
file {
path => [“/Users/mikehillwig/elastic/flights/*.csv”]
sincedb_path => “/dev/null”
start_position => “beginning”
}
}
output {
stdout {}
}[/code]
Some of this data makes sense, but some of it is just noise, Next time, I’ll show you how to parse this as a CSV file, then we’ll eliminate some noise.
Category Uncategorized | Tags:
