Heka Decoders

Heka Decoders

Decoders parse the contents of the inputs to extract data from the text format and map them onto a Heka message schema. List of all available decoders: https://hekad.readthedocs.org/en/v0.10.0/config/decoders/index.html
On controller we have the follwing decoders configured:

• decoder-collectd.toml
• decoder-http-check.toml
• decoder-keystone_7_0.toml
• decoder-keystone_wsgi.toml
• decoder-mysql.toml
• decoder-notification.toml
• decoder-openstack.toml
• decoder-ovs.toml
• decoder-pacemaker.toml
• decoder-rabbitmq.toml
• decoder-swift.toml
• decoder-system.toml

All custom decoders decoders are SandboxDecoder

SandboxDecoder simple example

Sandbox documentation: https://hekad.readthedocs.org/en/v0.10.0/sandbox/index.html
Sandbox decoder is complex part of Heka, so for better understanding there it is possible to create simple input and simple decoder. Idea is

• use the same input source as was used in collectd (read data from file "/var/log/collectd_in_data")
• use lua-based decoder to decode input data

Input

The best way to poll file is Heka. FilePollingInput. Configuration is pretty simple, this input plugin read data from file every ticker_interval. FilePollingInput can be used to get data from /proc file system, e.g. from /proc/loadavg. One more example is application which rewrites 'stats' files.
E.g. we have the following configuration:

[test_input]
type = "FilePollingInput"
ticker_interval = 1
file_path = "/var/log/collectd_in_data"
decoder = "test_decoder"

Decoder

Decoder in general are getting data from input plugins and convert it into Heka internal format.

[mm_decoder]
type = "SandboxDecoder"
filename = "/usr/share/lma_collector/decoders/mm.lua"