Metron Parsing Topology (also known as the Normalizing Topology) is designed to take a sensor input (in it's native format) and turn it into a Metron JSON Object. The topology consists of two components: a standard Storm Kafka Spout (reads data from a Kafka topics and ingests it into a storm topology) and a Metron Parser Kafka Bolt (which parses the message and puts it back onto a Kafka "enrichment" topic.
The Kafka Parser Bolt is an extensible bolt that can take two types of parsers: GROK and purpose-built Java. It is our strong preference to build up as much coverage as we can using the Grok parsers, as they require no coding. However, the drawback of the Grok parsers is that they are much slower than Java parsers and sometimes the telemetry is so complex that writing a Grok statement for it is not feasible. See our list of supported devices for which parser type supports a particular device.
| Parser Type | Telemetry Type | Parser Reference |
|---|---|---|
| Grok | Simple to parse, low volume | Grok Parsers |
| Java | Complex to parse, high volume | Java Parsers |
Each parser outputs a Metron JSON formatted message. The reason why JSON was chosen as a format for Metron is a very flexible data structure that allows nesting and lists. We do not dictate what the JSON structure should look like, however, Metron is looking for specific normalized fields in order to do downstream processing.