Logging in distributed system Part 5 — Parsing logs

This is the fifth part of the Logging series. For your convenience you can find other parts in the table of contents in Part 1 – Correlations

We start with classes representing logs:

Since our logs might contain additional data, we do not want to lose it. So we push two types of logs to the OMS: parsed log and raw log. Raw log is only splitted by each line (you could as well push whole file), parsed log contains parsed data so it is easier to filter it and create alerts.

Now the parsers:

Now when pushing data to OMS do not forget to set timestamp field header in order to avoid duplicates for parsed logs:


This short series shows how to implement logging infrastructure for distributed system. Please be advised that this is only an initial implementation which you should adapt to your needs.