Doc how to parse telegraf logs

influxdata · Jun 14, 2018 · 57f78a3 · 57f78a3
1 parent 0dda9b8
commit 57f78a3
Showing 1 changed file with 42 additions and 0 deletions.
diff --git a/plugins/inputs/logparser/README.md b/plugins/inputs/logparser/README.md
@@ -213,6 +213,48 @@ A multi-line literal string allows us to encode the pattern:
     custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
 ```
 
+#### Parsing Telegraf log file
+We can use logparser to convert the log lines generated by Telegraf in metrics.
+
+To do this we need to configure Telegraf to write logs to a file (if we have systemd, by
+default, it will be written to journald):
+```toml
+[agent]
+  logfile = "/var/log/telegraf.log"
+```
+
+Logparser configuration:
+```toml
+[[inputs.logparser]]
+  files = ["/var/log/telegraf.log"]
+
+  [inputs.logparser.grok]
+    measurement = "telegraf_log"
+    patterns = ['\A%{TIMESTAMP_ISO8601:timestamp:ts-rfc3339} %{TELEGRAF_LOG_LEVEL:level:tag}! %{GREEDYDATA:msg}']
+    custom_patterns = '''
+TELEGRAF_LOG_LEVEL (?:[DIWE]+)
+'''
+```
+
+Example log lines:
+```
+2018-06-14T06:41:35Z I! Starting Telegraf v1.6.4
+2018-06-14T06:41:35Z I! Agent Config: Interval:3s, Quiet:false, Hostname:"archer", Flush Interval:3s
+2018-02-20T22:39:20Z E! Error in plugin [inputs.docker]: took longer to collect than collection interval (10s)
+2018-06-01T10:34:05Z W! Skipping a scheduled flush because there is already a flush ongoing.
+2018-06-14T07:33:33Z D! Output [file] buffer fullness: 0 / 10000 metrics.
+```
+
+Generated metrics:
+```
+telegraf_log,host=somehostname,level=I msg="Starting Telegraf v1.6.4" 1528958495000000000
+telegraf_log,host=somehostname,level=I msg="Agent Config: Interval:3s, Quiet:false, Hostname:\"somehostname\", Flush Interval:3s" 1528958495001000000
+telegraf_log,host=somehostname,level=E msg="Error in plugin [inputs.docker]: took longer to collect than collection interval (10s)" 1519166360000000000
+telegraf_log,host=somehostname,level=W msg="Skipping a scheduled flush because there is already a flush ongoing." 1527849245000000000
+telegraf_log,host=somehostname,level=D msg="Output [file] buffer fullness: 0 / 10000 metrics." 1528961613000000000
+```
+
+
 ### Tips for creating patterns
 
 Writing complex patterns can be difficult, here is some advice for writing a