Skip to content

Commit 5d334fc

Browse files
vishaalcharpevishaalcharpe
committed
DEV-203529: DEV-203529-logstash-customisable-value-for-_resource.type-field
1 parent 5023e32 commit 5d334fc

File tree

5 files changed

+28
-15
lines changed

5 files changed

+28
-15
lines changed

‎CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,11 @@
1818
- Dont send event metadata by default. Add config include_metadata_keys for including custom metadata
1919
## 2.0.1
2020
- Fix incomplete json metadata error.
21+
## 2.0.2
22+
- Added domain support for GovCloud
23+
## 2.0.3
24+
- Added validation for domain support
25+
## 2.0.4
26+
- Added resource type in metadata.
27+
## 2.0.5
28+
- Added support for customizable resource type.

‎README.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -29,17 +29,17 @@ The allowed values for portal_domain are ["logicmonitor.com", "lmgov.us", "qa-lm
2929

3030
## Important options
3131

32-
| Option | Description| Default |
33-
| --- | --- | --- |
34-
| batch_size | Event batch size to send to LM Logs.| 100 |
35-
| message_key | Key that will be used by the plugin as the system key | "message" |
36-
| lm_property | Key that will be used by LM to match resource based on property | "system.hostname" |
37-
| keep_timestamp | If false, LM Logs will use the ingestion timestamp as the event timestamp | true |
38-
| timestamp_is_key | If true, LM Logs will use a specified key as the event timestamp | false |
39-
| timestamp_key | If timestamp_is_key is set, LM Logs will use this key in the event as the timestamp | "logtimestamp" |
40-
| include_metadata | If true, all metadata fields will be sent to LM Logs | false |
41-
| include_metadata_keys | Array of json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson. If config `include_metadata` is set to true, all metadata will be sent regardless of this config. | [] |
42-
32+
| Option | Description | Default |
33+
|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|
34+
| batch_size | Event batch size to send to LM Logs. | 100 |
35+
| message_key | Key that will be used by the plugin as the system key | "message" |
36+
| lm_property | Key that will be used by LM to match resource based on property | "system.hostname" |
37+
| keep_timestamp | If false, LM Logs will use the ingestion timestamp as the event timestamp | true |
38+
| timestamp_is_key | If true, LM Logs will use a specified key as the event timestamp | false |
39+
| timestamp_key | If timestamp_is_key is set, LM Logs will use this key in the event as the timestamp | "logtimestamp" |
40+
| include_metadata | If true, all metadata fields will be sent to LM Logs | false |
41+
| include_metadata_keys | Array of json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson. If config `include_metadata` is set to true, all metadata will be sent regardless of this config. | [] |
42+
| resource_type | If a Resource Type is explicitly specified, that value will be statically applied to all ingested logs. If set to `predef.externalResourceType`, the Resource Type will be assigned dynamically based on the `predef.externalResourceType` property set for the specific resource the logs are mapped to in LM. If left blank, the Resource Type field will remain unset in the ingested logs. | "" |
4343
See the [source code](lib/logstash/outputs/lmlogs.rb) for the full list of options
4444

4545
The syntax for `message_key` and `source_key` values are available in the [Logstash Event API Documentation](https://www.elastic.co/guide/en/logstash/current/event-api.html)

‎lib/logstash/outputs/lmlogs.rb

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,9 @@ class InvalidHTTPConfigError < StandardError; end
110110
# json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson.
111111
config :include_metadata_keys, :validate => :array, :required => false, :default => []
112112

113+
# Customised resource_type to map logs to existing LM resources
114+
config :resource_type, :validate => :string, :required => false
115+
113116
@@MAX_PAYLOAD_SIZE = 8*1024*1024
114117

115118
# For developer debugging.
@@ -308,7 +311,10 @@ def processEvent(event)
308311
if @include_metadata
309312
lmlogs_event = event_json
310313
lmlogs_event.delete("@timestamp") # remove redundant timestamp field
311-
lmlogs_event["_resource.type"]="Logstash"
314+
unless @resource_type.to_s.strip.empty?
315+
lmlogs_event["_resource.type"] = @resource_type
316+
end
317+
312318
if lmlogs_event.dig("event", "original") != nil
313319
lmlogs_event["event"].delete("original") # remove redundant log field
314320
end

‎lib/logstash/outputs/version.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# frozen_string_literal: true
22

33
module LmLogsLogstashPlugin
4-
VERSION = '2.0.4'
4+
VERSION = '2.0.5'
55
end

‎spec/outputs/metadata_spec.rb

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,7 @@ def check_same_hash(h1,h2)
7676
"nested" => {"nested2" => {"nested3" => "value",
7777
"nested3b" => "value"},
7878
"nested_ignored" => "somevalue"
79-
},
80-
"_resource.type"=>"Logstash"
79+
}
8180
}
8281
puts " actual : #{constructed_event} \n expected : #{expected_event}"
8382
puts " hash diff : #{Hashdiff.diff(constructed_event,expected_event)}"

0 commit comments

Comments
 (0)