Update changelog ahead of v0.29.0 release

[djaglowski]

This will be the final release of this repository before merging the codebase into the contrib collector.

This contains breaking changes, some of which concern similar the same functionality as was changed in v0.28.0. SInce v0.28.0 has not yet been included in a release of the collector, I plan to create an new upgrade guide that describes all changes in both v0.28.0 and v0.29.0, and present that as a single guide for the next collector release.

[djaglowski]

Several changes have been made that affect configuration for the filelog, syslog, tcplog, udplog, and journald receivers.

Update all usages of field syntax

Field syntax no longer requires the $ character. Instead each field must begin with body, attributes, or resource.

Deprecated Example	Updated Equivalent
$$attributes["log.file.name"]	attributes["log.file.name"]
$$resource["host.name"]	resource["host.name"]
$$body.foo	body.foo
$$.foo	body.foo
foo	body.foo

Tip for updating sub-parsers

To update the parse_from field in a "sub-parser", such as timestamp or severity, consider where the value would reside if the sub-parser was excluded.

Deprecated Example	Updated Equivalent
operators: - type: regex_parser regex: '^Time=(?P<time>\d{4}-\d{2}-\d{2})...' parse_to: body # default timestamp: parse_from: time ...	operators: - type: regex_parser regex: '^Time=(?P<time>\d{4}-\d{2}-\d{2})...' parse_to: attributes # default timestamp: parse_from: attributes.time ...

attributes and resource now support arbitrary structure

Field syntax can now refer to nested fields in both attributes and resource. Nested fields were previously only supported in body. For example, attributes.foo.bar refers to the value "hello" in the following entry:

{
  "attributes": {
    "foo": {
      "bar": "hello"
    }
  }
}

Update references to parsed values

The default value of parse_to has been changed from body to attributes. As a general rule of thumb, any parsers that previously specified the parse_to field are unaffected. If the parse_to field was not previously specified, then subsequent operators may need to be updated.

Deprecated Example	Updated Equivalent
operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body # parse_to: body (old default) - type: move from: foo to: attributes.bar ...	operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body # parse_to: attributes (new default) - type: move from: attributes.foo to: attributes.bar ...

Remove or replace usages of preserve_to

Parsers no longer support a preserve_to setting. Instead, the parse_from field is preserved by default. To preserve and move the field, use the move operator after the parse operator. To remove the field after parsing, use the remove operator.

Deprecated Example	Updated Equivalent
operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body preserve_to: attributes.backup ...	operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body - type: move from: body to: attributes.backup ...
operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body # implicitly deleted ...	operators: - type: regex_parser regex: '^Foo=(?P<foo>\s.*)...' parse_from: body - type: remove field: body ...

Replace usages of restructure operator

The restructure operator has been removed. Use add, copy, flatten, move, remove, and retain operators instead.

Deprecated Example	Updated Equivalent
operators: - type: restructure ops: - add: field: set_me value: foo - add: field: overwrite_me value: bar - move: from: details.env to: env - remove: field: delete_me	operators: - type: add field: body.set_me value: foo - type: add field: body.overwrite_me value: bar - type: move from: body.details.env to: body.env - type: remove field: body.delete_me

Replace usages of metadata operator

The metadata operator has been removed. Use add, copy, or move operators instead.

Deprecated Example	Updated Equivalent
operators: - type: metadata attributes: environment: production file: 'EXPR( $$body.file )' resource: cluster: blue	operators: - type: add field: attributes.environment value: production - type: copy from: body.file to: attributes.file - type: add field: resource.cluster value: blue - type: move from: body.foo to: attributes.bar

Update filelog attribute references

The filelog receiver has adopted slightly different attribute names in order to match newly established semantic conventions. Configurations that previously refered to the file.* attributes should be updated.

Deprecated Attribute	Updated Equivalent
file.name	log.file.name
file.path	log.file.path
file.name.resolved	log.file.name_resolved
file.path.resolved	log.file.path_resolved

Note to Vendors: A log record's Timestamp field may now be nil

A recent change to the Log Data Model has redefined the usage of the Timestamp field. Correspondingly, this field is no longer initialized by default. All Log Exporters should be evaluated to ensure this change is handled accordingly.

Log exporters can use the following logic to mimic the previous functionality (psuedocode):

timestamp := log.ObservedTimestamp
if log.Timestamp != nil {
  timestamp = log.Timestamp
}

[djaglowski]

#467 will address the unit test failures.

[djaglowski]

@open-telemetry/log-collection-approvers Please take a look. I would very much like to cut this release today in order to ensure it is in the next collector release on Wednesday.

[tigrannajaryan]

The explanation of changes is so nice in the PR description. Can we copy it into the CHANGELOG or if we don't want to make CHANGELOG so detailed, copy somewhere else and link from the CHANGELOG?

[tigrannajaryan]

LGTM.
I would like @dmitryax to also have a look before moving forward.

[djaglowski]

The explanation of changes is so nice in the PR description. Can we copy it into the CHANGELOG or if we don't want to make CHANGELOG so detailed, copy somewhere else and link from the CHANGELOG?

Done. I'll also include it in the release notes.

[djaglowski]

@tigrannajaryan, would you mind taking a look at this PR as well? This would enable the timestamp logic mentioned in these release notes.

[jsirianni]

Looks great 👍