Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project tooling to generate example property descriptions #104

Merged
merged 11 commits into from
Sep 19, 2024
Merged
18 changes: 18 additions & 0 deletions .github/workflows/build-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,21 @@ jobs:

- name: validate example
run: make validate-examples

- name: generate descriptions
run: make generate-descriptions

- name: check for diff
run: |
# need to "git add" to detect any changes to descriptions which are not checked in
# select files from both /examples
git add examples**
if git diff --cached --quiet
then
echo "No diff detected."
else
echo "Diff detected - did you run 'make generate-descriptions'?"
echo $(git diff --cached --name-only)
echo $(git diff --cached)
exit 1
fi
61 changes: 61 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,67 @@ You can perform all checks locally using this command:
make all
```

## Description generation

The [./examples](./examples) directory contains a variety of examples, which
include important comments describing the semantics of the configuration
properties. In order to keep these comments consistent across examples, we have
tooling which automatically generates comments for each property.

How it works:

* The [./schema/type_descriptions.yaml](./schema/type_descriptions.yaml) file
defines descriptions for each of the properties of each type defines in
the [JSON schema](./schema) data model.
* The [./scripts/generate-comments.js](./scripts/generate-comments.js) is a
script which for a given input configuration file will:
* Parse the YAML.
* Walk through each key / value pair, and for each:
* Compute the JSON dot notation location of the current key / value pair.
* Find the first matching rule
in [type_description.yaml](./schema/type_descriptions.yaml). Iterate
through the rules and evaluate the key / value pair dot notation location
against each of the rule's `path_patterns`.
* Inject / overwrite comments for its properties according
to `type_descriptions.yaml`.
* Write the resulting content to specified output file or to the console.

The `make generate-descriptions` command runs this process against each file
in `./examples` and overwrites each file in the process.

**NOTE:** The [build](./.github/workflows/build-check.yaml) will fail
if `make generate-descriptions` produces any changes which are not checked into
version control.

To run against a single file:

- Install the latest LTS release of **[Node](https://nodejs.org/)**.
For example, using **[nvm][]** under Linux run:

```bash
nvm install --lts
```

- Install tooling packages:

```bash
npm install
```

- Run the script:

```shell
npm run-script generate-comments -- /absolute/path/to/input/file.yaml /absolute/path/to/output/file.yaml
```

Optionally, include the `--debug` parameter. This prints out information about
each key value pair, including the computed dot notation location, the matched
rule, the previous description, the new description, etc.

```shell
npm run-script generate-comments -- /absolute/path/to/input/file.yaml /absolute/path/to/output/file.yaml --debug
```

## Pull requests

A PR is ready to merge when:
Expand Down
13 changes: 10 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ EXAMPLE_FILES := $(shell find . -path './examples/*.yaml' -exec basename {} \; |
$(shell mkdir -p out)

.PHONY: all
all: install-tools compile-schema validate-examples
all: install-tools compile-schema generate-descriptions validate-examples

.PHONY: compile-schema
compile-schema:
@if ! npm ls ajv-cli; then npm install; fi
@for f in $(SCHEMA_FILES); do \
npx --no ajv-cli compile --spec=draft2020 --allow-matching-properties -s ./schema/$$f -r "./schema/!($$f)" \
npx --no ajv-cli compile --spec=draft2020 --allow-matching-properties -s ./schema/$$f -r "./schema/!($$f|*.yaml)" \
|| exit 1; \
done

Expand All @@ -18,10 +18,17 @@ validate-examples:
@if ! npm ls ajv-cli; then npm install; fi
@for f in $(EXAMPLE_FILES); do \
npx envsub ./examples/$$f ./out/$$f || exit 1; \
npx --no ajv-cli validate --spec=draft2020 --allow-matching-properties --errors=text -s ./schema/opentelemetry_configuration.json -r "./schema/!(opentelemetry_configuration.json)" -d ./out/$$f \
npx --no ajv-cli validate --spec=draft2020 --allow-matching-properties --errors=text -s ./schema/opentelemetry_configuration.json -r "./schema/!(opentelemetry_configuration.json|*.yaml)" -d ./out/$$f \
|| exit 1; \
done

.PHONY: generate-descriptions
generate-descriptions:
@if ! npm ls minimatch yaml; then npm install; fi
@for f in $(EXAMPLE_FILES); do \
npm run-script generate-comments -- $(shell pwd)/examples/$$f $(shell pwd)/examples/$$f || exit 1; \
done

.PHONY: install-tools
install-tools:
npm install
Expand Down
34 changes: 27 additions & 7 deletions examples/anchors.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# anchors.yaml demonstrates anchor substitution to reuse OTLP exporter configuration across signals.

# The file format version.
file_format: "0.1"
exporters:
otlp: &otlp-exporter
Expand All @@ -8,37 +9,56 @@ exporters:
client_key: /app/cert.pem
client_certificate: /app/cert.pem
headers:
api-key: !!str 1234
api-key: !!str "1234"
compression: gzip
timeout: 10000

# Configure logger provider.
logger_provider:
# Configure log record processors.
processors:
- batch:
- # Configure a batch log record processor.
batch:
# Configure exporter.
exporter:
# Configure exporter to be OTLP.
otlp:
# expand the otlp-exporter anchor
<<: *otlp-exporter
# Configure endpoint.
endpoint: http://localhost:4318/v1/logs

# Configure meter provider.
meter_provider:
# Configure metric readers.
readers:
- periodic:
- # Configure a periodic metric reader.
periodic:
# Configure delay interval (in milliseconds) between start of two consecutive exports.
interval: 5000
# Configure maximum allowed time (in milliseconds) to export data.
timeout: 30000
# Configure exporter.
exporter:
# Configure exporter to be OTLP.
otlp:
# expand the otlp-exporter anchor and add metric specific configuration
<<: *otlp-exporter
# Configure endpoint.
endpoint: http://localhost:4318/v1/metrics
# Configure temporality preference.
temporality_preference: delta
# Configure default histogram aggregation.
default_histogram_aggregation: base2_exponential_bucket_histogram

# Configure tracer provider.
tracer_provider:
# Configure span processors.
processors:
- batch:
- # Configure a batch span processor.
batch:
# Configure exporter.
exporter:
# Configure exporter to be OTLP.
otlp:
# expand the otlp-exporter anchor
<<: *otlp-exporter
# Configure endpoint.
endpoint: http://localhost:4318/v1/traces
Loading
Loading