Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Otel trace workflow #607

Merged
merged 9 commits into from
Jun 24, 2024
Merged

Otel trace workflow #607

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4792aa1
test workflow with otel traces
sallyom Jun 7, 2024
f37bebe
add traces to testing_framework
sallyom Jun 7, 2024
8089c22
update workflow triggers
sallyom Jun 18, 2024
0a27f9f
fix indent
sallyom Jun 18, 2024
6a72550
add traces to chatbot workflow
sallyom Jun 18, 2024
3af8da0
mTLS & configure otel-collector to ROSA-ET
sallyom Jun 20, 2024
dc012ed
debug curl
sallyom Jun 20, 2024
1f0f1f9
update workflow
sallyom Jun 21, 2024
4e36394
another try
sallyom Jun 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 102 additions & 4 deletions .github/workflows/chatbot.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,65 @@ jobs:
steps:
- uses: actions/checkout@v4.1.7

- name: Set up Python
uses: actions/setup-python@v5.1.0
with:
python-version: '3.11'

- name: Install opentelemetry dependencies
run: |
pip install --no-cache-dir opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-instrumentation

- name: Configure OpenTelemetry Collector
run: |
echo '
extensions:
basicauth/client:
client_auth:
username: "${{ secrets.OTEL_USERNAME }}"
password: "${{ secrets.OTEL_PASSWORD }}"
receivers:
otlp:
protocols:
grpc:
http:
exporters:
otlphttp:
endpoint: https://otc.apps.platform-sts.pcbk.p1.openshiftapps.com
auth:
authenticator: basicauth/client
tls:
insecure: false
ca_pem: "${{ secrets.ROSA_ROOT_CERT }}"
debug:
verbosity: detailed
service:
extensions: [basicauth/client]
pipelines:
traces:
receivers: [otlp]
exporters: [debug, otlphttp]
' > otel-collector-config.yaml

- name: Run OpenTelemetry Collector
run: |
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.103.0/otelcol-contrib_0.103.0_linux_amd64.tar.gz
tar -xvf otelcol-contrib_0.103.0_linux_amd64.tar.gz
chmod +x otelcol-contrib
./otelcol-contrib --config otel-collector-config.yaml &

- name: Install qemu dependency
run: |
sudo apt-get update
sudo apt-get install -y qemu-user-static

- name: Start build trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="build-image"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Build Image
id: build_image
uses: redhat-actions/buildah-build@v2.13
Expand All @@ -54,24 +108,54 @@ jobs:
containerfiles: ./recipes/natural_language_processing/${{ env.IMAGE_NAME }}/app/Containerfile
context: recipes/natural_language_processing/${{ env.IMAGE_NAME }}/app

- name: Set up Python
uses: actions/setup-python@v5.1.0
with:
python-version: '3.11'
- name: End build trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="build-image"
export TRACE_ACTION="end"
python ci/trace-steps.py

- name: Install Dependencies
working-directory: ./recipes/natural_language_processing/${{ env.IMAGE_NAME }}
run: make install

- name: Start download model trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="download-model"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Download model
working-directory: ./models
run: make download-model-granite

- name: End download model trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="download-model"
export TRACE_ACTION="end"
python ci/trace-steps.py

- name: Start functional test run trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="run-functional-tests"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Run Functional Tests
shell: bash
run: make functional-tests
working-directory: ./recipes/natural_language_processing/${{ env.IMAGE_NAME }}

- name: End functional test run trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="run-functional-tests"
export TRACE_ACTION="end"
python ci/trace-steps.py

- name: Login to Registry
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: redhat-actions/podman-login@v1.7
Expand All @@ -80,6 +164,13 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Start push image trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="push-image"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Push Image
id: push_image
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
Expand All @@ -88,3 +179,10 @@ jobs:
image: ${{ steps.build_image.outputs.image }}
tags: ${{ steps.build_image.outputs.tags }}
registry: ${{ env.REGISTRY }}

- name: End push image trace
run: |
export WORKFLOW_NAME="chatbot-build-push"
export STEP_NAME="push-image"
export TRACE_ACTION="end"
python ci/trace-steps.py
112 changes: 112 additions & 0 deletions .github/workflows/test-trace-steps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# To run locally
# act -W .github/workflows/test-trace-steps.yaml --container-architecture linux/amd64 -b ci/logs:/logs

name: Test Workflow

on:
pull_request:
branches:
- main
paths:
- .github/workflows/test-trace-steps.yaml
workflow_dispatch:

jobs:
test:
if: "!contains(github.event.pull_request.labels.*.name, 'hold-tests')"
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4.1.7
- name: Set up Python
uses: actions/setup-python@v5.1.0
with:
python-version: '3.11'

- name: Install Python dependencies
run: |
pip install --no-cache-dir opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-instrumentation

- name: Download OpenTelemetry Collector Contrib
run: |
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.103.0/otelcol-contrib_0.103.0_linux_amd64.tar.gz
tar -xvf otelcol-contrib_0.103.0_linux_amd64.tar.gz

- name: Write secrets to files
run: |
echo "${{ secrets.ROSA_OTEL_TLS_CERT }}" > /tmp/tls.crt

- name: Configure OpenTelemetry Collector
run: |
echo '
extensions:
basicauth/client:
client_auth:
username: "${{ secrets.OTEL_USERNAME }}"
password: "${{ secrets.OTEL_PASSWORD }}"
receivers:
otlp:
protocols:
grpc:
http:
exporters:
otlphttp:
endpoint: https://otc.apps.platform-sts.pcbk.p1.openshiftapps.com:4318
auth:
authenticator: basicauth/client
tls:
insecure: false
ca_file: /tmp/tls.crt
debug:
verbosity: detailed
service:
extensions: [basicauth/client]
pipelines:
traces:
receivers: [otlp]
exporters: [debug, otlphttp]
' > otel-collector-config.yaml

- name: Run OpenTelemetry Collector
run: |
./otelcol-contrib --config otel-collector-config.yaml > otel-collector.log 2>&1 &

- name: Start build trace
run: |
export WORKFLOW_NAME="test-workflow"
export STEP_NAME="build"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Build
run: |
echo "Simulating build step..."
sleep 2

- name: End build trace
run: |
export WORKFLOW_NAME="test-workflow"
export STEP_NAME="build"
export TRACE_ACTION="end"
python ci/trace-steps.py

- name: Start test trace
run: |
export WORKFLOW_NAME="test-workflow"
export STEP_NAME="test"
export TRACE_ACTION="start"
python ci/trace-steps.py

- name: Test
run: |
echo "Simulating test step..."
sleep 2

- name: End test trace
run: |
export WORKFLOW_NAME="test-workflow"
export STEP_NAME="test"
export TRACE_ACTION="end"
python ci/trace-steps.py

- name: Display OpenTelemetry Collector Logs
run: cat otel-collector.log
Loading
Loading