Jenkins 2 GH Actions Workflow Migration

[reakaleek]

Details

While migrating to GH Actions I had to utilise GoReleaser in conjunction with buildx to be able to build ARM compatible artifacts.

How to test

I ran following steps locally:

Run without pushing or publishing to AWS:

export BUILD_DATE=202301251438.25
export DOCKER_REGISTRY=fakeregistry
export DOCKER_IMAGE_NAME=test
export GORELEASER_CURRENT_TAG=v0.0.9999
go run github.com/goreleaser/goreleaser@v1.14.1 release --snapshot --rm-dist

Test aws publish script:

⚠️ You also have to export AWS_* environments for a non production account.

export BUILD_DATE=202301251120.04
export GOOS=linux
export GOARCH=arm64
export ELASTIC_LAYER_NAME=reakaleek-test 
export ARCHITECTURE=arm64
export VERSION=v0.0.9999
./.ci/publish-aws.sh

Test the GitHub release:

⚠️ But you have to modify .ci/release-github.sh and add the --draft option to the gh release command.

export VERSION=v0.0.9999
./.ci/release-github.sh

Related Issues

• Part of: https://github.com/elastic/observability-robots/issues/1517
• Depends On: Smoketests 2 GH Actions Workflow Migration #370

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-01-24T15:57:55.255+0000

• Duration: 7 min 22 sec

Test stats 🧪

Test	Results
Failed	0
Passed	208
Skipped	2
Total	210

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[apmmachine]

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-01-25T12:53:25.256+0000

• Duration: 4 min 36 sec

Steps errors

Expand to view the steps failures

Run smoke tests

• Took 0 min 16 sec . View more details here
• Description: make smoketest/run TEST_DIR=./tf

Shell Script

• Took 0 min 0 sec . View more details here
• Description: rm /var/lib/jenkins/workspace/aws-lambda-smoketests-mbp_PR-362/src/github.com/elastic/apm-aws-lambda@tmp/aws-credentials.json

[reakaleek]

@elastic/observablt-ci there seems to be a status context naming collision.
Both

• https://apm-ci.elastic.co/job/library/job/apm-aws-lambda-mbp/
• https://apm-ci.elastic.co/job/library/job/apm-aws-lambda-smoketests-mbp/

are reporting the same status check.

[reakaleek]

Migrated to GoReleaser because we don't have ARM runners available in GH Actions workflows by default.

[v1v]

Great work!! A few comments

[kruskall]

issue: I don't think this is right. The dockerfile is only copying the build artifacts to the docker image. There should be no need to change the file time inside the docker image as the build artifacts should have already been built reproducibly.

[reakaleek]

The reason for this way of doing it is a "limitation" of GoReleaser as it copies the artifacts and files used in the Dockerfile to a temporary directory and builds it from there. Hence any touching of the files before running the build is lost.

You can see this behavior when running make dist

[kruskall]

I tried running the command and got the following:

release failed after 1s                  error=docker build failed: failed to build docker.elastic.co/observability/apm-lambda-extension-arm64:1.2.0: exit status 125: unknown flag: --builder

it seems goreleaser is using docker-buildx under the hood. Installing docker-buildx fixed the issues. (should be documented, opened a followup issue: #372)

building the docker image is failing but I think that's related to a typo. I've left a comment below.

[kruskall]

Ok, I see some progress but the build is still failing:

#8 [4/4] RUN find /opt -exec touch -amdt "1675338235" {} ;
#8 0.240 exec /bin/sh: exec format error
#8 ERROR: executor failed running [/bin/sh -c find /opt -exec touch -amdt "${COMMIT_TIMESTAMP}" {} \;]: exit code: 1
------
 > [4/4] RUN find /opt -exec touch -amdt "1675338235" {} ;:
#8 0.240 exec /bin/sh: exec format error
------
ERROR: failed to solve: executor failed running [/bin/sh -c find /opt -exec touch -amdt "${COMMIT_TIMESTAMP}" {} \;]: exit code: 1

[kruskall]

@reakaleek I think this is the last issue. Is the docker build working for you ?

[reakaleek]

Interesting. Yes, it works for me. (Apple macOS M1)

And this workflow shows that it should work with Ubuntu on amd64.

Looking at your error message, if I recall correctly, this could be a buildx setup issue.

[kruskall]

I was able to reproduce the issue by building the arm64 image.

docker buildx build --platform linux/arm64 .

I'm on linux (amd64). I'll investigate more

[kruskall]

Just a minor comment

[kruskall]

LGTM!