Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Codecov Uploader instead of Bash uploader #19

Merged
merged 17 commits into from
Jun 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version: '2'
version: '3.4'
services:
tests:
image: buildkite/plugin-tester
Expand Down
103 changes: 96 additions & 7 deletions hooks/post-command
Original file line number Diff line number Diff line change
@@ -1,6 +1,39 @@
#!/bin/bash
#!/usr/bin/env bash
set -euo pipefail

debug() {
>&2 echo "$*"
}

error() {
debug "$@"
exit 1
}

print_os() {
local OS
OS=$(uname -s)

if [[ "${OS}" = "Linux" ]] ; then
if [[ -f /etc/alpine-release ]] ; then
echo "alpine"
return
fi
echo "linux"
return

elif [[ "${OS}" == "Darwin" ]]; then
echo "macos"
return
fi

echo "${OS}"
}

OS=$(print_os)
CODECOV_VERSION="${BUILDKITE_PLUGIN_CODECOV_UPLOADER_VERSION:-latest}"
TMP_DIR="${BUILDKITE_PLUGIN_CODECOV_TMP_DIR:-/tmp}/codecov-buildkite-plugin/${OS}/${CODECOV_VERSION}"

# Reads a list from plugin config into a global result array
# Returns success if values were read
plugin_read_list_into_result() {
Expand All @@ -11,8 +44,7 @@ plugin_read_list_into_result() {
local parameter="${prefix}_${i}"

if [[ -n "${!prefix:-}" ]] ; then
echo "🚨 Plugin received a string for $prefix, expected an array" >&2
exit 1
error "🚨 Plugin received a string for $prefix, expected an array"
fi

while [[ -n "${!parameter:-}" ]]; do
Expand All @@ -25,9 +57,61 @@ plugin_read_list_into_result() {
[[ ${#result[@]} -gt 0 ]] || return 1
}

get_codecov_uploader() {
case "${OS}" in
alpine|linux|macos)
# We support these
;;
*)
error "Your platform (${OS}) is not supported."
;;
esac

if [[ "${OS}" = "alpine" ]]; then
apk add gnupg
fi

# One-time step
curl \
-fSs \
--retry 5 \
https://keybase.io/codecovsecurity/pgp_keys.asc \
| gpg \
--no-default-keyring \
--keyring trustedkeys.gpg \
--import
local file
for file in codecov codecov.SHA256SUM codecov.SHA256SUM.sig
do
local local_path="${TMP_DIR}/${file}"
if [[ ! -e "${local_path}" ]]; then
debug "Local path will be: ${local_path}"
local remote_source="https://uploader.codecov.io/${CODECOV_VERSION}/${OS}/${file}"
debug "Source is: ${remote_source}"
curl \
-fSs \
--create-dirs \
-o "${local_path}" \
"${remote_source}"
fi
done

(
pushd "${TMP_DIR}" >/dev/null
gpgv codecov.SHA256SUM.sig codecov.SHA256SUM
if [[ "${OS}" = "macos" ]]; then
shasum -a 256 -c codecov.SHA256SUM
else
sha256sum -c codecov.SHA256SUM
fi
chmod +x codecov
popd >/dev/null
)
}

main() {
if [[ "${BUILDKITE_PLUGIN_CODECOV_SKIP_ON_FAIL:-false}" =~ ^(true|on|1)$ ]] && [[ "${BUILDKITE_COMMAND_EXIT_STATUS}" -ne 0 ]]; then
echo "Codecov upload is skipped because step failed with status ${BUILDKITE_COMMAND_EXIT_STATUS}"
debug "Codecov upload is skipped because step failed with status ${BUILDKITE_COMMAND_EXIT_STATUS}"
exit 0
fi

Expand All @@ -47,17 +131,22 @@ main() {
local ci_env
ci_env=$(bash <(curl -s -S --connect-timeout 10 --retry 3 --retry-delay 10 https://codecov.io/env))

local codecov_command="${TMP_DIR}/codecov"
get_codecov_uploader
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that on each run a new uploader is downloaded; always latest currently. I wonder if we can allow a fixed version set by the plugin config and then cache away that particular version. Similar to what is done here: https://github.com/joscha/sauce-connect-buildkite-plugin/blob/master/hooks/pre-command#L30

That way the uploader stays stable longer and does not rely on the network fetch each time nor will cause trouble (read massive amounts of requests) when there are many many builds run from one subnet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 388c47d
I have added two new configs: uploader_version and tmp_dir with proper defaults.
The downloaded binary will be cached accordingly.


set +e
local exit_code
local exit_code
# shellcheck disable=SC2086
docker run \
$ci_env \
--label "com.buildkite.job-id=${BUILDKITE_JOB_ID}" \
--workdir=/workdir \
--volume=$(pwd):/workdir \
--volume="$(pwd)":/workdir \
--volume=/tmp:/tmp \
-it \
--rm \
buildpack-deps:jessie-scm \
bash -c "bash <(curl -s -S --connect-timeout 10 --retry 3 --retry-delay 10 https://codecov.io/bash) ${args[*]:-}"
bash -c "${codecov_command} ${args[*]:-}"
exit_code="$?"
set -e
popd >/dev/null
Expand Down
2 changes: 1 addition & 1 deletion hooks/pre-exit
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -euo pipefail

main() {
Expand Down
6 changes: 6 additions & 0 deletions plugin.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@ author: https://github.com/joscha
requirements:
- bash
- docker
- gpg
joscha marked this conversation as resolved.
Show resolved Hide resolved
- curl
configuration:
properties:
args:
type: array
skip_on_fail:
type: boolean
uploader_version:
type: string
tmp_dir:
type: string
additionalProperties: false
11 changes: 4 additions & 7 deletions tests/command.bats
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,12 @@ setup() {
export BUILDKITE_BUILD_CHECKOUT_PATH=$tmp_dir
export BUILDKITE_JOB_ID=0
export BUILDKITE_COMMAND=my-command
export codecov_command="/tmp/codecov-buildkite-plugin/alpine/latest/codecov"
}

@test "Post-command succeeds" {
cd "$BUILDKITE_BUILD_CHECKOUT_PATH"

stub docker \
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir -it --rm buildpack-deps:jessie-scm bash -c 'bash <(curl -s https://codecov.io/bash) ' : echo Ran Codecov in docker"
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir --volume=/tmp:/tmp -it --rm buildpack-deps:jessie-scm bash -c '${codecov_command} ' : echo Ran Codecov in docker"

run "$post_command_hook"

Expand All @@ -30,12 +29,11 @@ setup() {
}

@test "Post-command succeeds with arguments" {
cd "$BUILDKITE_BUILD_CHECKOUT_PATH"
export BUILDKITE_PLUGIN_CODECOV_ARGS_0="-v"
export BUILDKITE_PLUGIN_CODECOV_ARGS_1="-F my_flag"

stub docker \
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir -it --rm buildpack-deps:jessie-scm bash -c 'bash <(curl -s https://codecov.io/bash) -v -F my_flag' : echo Ran Codecov in docker"
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir --volume=/tmp:/tmp -it --rm buildpack-deps:jessie-scm bash -c '${codecov_command} -v -F my_flag' : echo Ran Codecov in docker"

run "$post_command_hook"

Expand All @@ -44,11 +42,10 @@ setup() {
}

@test "Post-command succeeds with -Z" {
cd "$BUILDKITE_BUILD_CHECKOUT_PATH"
export BUILDKITE_PLUGIN_CODECOV_ARGS_0="-Z"

stub docker \
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir -it --rm buildpack-deps:jessie-scm bash -c 'bash <(curl -s https://codecov.io/bash) -Z' : echo Ran Codecov in docker"
"run -e CODECOV_ENV -e CODECOV_TOKEN -e CODECOV_URL -e CODECOV_SLUG -e VCS_COMMIT_ID -e VCS_BRANCH_NAME -e VCS_PULL_REQUEST -e VCS_SLUG -e VCS_TAG -e CI_BUILD_URL -e CI_BUILD_ID -e CI_JOB_ID --label com.buildkite.job-id=${BUILDKITE_JOB_ID} --workdir=/workdir --volume=${BUILDKITE_BUILD_CHECKOUT_PATH}:/workdir --volume=/tmp:/tmp -it --rm buildpack-deps:jessie-scm bash -c '${codecov_command} -Z' : echo Ran Codecov in docker"

run "$post_command_hook"

Expand Down