mirror of https://github.com/WebAssembly/wasi-sdk
ci: document and script the release process (#295)
Previously, the release process for wasi-sdk was undocumented and manual. The `RELEASING.md` document in this commit describes the steps necessary to publish a new version of `wasi-sdk` as a GitHub release and provides helpful scripts to automate some of the steps. With this in place, a future change could add a workflow trigger that allows running the steps automatically in GitHub actions. Keeping the steps as scripts in the repository, however, allows developers to re-run steps themselves in the (likely) case that something goes awry.pull/307/head
Andrew Brown
3 years ago
committed by
GitHub
parent
7ef7e948fa
commit
22b1d89e64
@ -0,0 +1,48 @@
|
||||
# Release Process
|
||||
|
||||
To publish a new version of `wasi-sdk` as a GitHub release:
|
||||
|
||||
1. Tag a commit with an annotated tag. Note that this must be an annotated tag,
|
||||
not a lightweight tag, so that `version.sh` can use it for calculating the
|
||||
package version (use `git show wasi-sdk-...` to show other tag messages).
|
||||
|
||||
```shell script
|
||||
TAG=wasi-sdk-1
|
||||
git tag -a $TAG
|
||||
git push origin $TAG
|
||||
```
|
||||
|
||||
2. Find a successful workflow that CI has run for the tag. That successful
|
||||
workflow run will have build artifacts that need to be attached to the
|
||||
release. One could search around in the GitHub [actions], but the following
|
||||
script will list completed workflows for a tag (get a token [here][tokens]):
|
||||
|
||||
```shell script
|
||||
ci/get-workflows-for-tag.sh $TAG $GITHUB_TOKEN
|
||||
```
|
||||
|
||||
[actions]: https://github.com/WebAssembly/wasi-sdk/actions
|
||||
[tokens]: https://github.com/settings/tokens
|
||||
|
||||
3. Download and unzip the workflow artifacts. Note that artifacts with `+m` or
|
||||
`.m` suffixes indicate that the Git tree was modified. Expect some duplicates
|
||||
since some of the same artifacts are built on multiple CI runners (e.g.,
|
||||
Windows, MacOS, Linux). The following script does all of this automatically:
|
||||
|
||||
```shell script
|
||||
ci/download-workflow-artifacts.sh $TAG $WORKFLOW_RUN_ID $GITHUB_TOKEN
|
||||
```
|
||||
|
||||
4. Draft a new release. This could be done [manually][releases] but the
|
||||
following script simplifies the uploading of all the files and auto-generates
|
||||
the release description:
|
||||
|
||||
```shell script
|
||||
ci/draft-release.sh $TAG $ARTIFACTS_DIR $GITHUB_TOKEN
|
||||
```
|
||||
|
||||
[releases]: https://github.com/WebAssembly/wasi-sdk/releases
|
||||
|
||||
5. Publish the release; the previous step only creates a draft. Follow the link
|
||||
in the previous step or navigate to the GitHub [releases] to review the
|
||||
description, commit, tag, and assets before clicking "Publish"
|
||||
@ -0,0 +1,102 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# This script downloads and unzips the artifacts produced in a workflow run. It
|
||||
# also checks that the workflow commit corresponds to the tag commit that these
|
||||
# artifacts will be released under. The script has several pre-requisites:
|
||||
# - some standard Bash tools (curl, unzip) and one slightly more rare one (jq)
|
||||
# - an already-created tag in the repository (this marks the code to release)
|
||||
# - the ID of a workflow run that has run successfully--this is where we
|
||||
# retrieve the artifacts from
|
||||
# - a GitHub access token, see https://github.com/settings/tokens
|
||||
#
|
||||
# Usage: download-workflow-artifacts.sh <release tag> <workflow run ID> <token>
|
||||
|
||||
TAG=$1
|
||||
WORKFLOW_RUN_ID=$2
|
||||
GITHUB_TOKEN=$3
|
||||
GITHUB_API_VERSION=2022-11-28
|
||||
GITHUB_API_URL=https://api.github.com/repos/WebAssembly/wasi-sdk
|
||||
TMP_DIR=$(mktemp -d -t wasi-sdk-artifacts.XXXXXXX)
|
||||
|
||||
if [ -z "${TAG}" ] || [ -z "${WORKFLOW_RUN_ID}" ] || [ -z "${GITHUB_TOKEN}" ]; then
|
||||
>&2 echo "Missing parameter; exiting..."
|
||||
>&2 echo "Usage: download-worfklow-artifacts.sh <release tag> <workflow run ID> <token>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get the commit SHA for the passed tag.
|
||||
# See https://docs.github.com/en/rest/commits/commits#get-a-commit
|
||||
MATCHING_COMMIT=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/commits/${TAG}")
|
||||
COMMIT=$(echo $MATCHING_COMMIT | jq -r '.sha')
|
||||
>&2 echo "===== Found commit for tag ${TAG}: ${COMMIT} ====="
|
||||
|
||||
# Check that the commit of the workflow run matches the tag commit and that the
|
||||
# workflow was successful.
|
||||
# See https://docs.github.com/en/rest/actions/workflow-runs#get-a-workflow-run
|
||||
WORKFLOW_RUN=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/actions/runs/${WORKFLOW_RUN_ID}")
|
||||
WORKFLOW_COMMIT=$(echo $WORKFLOW_RUN | jq -r '.head_sha')
|
||||
WORKFLOW_STATUS=$(echo $WORKFLOW_RUN | jq -r '.status')
|
||||
>&2 echo "===== Found commit for workflow ${WORKFLOW_RUN_ID}: ${WORKFLOW_COMMIT} ====="
|
||||
if [ "${COMMIT}" != "${WORKFLOW_COMMIT}" ]; then
|
||||
>&2 echo "Commit at tag ${TAG} did not match the commit for workflow ${WORKFLOW_RUN_ID}, exiting...:"
|
||||
>&2 echo " ${COMMIT} != ${WORKFLOW_COMMIT}"
|
||||
exit 1
|
||||
fi
|
||||
if [ "${WORKFLOW_STATUS}" != "completed" ]; then
|
||||
>&2 echo "Workflow ${WORKFLOW_RUN_ID} did not end successfully, exiting...:"
|
||||
>&2 echo " status = ${WORKFLOW_STATUS}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# List out the artifacts in the given workflow run.
|
||||
# See https://docs.github.com/en/rest/actions/artifacts#list-workflow-run-artifacts
|
||||
ARTIFACTS=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/actions/runs/${WORKFLOW_RUN_ID}/artifacts" \
|
||||
| jq -r '.artifacts[] | [(.id|tostring), .name, .archive_download_url] | join(",")')
|
||||
|
||||
for A in $ARTIFACTS; do
|
||||
ID=$(echo $A | cut -d ',' -f 1)
|
||||
NAME=$(echo $A | cut -d ',' -f 2)
|
||||
URL=$(echo $A | cut -d ',' -f 3)
|
||||
TO=$TMP_DIR/$NAME.zip
|
||||
>&2 echo "===== Downloading: ${TO} ====="
|
||||
|
||||
# Download the artifacts to the temporary directory.
|
||||
# See https://docs.github.com/en/rest/actions/artifacts#download-an-artifact
|
||||
curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
--location --output "${TO}" \
|
||||
"${GITHUB_API_URL}/actions/artifacts/${ID}/zip"
|
||||
done
|
||||
|
||||
# Unzip the workflow artifacts into a `release` directory.
|
||||
pushd $TMP_DIR > /dev/null
|
||||
mkdir release
|
||||
ls -1 *.zip | xargs -n1 unzip -q -o -d release
|
||||
# Some explanation:
|
||||
# -1 prints each file on a separate line
|
||||
# -n1 runs the command once for each item
|
||||
# -q means quietly
|
||||
# -o allows unzip to overwrite existing files (e.g., multiple copies of `libclang_rt.builtins-wasm32-wasi-...`)
|
||||
# -d tells unzip which directory to place things in
|
||||
>&2 echo "===== Files to release: ${TMP_DIR}/release ====="
|
||||
>&2 ls -1 release
|
||||
popd > /dev/null
|
||||
|
||||
>&2 echo
|
||||
>&2 echo "Ensure the above artifacts look correct, then run \`draft-release.sh\` with the following directory:"
|
||||
echo "${TMP_DIR}/release"
|
||||
@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# This script creates a draft pre-release with the artifacts produced in a
|
||||
# workflow run (see `download-workflow-artifacts.sh`). Note that the pre-release
|
||||
# is not published publicly--this is kept as a manual step as a safeguard. The
|
||||
# script has several pre-requisites:
|
||||
# - some standard Bash tools (curl, unzip) and one slightly more rare one (jq)
|
||||
# - an already-created tag in the repository (this marks the code to release)
|
||||
# - a directory containing the artifacts to attach to the release.
|
||||
# - a GitHub access token, see https://github.com/settings/tokens
|
||||
#
|
||||
# Usage: draft-release.sh <release tag> <artifacts dir> <token>
|
||||
|
||||
TAG=$1
|
||||
ARTIFACTS_DIR=$2
|
||||
GITHUB_TOKEN=$3
|
||||
GITHUB_API_VERSION=2022-11-28
|
||||
GITHUB_API_URL=https://api.github.com/repos/WebAssembly/wasi-sdk
|
||||
TMP_DIR=$(mktemp -d -t release.sh.XXXXXXX)
|
||||
|
||||
if [ -z "${TAG}" ] || [ -z "${ARTIFACTS_DIR}" ] || [ -z "${GITHUB_TOKEN}" ]; then
|
||||
>&2 echo "Missing parameter; exiting..."
|
||||
>&2 echo "Usage: draft-release.sh <release tag> <artifacts dir> <token>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get the commit SHA for the passed tag.
|
||||
# See https://docs.github.com/en/rest/commits/commits#get-a-commit
|
||||
MATCHING_COMMIT=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/commits/${TAG}")
|
||||
COMMIT=$(echo $MATCHING_COMMIT | jq -r '.sha')
|
||||
>&2 echo "===== Found commit for tag ${TAG}: ${COMMIT} ====="
|
||||
|
||||
# Create a draft pre-release for this commit.
|
||||
# See https://docs.github.com/en/rest/releases/releases#create-a-release
|
||||
RELEASE_JSON=$(curl \
|
||||
-X POST \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}"\
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/releases" \
|
||||
-d '{"tag_name":"'${TAG}'","target_commitish":"'${COMMIT}'","name":"'${TAG}'","draft":true,"prerelease":true,"generate_release_notes":true}')
|
||||
UPLOAD_URL=$(echo $RELEASE_JSON | jq -r '.upload_url')
|
||||
# Remove the "helpful" but invalid URL suffix that GitHub adds:
|
||||
UPLOAD_URL=${UPLOAD_URL/\{?name,label\}}
|
||||
HTML_URL=$(echo $RELEASE_JSON | jq -r '.html_url')
|
||||
>&2 echo "===== Created draft release: ${HTML_URL} ====="
|
||||
|
||||
# Upload the unzipped artifact files to the release.
|
||||
# See https://docs.github.com/en/rest/releases/assets#upload-a-release-asset
|
||||
for FILE in $(ls "${ARTIFACTS_DIR}"); do
|
||||
FROM=$ARTIFACTS_DIR/$FILE
|
||||
>&2 echo "===== Uploading: ${FROM} ====="
|
||||
UPLOADED=$(curl \
|
||||
-X POST \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}"\
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
"${UPLOAD_URL}?name=${FILE}" \
|
||||
--data-binary "@${FROM}")
|
||||
done
|
||||
|
||||
>&2 echo
|
||||
>&2 echo "===== Completed ====="
|
||||
>&2 echo "This created a draft release, do not forget to manually publish it at:"
|
||||
echo "${HTML_URL}"
|
||||
@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# This script retrieves a list of GitHub workflow runs that have successfully completed for a given
|
||||
# Git tag. The list is printed to stdout (all other output to stderr). It has several
|
||||
# pre-requisites:
|
||||
# - some standard Bash tools (curl) and one slightly more rare one (jq)
|
||||
# - an already-created tag in the repository (this marks the code to release)
|
||||
# - a GitHub access token, see https://github.com/settings/tokens
|
||||
#
|
||||
# Usage: get-workflows-for-tag.sh <tag> <token>
|
||||
|
||||
TAG=$1
|
||||
GITHUB_TOKEN=$2
|
||||
GITHUB_API_VERSION=2022-11-28
|
||||
GITHUB_API_URL=https://api.github.com/repos/WebAssembly/wasi-sdk
|
||||
|
||||
if [ -z "${TAG}" ] || [ -z "${GITHUB_TOKEN}" ]; then
|
||||
>&2 echo "Missing parameter; exiting..."
|
||||
>&2 echo "Usage: get-workflows-for-tag.sh <tag> <token>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get the commit SHA for the passed tag.
|
||||
# See https://docs.github.com/en/rest/commits/commits#get-a-commit
|
||||
MATCHING_COMMIT=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/commits/${TAG}")
|
||||
COMMIT=$(echo $MATCHING_COMMIT | jq -r '.sha')
|
||||
>&2 echo "===== Found commit for tag ${TAG}: ${COMMIT} ====="
|
||||
|
||||
# Find the workflow runs matching the tag commit.
|
||||
# See https://docs.github.com/en/rest/actions/workflow-runs#list-workflow-runs-for-a-repository
|
||||
MATCHING_WORKFLOWS=$(curl \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: ${GITHUB_API_VERSION}" \
|
||||
"${GITHUB_API_URL}/actions/runs?head_sha=${COMMIT}&status=success")
|
||||
WORKFLOW_RUN_IDS=$(echo $MATCHING_WORKFLOWS | jq -r '.workflow_runs[].id')
|
||||
>&2 echo "===== Matching workflow run IDs: ====="
|
||||
echo "$WORKFLOW_RUN_IDS"
|
||||
Loading…
Reference in new issue