msb_14154
/
wasi-sdk
mirror of https://github.com/WebAssembly/wasi-sdk
1
0
Code Issues Projects Releases Wiki Activity

Rewrite the build system with CMake (#429)

Browse Source 
* Rewrite the build system with CMake

This commit is an attempt to provide a concrete path forward on
WebAssembly/wasi-sdk#425. I personally think it's pretty important to
get the ability to have more architectures here but at the same time I
also think it's important to to take this as an opportunity to refactor
and improve the build system of this repository. To that end this
represents my attempt to improve the status quo.

This removes the old `Makefile` and replaces it with a CMake-based
system to build all these projects. Overall this is intended to be a "no
functional change" intended sort of refactoring. Changing build systems
inevitably causes issues, however, so this change additionally has a
very high likelihood of needing follow-up fixes. At a high enough level
this commit introduces two major changes to how this repository is
built:

1. The `make`-based system (the root `Makefile`) is replaced with CMake.
   This additionally updates tests to use CMake.
2. A single "build" is split into either building a toolchain or
   building a sysroot. This enables builds to only build one or the
   other as necessary.

The first change, using CMake, is due to the fact that using `make` on
Windows basically is not pleasant coupled with the fact that more
advanced logic, such as changing flags, compilers, etc, is much easier
with a CMake-based system. The second change is intended to cover the
use case of #425 in addition to refactoring the current build.

Throughout this change I have intentionally not tried to keep a 1:1
correspondance with behaviors in the old `Makefile` because much of this
PR is intended to address shortcomings in the old build system. A list
of changes, improvements, etc, made here are:

* CMake provides a much nicer portability story to Windows than `make`.
  This is moving towards the direction of not needing `bash`, for
  example, to build an SDK. Currently `wasi-libc` still requires this,
  but that's now the only "hard" dependency.

* The set of targets built can now be configured for smaller builds
  and/or debugging just a single target. All WASI targets are still
  built by default but it's much easier to add/remove them.

* Different targets are now able to be built in parallel as opposed to
  the unconditional serial-nature of the `Makefile`.

* Use of `ninja` is no longer required and separate build systems can be
  used if desired.

* The sysroot and the toolchain can now be built with different CMake
  build profiles. For example the `Makefile` hardcoded `MinSizeRel` and
  `RelWithDebInfo` and this can now be much more easily customized by
  the SDK builder.

* Tarballs are now more consistently produced and named. For a tarball
  of the name `foo.tar.gz` it's guaranteed that there's a single folder
  `foo` created when unpacking the tarball.

* The macOS binaries are no longer hybrid x64/arm64 binaries which
  greatly inflates the size of the SDK. There's now a separate build for
  each architecture.

* CI now produces arm64-linux binaries. The sysroot is not built on the
  arm64-linux builder and the sysroot from the x86_64-linux builder is
  used instead.

* Tests are almost ready to execute on Windows, there's just a few minor
  issues related to exit statuses and probably line endings which need
  to be worked out. Will require someone with a Windows checkout, however.

* Tests are now integrated into CMake. This means that the wasm binaries
  are able to be built in parallel and the tests are additionally
  executed in parallel with `ctest`. It is possible to build/run a
  single test. Tests no longer place all of their output in the source
  tree.

* Out-of-tree builds are now possible and the build/installation
  directories can both be customized.

* CI configuration of Windows/macOS/Linux is much more uniform by having
  everything in one build matrix instead of separate matrices.

* Linux builds are exclusively done in docker containers in CI now. CI
  no longer produces two Linux builds only for one to be discarded when
  artifacts are published.

* Windows 32-bit builds are no longer produced in CI since it's expected
  that everyone actually wants the 64-bit ones instead.

* Use of `ccache` is now automatically enabled if it's detected on the
  system.

* Many preexisting shell scripts are now translated to CMake one way or
  another.

* There's no longer a separate build script for how to build wasi-sdk in
  docker and outside of docker which needs to be kept in sync,
  everything funnels through the same script.

* The `docker/Dockerfile` build of wasi-sdk now uses the actual
  toolchain built from CI and additionally doesn't duplicate various
  CMake-based configuration files.

Overall one thing I want to additionally point out is that I'm not CMake
expert. I suspect there's lots of little stylistic and such improvements
that can be made.

* Fix skipping tests on windows

* Fetch a full depth in the finalize workflow too

* Fix multi-arch docker build

* Fix assembling of sysroot

* Fix script syntax

* Clean up the merge script slightly

* Remove Pat's email

* Move configuration of CMAKE_EXECUTABLE_SUFFIX

* Remove redundant sysroot option

* Fix comment in testcase.sh

* Update new p2 cmake files

* Remove now-duplicate wasi-sdk-p2.cmake
pull/437/head
Alex Crichton 5 months ago committed by GitHub
parent 873163a2f0
commit e29a3fef8b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
46 changed files with 1067 additions and 906 deletions
Show Stats Download Patch File Download Diff File

238
.github/workflows/main.yml vendored
Unescape View File

@ -11,37 +11,39 @@ on:
jobs: jobs:
build: build:
name: Native Build name: Build ${{ matrix.artifact }}
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os: include:
- ubuntu-latest - artifact: x86_64-linux
- macos-latest os: ubuntu-latest
- artifact: arm64-linux
os: ubuntu-latest
rust_target: aarch64-unknown-linux-gnu
- artifact: arm64-macos
os: macos-latest
llvm_cmake_flags: -DCMAKE_OSX_DEPLOYMENT_TARGET=10.12 -DCMAKE_OSX_ARCHITECTURES=arm64
rust_target: aarch64-apple-darwin
- artifact: x86_64-macos
os: macos-latest
llvm_cmake_flags: -DCMAKE_OSX_DEPLOYMENT_TARGET=10.12 -DCMAKE_OSX_ARCHITECTURES=x86_64
rust_target: x86_64-apple-darwin
skip_sysroot: 1
- artifact: x86_64-windows
os: windows-latest
# TODO: tests are pretty close to passing on Windows but need some
# final tweaks, namely testing the exit code doesn't work since
# exit codes are different on Windows and the `mmap.c` tests seems
# to have issues probably with line endings. Needs someone with a
# Windows checkout tot test further.
skip_tests: 1
steps: steps:
- uses: actions/cache@v4
with:
path: ~/.cache/ccache
# Bump the prefix number to evict all previous caches and
# enforce a clean build, in the unlikely case that some
# weird build error occur and ccache becomes a potential
# suspect.
key: 0-cache-ubuntu-latest-${{ github.run_id }}
restore-keys: |
0-cache-ubuntu-latest
if: matrix.os == 'ubuntu-latest'
- uses: actions/cache@v4
with:
path: ~/Library/Caches/ccache
key: 0-cache-macos-latest-${{ github.run_id }}
restore-keys: |
0-cache-macos-latest
if: matrix.os == 'macos-latest'
- name: Setup `wasmtime` for tests
uses: bytecodealliance/actions/wasmtime/setup@v1
with:
version: "18.0.2"
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
@ -53,108 +55,130 @@ jobs:
# Server does not allow request for unadvertised object" in the # Server does not allow request for unadvertised object" in the
# future. # future.
- run: git submodule update --init --depth 32 --jobs 3 - run: git submodule update --init --depth 32 --jobs 3
# Persist ccache-based caches across builds. This directory is configured
# via the CCACHE_DIR env var below for ccache to use.
#
# Bump the prefix number to evict all previous caches and enforce a clean
# build, in the unlikely case that some weird build error occur and ccache
# becomes a potential suspect.
- uses: actions/cache@v4
id: cache-restore
with:
path: ${{ runner.tool_cache }}/ccache
key: 0-cache-${{ matrix.artifact }}-${{ github.run_id }}
restore-keys: |
0-cache-${{ matrix.artifact }}-
- run: |
mkdir -p '${{ runner.tool_cache }}/ccache'
echo 'CCACHE_DIR=${{ runner.tool_cache }}/ccache' >> $GITHUB_ENV
shell: bash
# Configure CMake flags for `ci/build.sh` as necessary for each
# matrix entry.
- run: echo WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS=${{ matrix.llvm_cmake_flags }} >> $GITHUB_ENV
if: matrix.llvm_cmake_flags != ''
shell: bash
- run: |
cmake_args=-DWASI_SDK_ARTIFACT=${{ matrix.artifact }}
if [ "${{ matrix.rust_target }}" != "" ]; then
rustup target add ${{ matrix.rust_target }}
cmake_args="$cmake_args -DRUST_TARGET=${{ matrix.rust_target }}"
fi
echo WASI_SDK_CI_TOOLCHAIN_CMAKE_ARGS="$cmake_args" >> $GITHUB_ENV
shell: bash
- run: echo WASI_SDK_CI_SKIP_SYSROOT=1 >> $GITHUB_ENV
shell: bash
if: matrix.skip_sysroot != ''
- run: echo WASI_SDK_CI_SKIP_TESTS=1 >> $GITHUB_ENV
shell: bash
if: matrix.skip_tests != ''
# Add some extra installed software on each runner as necessary.
- name: Setup `wasmtime` for tests
uses: bytecodealliance/actions/wasmtime/setup@v1
with:
version: "18.0.2"
- name: Install ccache, ninja (macOS) - name: Install ccache, ninja (macOS)
run: brew install ccache ninja run: brew install ccache ninja
if: matrix.os == 'macos-latest' if: runner.os == 'macOS'
- name: Install ccache, ninja (Windows)
run: choco install ccache ninja
if: runner.os == 'Windows'
- name: Install ccache, ninja (Linux) - name: Install ccache, ninja (Linux)
run: sudo apt install ccache ninja-build run: sudo apt install ccache
if: matrix.os == 'ubuntu-latest' if: runner.os == 'Linux'
- name: Build
run: NINJA_FLAGS=-v make package LLVM_CMAKE_FLAGS=-DLLVM_CCACHE_BUILD=ON - name: Build and test (macOS)
run: ./ci/build.sh
if: runner.os == 'macOS'
- name: Build and test (Linux)
run: ./ci/docker-build.sh ${{ matrix.artifact }}
if: runner.os == 'Linux'
# Use a shorter build directory than the default on Windows to avoid
# hitting path length and command line length limits.
- name: Build and test (Windows)
run: ./ci/build.sh C:/wasi-sdk
shell: bash shell: bash
- name: Run the testsuite if: runner.os == 'Windows'
run: NINJA_FLAGS=-v make check RUNTIME=wasmtime
# Upload the `dist` folder from the build as the artifacts for this
# runner.
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
# Upload the dist folder. Give it a name according to the OS it was built for. name: ${{ format( 'dist-{0}', matrix.artifact) }}
name: ${{ format( 'dist-{0}', matrix.os) }} path: build/dist
path: dist
winbuild: # Help debug ccache issues by showing what happened.
name: Windows Build - if: always()
runs-on: windows-latest name: Show ccache statistics
strategy: run: ccache --show-stats
fail-fast: false
matrix: # Always save a cache, even if the build failed. This ensures that if
include: # live-debugging via CI the build gets to pick up where it left off last
- arch: x64 # time instead of having to recreate everything each time a failure
sys: clang64 # happens.
env: clang-x86_64 - if: always() && steps.cache-restore.outputs.cache-hit != 'true'
- arch: x86 uses: actions/cache/save@v4
sys: clang32
env: clang-i686
steps:
- uses: actions/cache@v4
with:
path: ~/AppData/Local/ccache
key: 0-${{ format( 'cache-windows-latest-{0}', matrix.arch) }}-${{ github.run_id }}
restore-keys: |
0-${{ format( 'cache-windows-latest-{0}', matrix.arch) }}
- uses: msys2/setup-msys2@v2
with:
install: >-
base-devel
git
mingw-w64-${{ matrix.env }}-ccache
mingw-w64-${{ matrix.env }}-cmake
mingw-w64-${{ matrix.env }}-ninja
mingw-w64-${{ matrix.env }}-toolchain
msystem: ${{ matrix.sys }}
update: true
release: false
path-type: inherit
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: git fetch --tags --force
name: Force-fetch tags to work around actions/checkout#290
- run: git submodule update --init --depth 32 --jobs 3
- name: Build
shell: msys2 {0}
run: |
make package LLVM_CMAKE_FLAGS=-DLLVM_CCACHE_BUILD=ON
make check
- name: Does it work sans msys2?
run: |
C:\wasi-sdk\bin\clang.exe --version
C:\wasi-sdk\bin\llvm-ar.exe --version
C:\wasi-sdk\bin\wasm-ld.exe --version
- name: Upload artifacts
uses: actions/upload-artifact@v4
with: with:
# Upload the dist folder. Give it a name according to the OS it was built for. path: ${{ runner.tool_cache }}/ccache
name: ${{ format( 'dist-windows-latest-{0}', matrix.arch) }} key: 0-cache-${{ matrix.artifact }}-${{ github.run_id }}
path: dist
dockerbuild: # Once all of the above matrix entries have completed this job will run and
name: Docker Build # assemble the final `wasi-sdk-*` artifacts by fusing the toolchain/sysroot
# artifacts.
finalize:
name: Finalize wasi-sdk artifacts
needs: build
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/cache@v4
with:
path: ~/.ccache
key: 0-cache-ubuntu-bionic-${{ github.run_id }}
restore-keys: |
0-cache-ubuntu-bionic
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- run: git fetch --tags --force - run: git fetch --tags --force
name: Force-fetch tags to work around actions/checkout#290 name: Force-fetch tags to work around actions/checkout#290
- run: git submodule update --init --depth 32 --jobs 3 # Download all artifacts from all platforms in `build`, merge them into
# final wasi-sdk-* artifacts, and then upload them.
- uses: actions/download-artifact@v4
- run: ./ci/merge-artifacts.sh
- uses: actions/upload-artifact@v4
with:
name: release-artifacts
path: dist
# Use the `wasi-sdk-*` artifacts just created to create a docker image
# with a toolchain pre-installed.
- uses: docker/login-action@v2 - uses: docker/login-action@v2
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/setup-qemu-action@v2 - uses: docker/setup-qemu-action@v2
- uses: docker/setup-buildx-action@v2 - uses: docker/setup-buildx-action@v2
- uses: docker/metadata-action@v4 - uses: docker/metadata-action@v4
id: meta id: meta
with: with:
@ -165,16 +189,6 @@ jobs:
type=ref,event=tag type=ref,event=tag
type=ref,event=pr type=ref,event=pr
type=sha type=sha
- name: Run docker_build script
run: ./docker_build.sh
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
# Upload the dist folder. Give it a name according to the OS it was built for.
name: dist-ubuntu-bionic
path: dist
- name: Build and push wasi-sdk docker image - name: Build and push wasi-sdk docker image
uses: docker/build-push-action@v3 uses: docker/build-push-action@v3
with: with:

45
CMakeLists.txt
Unescape View File

@ -0,0 +1,45 @@
# Build logic for building both a toolchain and a sysroot for WASI.
#
# This top level `CMakeLists.txt` file can be used either to build a clang
# toolchain or a WASI sysroot. Note that this can't be done at the same time.
# A toolchain build requires a compiler for the target architecture. A
# WASI sysroot build requires this previous compiler and must be runnable on
# the host.
cmake_minimum_required(VERSION 3.26)
project(wasi-sdk)
include(ExternalProject)
set(WASI_SDK_TARGETS "wasm32-wasi;wasm32-wasip1;wasm32-wasip2;wasm32-wasip1-threads;wasm32-wasi-threads"
CACHE STRING "List of WASI targets to build")
option(WASI_SDK_BUILD_TOOLCHAIN "Build a toolchain instead of the sysroot" OFF)
set(llvm_proj_dir ${CMAKE_CURRENT_SOURCE_DIR}/src/llvm-project)
set(wasi_libc ${CMAKE_CURRENT_SOURCE_DIR}/src/wasi-libc)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
include(wasi-sdk-enable-ccache)
find_program(PYTHON python3 python REQUIRED)
# Set some variables based on the `version.py` script
set(version_script ${CMAKE_CURRENT_SOURCE_DIR}/version.py)
execute_process(
COMMAND ${PYTHON} ${version_script} llvm-major --llvm-dir=${llvm_proj_dir}
OUTPUT_VARIABLE clang_version
OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(
COMMAND ${PYTHON} ${version_script}
OUTPUT_VARIABLE wasi_sdk_version
OUTPUT_STRIP_TRAILING_WHITESPACE)
message(STATUS "wasi-sdk toolchain LLVM version is ${clang_version}")
message(STATUS "wasi-sdk version is ${wasi_sdk_version}")
# Only include one version of the build logic as pulling in both isn't
# supported at this time.
if(WASI_SDK_BUILD_TOOLCHAIN)
include(wasi-sdk-toolchain)
else()
include(wasi-sdk-sysroot)
endif()

39
Dockerfile
Unescape View File

@ -1,39 +0,0 @@
# Use a relatively old/stable distro here to maximize the supported platforms
# and avoid depending on more recent version of, say, libc.
# Here we choose Bionic 18.04.
FROM ubuntu:bionic
# We want to use the same UID/GID of the external user to avoid permission
# issues. See the user setup at the end of the file.
ARG UID=1000
ARG GID=1000
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ccache \
curl \
ca-certificates \
build-essential \
clang \
python3 \
git \
ninja-build \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN curl -sSLO https://github.com/Kitware/CMake/releases/download/v3.25.1/cmake-3.25.1-linux-x86_64.tar.gz \
&& tar xf cmake-3.25.1-linux-x86_64.tar.gz \
&& rm cmake-3.25.1-linux-x86_64.tar.gz \
&& mkdir -p /opt \
&& mv cmake-3.25.1-linux-x86_64 /opt/cmake
ENV PATH /opt/cmake/bin:$PATH
ENV RUSTUP_HOME=/rust/rustup CARGO_HOME=/rust/cargo PATH=$PATH:/rust/cargo/bin
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --profile=minimal && \
chmod -R a+w /rust
RUN groupadd -g ${GID} builder && \
useradd --create-home --uid ${UID} --gid ${GID} builder
USER builder
WORKDIR /workspace

291
Makefile
Unescape View File

@ -1,291 +0,0 @@
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
ROOT_DIR=${CURDIR}
LLVM_PROJ_DIR?=$(ROOT_DIR)/src/llvm-project
VERSION_SCRIPT=python3 ./version.py
# Windows needs munging
ifeq ($(OS),Windows_NT)
PREFIX?=c:/wasi-sdk
# we need to explicitly call bash -c for makefile $(shell ...), otherwise we'll try under
# who knows what
BASH=bash -c
ifeq (x$(MSYSTEM),x)
$(error On Windows, this Makefile only works in MSYS2 environments such as git-bash.)
endif
# msys needs any /-prefixed arguments, or =/ containing, to turn into //
# because it tries to path-expand the / into the msys root. // escapes this.
ESCAPE_SLASH=/
BUILD_PREFIX=$(PREFIX)
# assuming we're running under msys2 (git-bash), PATH needs /c/foo format directories (because
# it itself is :-delimited)
PATH_PREFIX=$(shell cygpath.exe -u $(BUILD_PREFIX))
else
PREFIX?=/opt/wasi-sdk
DESTDIR=$(abspath build/install)
BUILD_PREFIX=$(DESTDIR)$(PREFIX)
PATH_PREFIX=$(BUILD_PREFIX)
ESCAPE_SLASH?=
BASH=
endif
ifeq ($(shell uname),Darwin)
override LLVM_CMAKE_FLAGS += -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" \
-DCMAKE_OSX_DEPLOYMENT_TARGET=10.12
endif
TARGETS = wasm32-wasi wasm32-wasip1 wasm32-wasip2 wasm32-wasip1-threads wasm32-wasi-threads
# Only the major version is needed for Clang, see https://reviews.llvm.org/D125860.
CLANG_VERSION=$(shell $(VERSION_SCRIPT) llvm-major --llvm-dir=$(LLVM_PROJ_DIR))
VERSION:=$(shell $(VERSION_SCRIPT))
DEBUG_PREFIX_MAP=-fdebug-prefix-map=$(ROOT_DIR)=wasisdk://v$(VERSION)
# Generate debuginfo by default for wasi-sdk since it's easily strippable and
# otherwise quite useful for debugging.
WASI_SDK_CFLAGS := $(DEBUG_PREFIX_MAP) -g
WASI_SDK_CXXFLAGS := $(WASI_SDK_CFLAGS)
default: build
@echo "Use -fdebug-prefix-map=$(ROOT_DIR)=wasisdk://v$(VERSION)"
check:
TARGETS="$(TARGETS)" tests/run.sh "$(BUILD_PREFIX)" "$(RUNTIME)"
clean:
rm -rf build $(DESTDIR)
# Default symlinks that clang creates to the `clang` executable
CLANG_LINKS_TO_CREATE = clang++ clang-cl clang-cpp
# Add target-prefixed versions of `clang` and `clang++` so they can be used
# without `--target` as it's auto-inferred from the executable name by clang.
CLANG_LINKS_TO_CREATE += $(foreach target,$(TARGETS),$(target)-clang)
CLANG_LINKS_TO_CREATE += $(foreach target,$(TARGETS),$(target)-clang++)
# Small helper to create a `join-with` function that can join elements of a
# list with a defined separator.
noop =
space = $(noop) $(noop)
join-with = $(subst $(space),$1,$(strip $2))
build/llvm.BUILT:
mkdir -p build/llvm
cd build/llvm && cmake -G Ninja \
-DCLANG_LINKS_TO_CREATE="$(call join-with,;,$(CLANG_LINKS_TO_CREATE))" \
-DCMAKE_BUILD_TYPE=MinSizeRel \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
-DLLVM_STATIC_LINK_CXX_STDLIB=ON \
-DCMAKE_INSTALL_PREFIX=$(PREFIX) \
-DLLVM_INCLUDE_TESTS=OFF \
-DLLVM_INCLUDE_UTILS=OFF \
-DLLVM_INCLUDE_BENCHMARKS=OFF \
-DLLVM_INCLUDE_EXAMPLES=OFF \
-DLLVM_TARGETS_TO_BUILD=WebAssembly \
-DLLVM_DEFAULT_TARGET_TRIPLE=wasm32-wasi \
-DLLVM_ENABLE_PROJECTS="lld;clang;clang-tools-extra" \
$(if $(patsubst 9,,$(CLANG_VERSION)), \
$(if $(patsubst 10,,$(CLANG_VERSION)), \
-DDEFAULT_SYSROOT=../share/wasi-sysroot, \
-DDEFAULT_SYSROOT=$(PREFIX)/share/wasi-sysroot), \
-DDEFAULT_SYSROOT=$(PREFIX)/share/wasi-sysroot) \
-DLLVM_INSTALL_BINUTILS_SYMLINKS=TRUE \
-DLLVM_ENABLE_LIBXML2=OFF \
$(LLVM_CMAKE_FLAGS) \
$(LLVM_PROJ_DIR)/llvm
DESTDIR=$(DESTDIR) ninja $(NINJA_FLAGS) -C build/llvm \
install-clang \
install-clang-format \
install-clang-tidy \
install-clang-apply-replacements \
install-lld \
install-llvm-mc \
install-llvm-ranlib \
install-llvm-strip \
install-llvm-dwarfdump \
install-clang-resource-headers \
install-ar \
install-ranlib \
install-strip \
install-nm \
install-size \
install-strings \
install-objdump \
install-objcopy \
install-c++filt \
llvm-config
touch build/llvm.BUILT
# Build the `wasm-component-ld` linker from source via `cargo install`. This is
# used for the `wasm32-wasip2` target natively by Clang. Note that `--root`
# passed to `cargo install` will place it in the output directory automatically.
build/wasm-component-ld.BUILT: build/llvm.BUILT
cargo install --no-track wasm-component-ld@0.5.0 --root $(BUILD_PREFIX)
touch build/wasm-component-ld.BUILT
# Flags for running `make` in wasi-libc
# $(1): the target that's being built
WASI_LIBC_MAKEFLAGS = \
-C $(ROOT_DIR)/src/wasi-libc \
CC=$(BUILD_PREFIX)/bin/clang \
AR=$(BUILD_PREFIX)/bin/llvm-ar \
NM=$(BUILD_PREFIX)/bin/llvm-nm \
SYSROOT=$(BUILD_PREFIX)/share/wasi-sysroot \
EXTRA_CFLAGS="$(WASI_SDK_CFLAGS) -O2 -DNDEBUG" \
TARGET_TRIPLE=$(1)
build/wasi-libc.BUILT: build/compiler-rt.BUILT build/wasm-component-ld.BUILT
$(MAKE) $(call WASI_LIBC_MAKEFLAGS,wasm32-wasi) default libc_so
$(MAKE) $(call WASI_LIBC_MAKEFLAGS,wasm32-wasip1) default libc_so
$(MAKE) $(call WASI_LIBC_MAKEFLAGS,wasm32-wasip2) WASI_SNAPSHOT=p2 default libc_so
$(MAKE) $(call WASI_LIBC_MAKEFLAGS,wasm32-wasi-threads) THREAD_MODEL=posix
$(MAKE) $(call WASI_LIBC_MAKEFLAGS,wasm32-wasip1-threads) THREAD_MODEL=posix
touch build/wasi-libc.BUILT
build/compiler-rt.BUILT: build/llvm.BUILT
# Do the build, and install it.
mkdir -p build/compiler-rt
cd build/compiler-rt && cmake -G Ninja \
-DCMAKE_SYSROOT=$(BUILD_PREFIX)/share/wasi-sysroot \
-DCMAKE_C_COMPILER_WORKS=ON \
-DCMAKE_CXX_COMPILER_WORKS=ON \
-DCMAKE_AR=$(BUILD_PREFIX)/bin/ar \
-DCMAKE_MODULE_PATH=$(ROOT_DIR)/cmake \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_TOOLCHAIN_FILE=$(ROOT_DIR)/wasi-sdk.cmake \
-DCOMPILER_RT_BAREMETAL_BUILD=On \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_INCLUDE_TESTS=OFF \
-DCOMPILER_RT_HAS_FPIC_FLAG=OFF \
-DCOMPILER_RT_ENABLE_IOS=OFF \
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=On \
-DWASI_SDK_PREFIX=$(BUILD_PREFIX) \
-DCMAKE_C_FLAGS="$(WASI_SDK_CFLAGS)" \
-DLLVM_CONFIG_PATH=$(ROOT_DIR)/build/llvm/bin/llvm-config \
-DCOMPILER_RT_OS_DIR=wasi \
-DCMAKE_INSTALL_PREFIX=$(PREFIX)/lib/clang/$(CLANG_VERSION)/ \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
$(LLVM_PROJ_DIR)/compiler-rt/lib/builtins
DESTDIR=$(DESTDIR) ninja $(NINJA_FLAGS) -C build/compiler-rt install
# Install clang-provided headers.
cp -R $(ROOT_DIR)/build/llvm/lib/clang $(BUILD_PREFIX)/lib/
cp -R $(BUILD_PREFIX)/lib/clang/$(CLANG_VERSION)/lib/wasi $(BUILD_PREFIX)/lib/clang/$(CLANG_VERSION)/lib/wasip1
cp -R $(BUILD_PREFIX)/lib/clang/$(CLANG_VERSION)/lib/wasi $(BUILD_PREFIX)/lib/clang/$(CLANG_VERSION)/lib/wasip2
touch build/compiler-rt.BUILT
# Flags for libcxx and libcxxabi.
# $(1): pthreads ON or OFF
# $(2): shared libraries ON or OFF
# $(3): the name of the target being built for
# $(4): extra compiler flags to pass
LIBCXX_CMAKE_FLAGS = \
-DCMAKE_C_COMPILER_WORKS=ON \
-DCMAKE_CXX_COMPILER_WORKS=ON \
-DCMAKE_AR=$(BUILD_PREFIX)/bin/ar \
-DCMAKE_MODULE_PATH=$(ROOT_DIR)/cmake \
-DCMAKE_TOOLCHAIN_FILE=$(ROOT_DIR)/wasi-sdk.cmake \
-DCMAKE_STAGING_PREFIX=$(PREFIX)/share/wasi-sysroot \
-DCMAKE_POSITION_INDEPENDENT_CODE=$(2) \
-DLLVM_CONFIG_PATH=$(ROOT_DIR)/build/llvm/bin/llvm-config \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
-DCXX_SUPPORTS_CXX11=ON \
-DLIBCXX_ENABLE_THREADS:BOOL=$(1) \
-DLIBCXX_HAS_PTHREAD_API:BOOL=$(1) \
-DLIBCXX_HAS_EXTERNAL_THREAD_API:BOOL=OFF \
-DLIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY:BOOL=OFF \
-DLIBCXX_HAS_WIN32_THREAD_API:BOOL=OFF \
-DLLVM_COMPILER_CHECKED=ON \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DLIBCXX_ENABLE_SHARED:BOOL=$(2) \
-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY:BOOL=OFF \
-DLIBCXX_ENABLE_EXCEPTIONS:BOOL=OFF \
-DLIBCXX_ENABLE_FILESYSTEM:BOOL=ON \
-DLIBCXX_ENABLE_ABI_LINKER_SCRIPT:BOOL=OFF \
-DLIBCXX_CXX_ABI=libcxxabi \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS=$(LLVM_PROJ_DIR)/libcxxabi/include \
-DLIBCXX_HAS_MUSL_LIBC:BOOL=ON \
-DLIBCXX_ABI_VERSION=2 \
-DLIBCXXABI_ENABLE_EXCEPTIONS:BOOL=OFF \
-DLIBCXXABI_ENABLE_SHARED:BOOL=$(2) \
-DLIBCXXABI_SILENT_TERMINATE:BOOL=ON \
-DLIBCXXABI_ENABLE_THREADS:BOOL=$(1) \
-DLIBCXXABI_HAS_PTHREAD_API:BOOL=$(1) \
-DLIBCXXABI_HAS_EXTERNAL_THREAD_API:BOOL=OFF \
-DLIBCXXABI_BUILD_EXTERNAL_THREAD_LIBRARY:BOOL=OFF \
-DLIBCXXABI_HAS_WIN32_THREAD_API:BOOL=OFF \
-DLIBCXXABI_ENABLE_PIC:BOOL=$(2) \
-DLIBCXXABI_USE_LLVM_UNWINDER:BOOL=OFF \
-DWASI_SDK_PREFIX=$(BUILD_PREFIX) \
-DUNIX:BOOL=ON \
--debug-trycompile \
-DCMAKE_SYSROOT=$(BUILD_PREFIX)/share/wasi-sysroot \
-DCMAKE_C_FLAGS="$(WASI_SDK_CFLAGS) $(EXTRA_CFLAGS) $(4) --target=$(3)" \
-DCMAKE_CXX_FLAGS="$(WASI_SDK_CXXFLAGS) $(EXTRA_CXXFLAGS) $(4) --target=$(3)" \
-DLIBCXX_LIBDIR_SUFFIX=$(ESCAPE_SLASH)/$(3) \
-DLIBCXXABI_LIBDIR_SUFFIX=$(ESCAPE_SLASH)/$(3) \
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" \
$(LLVM_PROJ_DIR)/runtimes
# Rules to build libcxx, factored out here to deduplicate the below
# $(1): pthreads ON or OFF
# $(2): shared libraries ON or OFF
# $(3): the name of the target being built for
define BUILD_LIBCXX
mkdir -p build/libcxx-$(3)
cd build/libcxx-$(3) && cmake -G Ninja $(call LIBCXX_CMAKE_FLAGS,$(1),$(2),$(3),$(4))
ninja $(NINJA_FLAGS) -C build/libcxx-$(3)
DESTDIR=$(DESTDIR) ninja $(NINJA_FLAGS) -C build/libcxx-$(3) install
rm -rf $(BUILD_PREFIX)/share/wasi-sysroot/include/$(3)/c++
mv $(BUILD_PREFIX)/share/wasi-sysroot/include/c++ $(BUILD_PREFIX)/share/wasi-sysroot/include/$(3)/
endef
build/libcxx.BUILT: build/llvm.BUILT build/wasi-libc.BUILT
$(call BUILD_LIBCXX,OFF,ON,wasm32-wasi)
$(call BUILD_LIBCXX,OFF,ON,wasm32-wasip1)
$(call BUILD_LIBCXX,OFF,ON,wasm32-wasip2)
$(call BUILD_LIBCXX,ON,OFF,wasm32-wasi-threads,-pthread)
$(call BUILD_LIBCXX,ON,OFF,wasm32-wasip1-threads,-pthread)
# As of this writing, `clang++` will ignore the above include dirs unless this one also exists:
mkdir -p $(BUILD_PREFIX)/share/wasi-sysroot/include/c++/v1
touch build/libcxx.BUILT
build/config.BUILT:
mkdir -p $(BUILD_PREFIX)/share/misc
cp src/config/config.sub src/config/config.guess $(BUILD_PREFIX)/share/misc
mkdir -p $(BUILD_PREFIX)/share/cmake/Platform
cp wasi-sdk.cmake $(BUILD_PREFIX)/share/cmake
cp wasi-sdk-pthread.cmake $(BUILD_PREFIX)/share/cmake
cp wasi-sdk-p2.cmake $(BUILD_PREFIX)/share/cmake
cp cmake/Platform/WASI.cmake $(BUILD_PREFIX)/share/cmake/Platform
touch build/config.BUILT
build/version.BUILT:
$(VERSION_SCRIPT) dump > $(BUILD_PREFIX)/VERSION
touch build/version.BUILT
build: build/llvm.BUILT build/wasi-libc.BUILT build/compiler-rt.BUILT build/libcxx.BUILT build/config.BUILT build/version.BUILT
strip: build/llvm.BUILT
./strip_symbols.sh $(BUILD_PREFIX)/bin
package: build/package.BUILT
build/package.BUILT: build strip
mkdir -p dist
./deb_from_installation.sh $(shell pwd)/dist "$(VERSION)" "$(BUILD_PREFIX)"
./tar_from_installation.sh "$(shell pwd)/dist" "$(VERSION)" "$(BUILD_PREFIX)"
touch build/package.BUILT
.PHONY: default clean build strip package check

3
RELEASING.md
Unescape View File

@ -1,5 +1,8 @@
# Release Process # Release Process
> **Note**: These instructions are out-of-date given the latest refactoring of
> the build system and should get updated before the next release.
To publish a new version of `wasi-sdk` as a GitHub release: To publish a new version of `wasi-sdk` as a GitHub release:
1. Tag a commit with an annotated tag. Note that this must be an annotated tag, 1. Tag a commit with an annotated tag. Note that this must be an annotated tag,

49
ci/build.sh
Unescape View File

@ -0,0 +1,49 @@
#!/bin/bash
# Build logic executed in CI. This is intentionally kept relatively minimal to
# one day not live in bash to have a bash-less build on Windows. For now though
# this will unconditionally build a toolchain and then optionally build a
# sysroot. Builders which can't actually execute the toolchain they produce
# skip the sysroot step below.
set -ex
# Optionally allow the first argument to this script to be the install
# location.
if [ "$1" = "" ]; then
install_dir=`pwd`/build/install
else
install_dir="$1"
fi
cmake -G Ninja -B build/toolchain -S . \
-DWASI_SDK_BUILD_TOOLCHAIN=ON \
"-DCMAKE_INSTALL_PREFIX=$install_dir" \
$WASI_SDK_CI_TOOLCHAIN_CMAKE_ARGS \
"-DLLVM_CMAKE_FLAGS=$WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS"
ninja -C build/toolchain install dist -v
mv build/toolchain/dist build/dist
if [ "$WASI_SDK_CI_SKIP_SYSROOT" = "1" ]; then
exit 0
fi
# Use the just-built toolchain and its `CMAKE_TOOLCHAIN_FILE` to build a
# sysroot.
cmake -G Ninja -B build/sysroot -S . \
"-DCMAKE_TOOLCHAIN_FILE=$install_dir/share/cmake/wasi-sdk.cmake" \
-DCMAKE_C_COMPILER_WORKS=ON \
-DCMAKE_CXX_COMPILER_WORKS=ON \
-DWASI_SDK_INCLUDE_TESTS=ON \
"-DCMAKE_INSTALL_PREFIX=$install_dir"
ninja -C build/sysroot install dist -v
mv build/sysroot/dist/* build/dist
if [ "$WASI_SDK_CI_SKIP_TESTS" = "1" ]; then
exit 0
fi
# Run tests to ensure that the sysroot works.
ctest --output-on-failure --parallel 10 --test-dir build/sysroot/tests

58
ci/docker-build.sh
Unescape View File

@ -0,0 +1,58 @@
#!/bin/sh
# This is a helper script invoked from CI which will execute the `ci/build.sh`
# script within a docker container. This builds `ci/docker/Dockerfile.common`
# along with the specified `ci/docker/Dockerfile.$x` from the command line.
# This container is then used to execute `ci/build.sh`.
set -e
if [ "$1" = "" ]; then
echo "Usage: $0 <image>"
echo ""
echo "example: $0 x86_64-linux"
exit 1
fi
set -x
# Build the base image which the image below can used.
docker build \
--file ci/docker/Dockerfile.common \
--tag wasi-sdk-builder-base \
ci/docker
# Build the container that is going to be used
docker build \
--file ci/docker/Dockerfile.$1 \
--tag wasi-sdk-builder \
ci/docker
# Perform the build in `/src`. The current directory is mounted read-write at
# this location as well. To ensure that container-created files are reasonable
# on the host as well the `--user` is passed to configure various permissions.
args="--workdir /src --volume `pwd`:/src:Z"
args="$args --user $(id -u):$(id -g)"
# Persist the ccache directory on the host to ensure repeated runs/debugging
# of this container don't take forever. Also enables caching in CI.
ccache_dir=$CCACHE_DIR
if [ "$ccache_dir" = "" ]; then
ccache_dir=$HOME/.ccache
fi
args="$args --volume $ccache_dir:/ccache:Z --env CCACHE_DIR=/ccache"
# Inherit some tools from the host into this container. This ensures that the
# decision made on CI of what versions to use is the canonical source of truth
# for theset ools
args="$args --volume `rustc --print sysroot`:/rustc:ro"
args="$args --volume $(dirname $(which wasmtime)):/wasmtime:ro"
# Before running `ci/build.sh` set up some rust/PATH related info to use what
# was just mounted above, and then execute the build.
docker run \
$args \
--tty \
--init \
wasi-sdk-builder \
bash -c 'CARGO_HOME=/tmp/cargo-home PATH=$PATH:/rustc/bin:/wasmtime exec ci/build.sh'

23
ci/docker/Dockerfile.arm64-linux
Unescape View File

@ -0,0 +1,23 @@
FROM wasi-sdk-builder-base
RUN apt-get install -y g++-aarch64-linux-gnu
# Don't build a sysroot for this cross-compiled target since it would require a
# host compiler and the sysroot is otherwise already built on other CI builders.
ENV WASI_SDK_CI_SKIP_SYSROOT 1
ENV WASI_SDK_CI_TOOLCHAIN_LLVM_CMAKE_ARGS \
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ \
-DCMAKE_CROSSCOMPILING=True \
-DCMAKE_CXX_FLAGS=-march=armv8-a \
-DCMAKE_SYSTEM_PROCESSOR=arm64 \
-DCMAKE_SYSTEM_NAME=Linux \
-DLLVM_HOST_TRIPLE=aarch64-linux-gnu \
-DRUST_TARGET=aarch64-unknown-linux-gnu
ENV WASI_SDK_CI_TOOLCHAIN_CMAKE_ARGS \
-DWASI_SDK_ARTIFACT=arm64-linux \
-DRUST_TARGET=aarch64-unknown-linux-gnu
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER aarch64-linux-gnu-gcc

32
ci/docker/Dockerfile.common
Unescape View File

@ -0,0 +1,32 @@
# Use a relatively old/stable distro here to maximize the supported platforms
# and avoid depending on more recent version of, say, libc.
# Here we choose Bionic 18.04.
FROM ubuntu:18.04
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ccache \
curl \
ca-certificates \
build-essential \
clang \
python3 \
git \
unzip \
xz-utils
RUN curl -sSLO https://github.com/Kitware/CMake/releases/download/v3.29.5/cmake-3.29.5-linux-x86_64.tar.gz \
&& tar xf cmake-3.29.5-linux-x86_64.tar.gz \
&& rm cmake-3.29.5-linux-x86_64.tar.gz \
&& mkdir -p /opt \
&& mv cmake-3.29.5-linux-x86_64 /opt/cmake
ENV PATH /opt/cmake/bin:$PATH
RUN curl -sSLO https://github.com/ninja-build/ninja/releases/download/v1.12.1/ninja-linux.zip \
&& unzip ninja-linux.zip \
&& rm *.zip \
&& mv ninja /opt/cmake/bin
ENV XDG_CACHE_HOME /tmp/cache

4
ci/docker/Dockerfile.x86_64-linux
Unescape View File

@ -0,0 +1,4 @@
FROM wasi-sdk-builder-base
ENV WASI_SDK_CI_TOOLCHAIN_CMAKE_ARGS \
-DWASI_SDK_ARTIFACT=x86_64-linux

10
ci/docker/README.md
Unescape View File

@ -0,0 +1,10 @@
# About
This folder contains the docker images that are used in CI to build the wasi-sdk
release toolchains. Docker is used to intentionally use older Linux
distributions to build the toolchain to have a more maximal set of glibc
compatibility.
These images are intended to be used on an x86\_64 host. Images start from the
`Dockerfile.common` file and then layer on target-specific
toolchains/options/etc as necessary.

79
ci/merge-artifacts.sh
Unescape View File

@ -0,0 +1,79 @@
#!/bin/sh
# Helper script executed on CI once all builds have completed. This takes
# `wasi-toolchain-*` artifacts and `wasi-sysroot-*` artifacts and merges
# them together into a single `wasi-sdk-*` artifact. Toolchains which don't
# have a sysroot that they themselves built use a sysroot from the x86_64-linux
# toolchain.
set -ex
rm -rf dist
mkdir dist
version=$(./version.py)
make_deb() {
build=$1
dir=$2
if ! command -v dpkg-deb >/dev/null; then
return
fi
case $build in
dist-x86_64-linux) deb_arch=amd64 ;;
dist-arm64-linux) deb_arch=arm64 ;;
*)
echo "unknown build $build"
exit 1
esac
mkdir dist/pkg
mkdir dist/pkg/opt
mkdir dist/pkg/DEBIAN
sed s/VERSION/$version/ wasi-sdk.control | \
sed s/ARCH/$deb_arch/ > dist/pkg/DEBIAN/control
cp -R $dir dist/pkg/opt/wasi-sdk
deb_name=$(echo $(basename $dir) | sed 's/.tar.gz//')
(cd dist && dpkg-deb -b pkg $deb_name.deb)
rm -rf dist/pkg
}
compiler_rt=`ls dist-x86_64-linux/libclang_rt*`
for build in dist-*; do
toolchain=`ls $build/wasi-toolchain-*`
if [ -f $build/wasi-sysroot-* ]; then
sysroot=`ls $build/wasi-sysroot-*`
else
sysroot=`ls dist-x86_64-linux/wasi-sysroot-*`
fi
sdk_dir=`basename $toolchain | sed 's/.tar.gz//' | sed s/toolchain/sdk/`
mkdir dist/$sdk_dir
# Start with the toolchain and then overlay the sysroot into
# `share/wasi-sysroot`, the default sysroot.
tar xf $toolchain -C dist/$sdk_dir --strip-components 1
mkdir -p dist/$sdk_dir/share/wasi-sysroot
tar xf $sysroot -C dist/$sdk_dir/share/wasi-sysroot --strip-components 1
# Setup the compiler-rt library for wasi,wasip1,wasip2
rtlibdir=$(dirname $(find dist/$sdk_dir/lib -name include))/lib
mkdir -p $rtlibdir/wasi
tar xf $compiler_rt -C $rtlibdir/wasi --strip-components 1
cp -r $rtlibdir/wasi $rtlibdir/wasip1
cp -r $rtlibdir/wasi $rtlibdir/wasip2
tar czf dist/$sdk_dir.tar.gz -C dist $sdk_dir
if echo $build | grep -q linux; then
make_deb $build dist/$sdk_dir
fi
rm -rf dist/$sdk_dir
done
# In addition to `wasi-sdk-*` also preserve artifacts for just the sysroot
# and just compiler-rt.
cp dist-x86_64-linux/wasi-sysroot-* dist
cp dist-x86_64-linux/libclang_rt* dist

37
cmake/wasi-sdk-dist.cmake
Unescape View File

@ -0,0 +1,37 @@
# Helper function to create tarballs for wasi-sdk.
#
# The `target` is the name of the CMake target to create for the creation of
# this tarball. The `tarball` argument is where the final tarball will be
# located. The name of the tarball is also used for the name of the root folder
# in the tarball. The `dir` argument is is the directory that will get packaged
# up within the tarball.
function(wasi_sdk_add_tarball target tarball dir)
cmake_path(GET tarball PARENT_PATH tarball_dir)
# Run STEM twice to chop of both `.gz` and `.tar` in `.tar.gz`
cmake_path(GET tarball STEM LAST_ONLY tarball_stem)
cmake_path(GET tarball_stem STEM LAST_ONLY tarball_stem)
if(CMAKE_SYSTEM_NAME MATCHES Windows)
# Copy the contents of symlinks on Windows to avoid dealing with symlink
set(copy_dir ${CMAKE_COMMAND} -E copy_directory ${dir} ${tarball_stem})
else()
# ... but on non-Windows copy symlinks themselves to cut down on
# distribution size.
set(copy_dir cp -R ${dir} ${tarball_stem})
endif()
add_custom_command(
OUTPUT ${tarball}
# First copy the directory under a different name, the filestem of the
# tarball.
COMMAND ${copy_dir}
# Next use CMake to create the tarball itself
COMMAND ${CMAKE_COMMAND} -E tar cfz ${tarball} ${tarball_stem}
# Finally delete the temporary directory created above.
COMMAND ${CMAKE_COMMAND} -E rm -rf ${tarball_stem}
WORKING_DIRECTORY ${tarball_dir}
COMMENT "Creating ${tarball}..."
)
add_custom_target(${target} DEPENDS ${tarball})
endfunction()

17
cmake/wasi-sdk-enable-ccache.cmake
Unescape View File

@ -0,0 +1,17 @@
# Helper module to auto-enable ccache if detected.
find_program(CCACHE ccache)
option(WASI_SDK_DISABLE_CCACHE "Force disable ccache even if it's found" OFF)
if(NOT CMAKE_C_COMPILER_LAUNCHER)
if(NOT WASI_SDK_DISABLE_CCACHE)
if(CCACHE)
set(CMAKE_C_COMPILER_LAUNCHER ccache)
set(CMAKE_CXX_COMPILER_LAUNCHER ccache)
message(STATUS "Auto-enabling ccache")
else()
message(STATUS "Failed to auto-enable ccache, not found on system")
endif()
endif()
endif()

276
cmake/wasi-sdk-sysroot.cmake
Unescape View File

@ -0,0 +1,276 @@
# Build logic for building a sysroot for wasi-sdk which includes compiler-rt,
# wasi-libc, libcxx, and libcxxabi.
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE RelWithDebInfo)
endif()
find_program(MAKE make REQUIRED)
option(WASI_SDK_DEBUG_PREFIX_MAP "Pass `-fdebug-prefix-map` for built artifacts" ON)
option(WASI_SDK_INCLUDE_TESTS "Whether or not to build tests by default" OFF)
set(wasi_sysroot ${CMAKE_INSTALL_PREFIX}/share/wasi-sysroot)
if(WASI_SDK_DEBUG_PREFIX_MAP)
add_compile_options(
-fdebug-prefix-map=${CMAKE_CURRENT_SOURCE_DIR}=wasisdk://v${wasi_sdk_version})
endif()
# Default arguments for builds of cmake projects (mostly LLVM-based) to forward
# along much of our own configuration into these projects.
set(default_cmake_args
-DCMAKE_SYSTEM_NAME=WASI
-DCMAKE_SYSTEM_VERSION=1
-DCMAKE_SYSTEM_PROCESSOR=wasm32
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_AR=${CMAKE_AR}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER_WORKS=ON
-DCMAKE_CXX_COMPILER_WORKS=ON
-DCMAKE_SYSROOT=${wasi_sysroot}
-DCMAKE_MODULE_PATH=${CMAKE_CURRENT_SOURCE_DIR}/cmake)
if(CMAKE_C_COMPILER_LAUNCHER)
list(APPEND default_cmake_args -DCMAKE_C_COMPILER_LAUNCHER=${CMAKE_C_COMPILER_LAUNCHER})
endif()
if(CMAKE_CXX_COMPILER_LAUNCHER)
list(APPEND default_cmake_args -DCMAKE_CXX_COMPILER_LAUNCHER=${CMAKE_CXX_COMPILER_LAUNCHER})
endif()
# =============================================================================
# compiler-rt build logic
# =============================================================================
set(compiler_rt_dst ${CMAKE_INSTALL_PREFIX}/lib/clang/${clang_version})
ExternalProject_Add(compiler-rt-build
SOURCE_DIR "${llvm_proj_dir}/compiler-rt"
CMAKE_ARGS
${default_cmake_args}
-DCOMPILER_RT_BAREMETAL_BUILD=ON
-DCOMPILER_RT_BUILD_XRAY=OFF
-DCOMPILER_RT_INCLUDE_TESTS=OFF
-DCOMPILER_RT_HAS_FPIC_FLAG=OFF
-DCOMPILER_RT_ENABLE_IOS=OFF
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
-DCMAKE_C_COMPILER_TARGET=wasm32-wasi
-DCOMPILER_RT_OS_DIR=wasi
-DCMAKE_INSTALL_PREFIX=${compiler_rt_dst}
EXCLUDE_FROM_ALL ON
USES_TERMINAL_CONFIGURE ON
USES_TERMINAL_BUILD ON
USES_TERMINAL_INSTALL ON
)
# In addition to the default installation of `compiler-rt` itself also copy
# around some headers and make copies of the `wasi` directory as `wasip1` and
# `wasip2`
execute_process(
COMMAND ${CMAKE_C_COMPILER} -print-runtime-dir
OUTPUT_VARIABLE clang_runtime_dir
OUTPUT_STRIP_TRAILING_WHITESPACE)
cmake_path(GET clang_runtime_dir PARENT_PATH clang_runtime_libdir) # chop off `wasi`
cmake_path(GET clang_runtime_libdir PARENT_PATH clang_sysroot_dir) # chop off `lib`
add_custom_target(compiler-rt-post-build
COMMAND ${CMAKE_COMMAND} -E copy_directory
${clang_sysroot_dir} ${compiler_rt_dst}
COMMAND ${CMAKE_COMMAND} -E copy_directory
${compiler_rt_dst}/lib/wasi ${compiler_rt_dst}/lib/wasip1
COMMAND ${CMAKE_COMMAND} -E copy_directory
${compiler_rt_dst}/lib/wasi ${compiler_rt_dst}/lib/wasip2
COMMAND ${CMAKE_COMMAND} -E copy_directory_if_different
${compiler_rt_dst}/lib ${clang_runtime_libdir}
COMMENT "finalizing compiler-rt installation"
)
add_dependencies(compiler-rt-post-build compiler-rt-build)
add_custom_target(compiler-rt DEPENDS compiler-rt-build compiler-rt-post-build)
# =============================================================================
# wasi-libc build logic
# =============================================================================
function(define_wasi_libc target)
set(build_dir ${CMAKE_CURRENT_BINARY_DIR}/wasi-libc-${target})
if(${target} MATCHES threads)
set(extra_make_flags THREAD_MODEL=posix)
elseif(${target} MATCHES p2)
set(extra_make_flags WASI_SNAPSHOT=p2 default libc_so)
else()
set(extra_make_flags default libc_so)
endif()
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UPPER)
get_property(directory_cflags DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY COMPILE_OPTIONS)
set(extra_cflags_list
"${CMAKE_C_FLAGS} ${directory_cflags} ${CMAKE_C_FLAGS_${CMAKE_BUILD_TYPE_UPPER}}")
list(JOIN extra_cflags_list " " extra_cflags)
ExternalProject_Add(wasi-libc-${target}
# Currently wasi-libc doesn't support out-of-tree builds so feigh a
# "download command" which copies the source tree to a different location
# so out-of-tree builds are supported.
DOWNLOAD_COMMAND
${CMAKE_COMMAND} -E copy_directory ${wasi_libc} ${build_dir}
SOURCE_DIR "${build_dir}"
CONFIGURE_COMMAND ""
BUILD_COMMAND
${MAKE} -j8 -C ${build_dir}
CC=${CMAKE_C_COMPILER}
AR=${CMAKE_AR}
NM=${CMAKE_NM}
SYSROOT=${wasi_sysroot}
EXTRA_CFLAGS=${extra_cflags}
TARGET_TRIPLE=${target}
${extra_make_flags}
INSTALL_COMMAND ""
DEPENDS compiler-rt
EXCLUDE_FROM_ALL ON
USES_TERMINAL_CONFIGURE ON
USES_TERMINAL_BUILD ON
USES_TERMINAL_INSTALL ON
)
endfunction()
foreach(target IN LISTS WASI_SDK_TARGETS)
define_wasi_libc(${target})
endforeach()
# =============================================================================
# libcxx build logic
# =============================================================================
function(define_libcxx target)
if(${target} MATCHES threads)
set(threads ON)
set(pic OFF)
set(target_flags -pthread)
else()
set(threads OFF)
set(pic ON)
set(target_flags "")
endif()
set(runtimes "libcxx;libcxxabi")
get_property(dir_compile_opts DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY COMPILE_OPTIONS)
get_property(dir_link_opts DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY LINK_OPTIONS)
set(extra_cflags_list ${CMAKE_C_FLAGS} ${target_flags} --target=${target} ${dir_compile_opts} ${dir_link_opts})
list(JOIN extra_cflags_list " " extra_cflags)
set(extra_cxxflags_list ${CMAKE_CXX_FLAGS} ${target_flags} --target=${target} ${dir_compile_opts} ${dir_link_opts})
list(JOIN extra_cxxflags_list " " extra_cxxflags)
ExternalProject_Add(libcxx-${target}-build
SOURCE_DIR ${llvm_proj_dir}/runtimes
CMAKE_ARGS
${default_cmake_args}
# Ensure headers are installed in a target-specific path instead of a
# target-generic path.
-DCMAKE_INSTALL_INCLUDEDIR=${wasi_sysroot}/include/${target}
-DCMAKE_STAGING_PREFIX=${wasi_sysroot}
-DCMAKE_POSITION_INDEPENDENT_CODE=${pic}
-DCXX_SUPPORTS_CXX11=ON
-DLIBCXX_ENABLE_THREADS:BOOL=${threads}
-DLIBCXX_HAS_PTHREAD_API:BOOL=${threads}
-DLIBCXX_HAS_EXTERNAL_THREAD_API:BOOL=OFF
-DLIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY:BOOL=OFF
-DLIBCXX_HAS_WIN32_THREAD_API:BOOL=OFF
-DLLVM_COMPILER_CHECKED=ON
-DLIBCXX_ENABLE_SHARED:BOOL=${pic}
-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY:BOOL=OFF
-DLIBCXX_ENABLE_EXCEPTIONS:BOOL=OFF
-DLIBCXX_ENABLE_FILESYSTEM:BOOL=ON
-DLIBCXX_ENABLE_ABI_LINKER_SCRIPT:BOOL=OFF
-DLIBCXX_CXX_ABI=libcxxabi
-DLIBCXX_CXX_ABI_INCLUDE_PATHS=${llvm_proj_dir}/libcxxabi/include
-DLIBCXX_HAS_MUSL_LIBC:BOOL=ON
-DLIBCXX_ABI_VERSION=2
-DLIBCXXABI_ENABLE_EXCEPTIONS:BOOL=OFF
-DLIBCXXABI_ENABLE_SHARED:BOOL=${pic}
-DLIBCXXABI_SILENT_TERMINATE:BOOL=ON
-DLIBCXXABI_ENABLE_THREADS:BOOL=${threads}
-DLIBCXXABI_HAS_PTHREAD_API:BOOL=${threads}
-DLIBCXXABI_HAS_EXTERNAL_THREAD_API:BOOL=OFF
-DLIBCXXABI_BUILD_EXTERNAL_THREAD_LIBRARY:BOOL=OFF
-DLIBCXXABI_HAS_WIN32_THREAD_API:BOOL=OFF
-DLIBCXXABI_ENABLE_PIC:BOOL=${pic}
-DLIBCXXABI_USE_LLVM_UNWINDER:BOOL=OFF
-DUNIX:BOOL=ON
-DCMAKE_C_FLAGS=${extra_cflags}
-DCMAKE_CXX_FLAGS=${extra_cxxflags}
-DLIBCXX_LIBDIR_SUFFIX=/${target}
-DLIBCXXABI_LIBDIR_SUFFIX=/${target}
# See https://www.scivision.dev/cmake-externalproject-list-arguments/ for
# why this is in `CMAKE_CACHE_ARGS` instead of above
CMAKE_CACHE_ARGS
-DLLVM_ENABLE_RUNTIMES:STRING=${runtimes}
DEPENDS
wasi-libc-${target}
compiler-rt
EXCLUDE_FROM_ALL ON
USES_TERMINAL_CONFIGURE ON
USES_TERMINAL_BUILD ON
USES_TERMINAL_INSTALL ON
)
# As of this writing, `clang++` will ignore the target-specific include dirs
# unless this one also exists:
add_custom_target(libcxx-${target}-extra-dir
COMMAND ${CMAKE_COMMAND} -E make_directory ${wasi_sysroot}/include/c++/v1
COMMENT "creating libcxx-specific header file folder")
add_custom_target(libcxx-${target}
DEPENDS libcxx-${target}-build libcxx-${target}-extra-dir)
endfunction()
foreach(target IN LISTS WASI_SDK_TARGETS)
define_libcxx(${target})
endforeach()
# =============================================================================
# misc build logic
# =============================================================================
# Add a top-level `build` target as well as `build-$target` targets.
add_custom_target(build ALL)
foreach(target IN LISTS WASI_SDK_TARGETS)
add_custom_target(build-${target})
add_dependencies(build-${target} libcxx-${target} wasi-libc-${target} compiler-rt)
add_dependencies(build build-${target})
endforeach()
# Install a `VERSION` file in the output prefix with a dump of version
# information.
set(version_file_tmp ${CMAKE_CURRENT_BINARY_DIR}/VERSION)
execute_process(
COMMAND ${PYTHON} ${version_script} dump
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
OUTPUT_FILE ${version_file_tmp})
install(
FILES ${version_file_tmp}
DESTINATION ${CMAKE_INSTALL_PREFIX})
if(WASI_SDK_INCLUDE_TESTS)
add_subdirectory(tests)
endif()
include(wasi-sdk-dist)
set(dist_dir ${CMAKE_CURRENT_BINARY_DIR}/dist)
# Tarball with just `compiler-rt` builtins within it
wasi_sdk_add_tarball(dist-compiler-rt
${dist_dir}/libclang_rt.builtins-wasm32-wasi-${wasi_sdk_version}.tar.gz
${CMAKE_INSTALL_PREFIX}/lib/clang/${clang_version}/lib/wasi)
add_dependencies(dist-compiler-rt compiler-rt)
# Tarball with the whole sysroot
wasi_sdk_add_tarball(dist-sysroot
${dist_dir}/wasi-sysroot-${wasi_sdk_version}.tar.gz
${CMAKE_INSTALL_PREFIX}/share/wasi-sysroot)
add_dependencies(dist-sysroot build install)
add_custom_target(dist DEPENDS dist-compiler-rt dist-sysroot)

148
cmake/wasi-sdk-toolchain.cmake
Unescape View File

@ -0,0 +1,148 @@
# Build logic and support for building a Clang toolchain that can target
# WebAssembly and build a WASI sysroot.
set(LLVM_CMAKE_FLAGS "" CACHE STRING "Extra cmake flags to pass to LLVM's build")
set(RUST_TARGET "" CACHE STRING "Target to build Rust code for, if not the host")
set(WASI_SDK_ARTIFACT "" CACHE STRING "Name of the wasi-sdk artifact being produced")
string(REGEX REPLACE "[ ]+" ";" llvm_cmake_flags_list "${LLVM_CMAKE_FLAGS}")
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE MinSizeRel)
endif()
set(default_cmake_args
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_AR=${CMAKE_AR}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX})
if(CMAKE_C_COMPILER_LAUNCHER)
list(APPEND default_cmake_args -DCMAKE_C_COMPILER_LAUNCHER=${CMAKE_C_COMPILER_LAUNCHER})
endif()
if(CMAKE_CXX_COMPILER_LAUNCHER)
list(APPEND default_cmake_args -DCMAKE_CXX_COMPILER_LAUNCHER=${CMAKE_CXX_COMPILER_LAUNCHER})
endif()
set(links_to_create clang-cl clang-cpp clang++)
foreach(target IN LISTS WASI_SDK_TARGETS)
list(APPEND links_to_create ${target}-clang)
list(APPEND links_to_create ${target}-clang++)
endforeach()
set(projects "lld;clang;clang-tools-extra")
set(tools
clang
clang-format
clang-tidy
clang-apply-replacements
lld
llvm-mc
llvm-ranlib
llvm-strip
llvm-dwarfdump
clang-resource-headers
ar
ranlib
strip
nm
size
strings
objdump
objcopy
c++filt
llvm-config)
list(TRANSFORM tools PREPEND --target= OUTPUT_VARIABLE build_targets)
list(TRANSFORM tools PREPEND --target=install- OUTPUT_VARIABLE install_targets)
ExternalProject_Add(llvm-build
SOURCE_DIR "${llvm_proj_dir}/llvm"
CMAKE_ARGS
${default_cmake_args}
-DLLVM_ENABLE_TERMINFO=OFF
-DLLVM_ENABLE_ZLIB=OFF
-DLLVM_ENABLE_ZSTD=OFF
-DLLVM_STATIC_LINK_CXX_STDLIB=ON
-DLLVM_INCLUDE_TESTS=OFF
-DLLVM_INCLUDE_UTILS=OFF
-DLLVM_INCLUDE_BENCHMARKS=OFF
-DLLVM_INCLUDE_EXAMPLES=OFF
-DLLVM_TARGETS_TO_BUILD=WebAssembly
-DLLVM_DEFAULT_TARGET_TRIPLE=wasm32-wasi
-DLLVM_INSTALL_BINUTILS_SYMLINKS=TRUE
-DLLVM_ENABLE_LIBXML2=OFF
-DDEFAULT_SYSROOT=../share/wasi-sysroot
# Pass `-s` to strip symbols by default and shrink the size of the
# distribution
-DCMAKE_EXE_LINKER_FLAGS=-s
${llvm_cmake_flags_list}
# See https://www.scivision.dev/cmake-externalproject-list-arguments/ for
# why this is in `CMAKE_CACHE_ARGS` instead of above
CMAKE_CACHE_ARGS
-DLLVM_ENABLE_PROJECTS:STRING=${projects}
-DCLANG_LINKS_TO_CREATE:STRING=${links_to_create}
BUILD_COMMAND
cmake --build . ${build_targets}
INSTALL_COMMAND
cmake --build . ${install_targets}
USES_TERMINAL_CONFIGURE ON
USES_TERMINAL_BUILD ON
USES_TERMINAL_INSTALL ON
)
# Build logic for `wasm-component-ld` installed from Rust code.
set(wasm_component_ld_root ${CMAKE_CURRENT_BINARY_DIR}/wasm-component-ld)
set(wasm_component_ld ${wasm_component_ld_root}/bin/wasm-component-ld${CMAKE_EXECUTABLE_SUFFIX})
set(wasm_component_ld_version 0.5.0)
if(RUST_TARGET)
set(rust_target_flag --target=${RUST_TARGET})
endif()
add_custom_command(
OUTPUT ${wasm_component_ld}
COMMAND
cargo install --root ${wasm_component_ld_root} ${rust_target_flag}
wasm-component-ld@${wasm_component_ld_version}
COMMENT "Building `wasm-component-ld` ...")
add_custom_target(wasm-component-ld ALL DEPENDS ${wasm_component_ld})
install(
PROGRAMS ${wasm_component_ld}
DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
# Setup installation logic for CMake support files.
install(
PROGRAMS src/config/config.sub src/config/config.guess
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/misc)
install(
FILES wasi-sdk.cmake wasi-sdk-pthread.cmake wasi-sdk-p2.cmake
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/cmake)
install(
DIRECTORY cmake/Platform
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/cmake)
include(wasi-sdk-dist)
# Figure out the name of the artifact which is either explicitly specified or
# inferred from CMake default variables.
if(WASI_SDK_ARTIFACT)
set(wasi_sdk_artifact ${WASI_SDK_ARTIFACT})
else()
if(APPLE)
set(wasi_sdk_os macos)
else()
string(TOLOWER ${CMAKE_SYSTEM_NAME} wasi_sdk_os)
endif()
set(wasi_sdk_arch ${CMAKE_SYSTEM_PROCESSOR})
set(wasi_sdk_artifact ${wasi_sdk_arch}-${wasi_sdk_os})
endif()
set(dist_dir ${CMAKE_CURRENT_BINARY_DIR}/dist)
wasi_sdk_add_tarball(dist-toolchain
${dist_dir}/wasi-toolchain-${wasi_sdk_version}-${wasi_sdk_artifact}.tar.gz
${CMAKE_INSTALL_PREFIX})
add_dependencies(dist-toolchain llvm-build install)
add_custom_target(dist DEPENDS dist-toolchain)

43
deb_from_installation.sh
Unescape View File

@ -1,43 +0,0 @@
#!/usr/bin/env sh
set -x
command -v dpkg-deb >/dev/null
if [ $? -ne 0 ]; then
echo "required tool dpkg-deb missing. exiting"
exit 0
fi
set -ex
if [ -n "$1" ]; then
OUTDIR=$1
else
OUTDIR=$PWD/dist
fi
if [ -n "$2" ]; then
VERSION="$2"
else
VERSION=`./version.py`
fi
if [ -n "$3" ]; then
INSTALL_DIR="$3"
else
INSTALL_DIR=/opt/wasi-sdk
fi
if [ ! -d $INSTALL_DIR ] ; then
echo "Directory $INSTALL_DIR doesn't exist. Nothing to copy from."
exit 1
fi
ARCH=$(dpkg --print-architecture)
rm -rf build/pkg
mkdir -p build/pkg/opt
mkdir -p build/pkg/DEBIAN
sed -e s/VERSION/$VERSION/ wasi-sdk.control > build/pkg/DEBIAN/control
cp -R $INSTALL_DIR build/pkg/opt/
cd build && dpkg-deb -b pkg wasi-sdk_$VERSION\_$ARCH\.deb && cd ..
mv build/wasi-sdk_$VERSION\_$ARCH\.deb $OUTDIR/

68
docker/Dockerfile
Unescape View File

@ -1,53 +1,27 @@
# Docker image with a build toolchain and environment variables set to use # A container which has a number of build tools pre-installed plus a build of
# the wasi-sdk sysroot. The SDK distribution must have first been built, # `wasi-sdk` installed at `/opt/wasi-sdk`. This also has environment variables
# for example using docker_build.sh # pre-configued to use the installed toolchain.
#
# Extract built SDK archive to copy out the sysroot. We use an initial build # This container is built as the last step on CI for this repository and
# stage to do this to make sure it is only the sysroot, not the entire SDK # pre-built versions of this container are pushed as a package to the repository
# with binaries, that is included in the final image since we install those # as well.
# separately.
FROM ubuntu:22.04 as dist
ADD dist/wasi-sdk-*.*-linux.tar.gz /
ADD dist/libclang_rt.builtins-wasm32-wasi-*.*.tar.gz /wasi-sysroot-clang_rt
# Move versioned folder to unversioned to using bash glob to allow
# this file to be independent of major version number.
RUN mv /wasi-sdk-* /wasi-sdk
FROM ubuntu:22.04 FROM ubuntu:22.04
ENV LLVM_VERSION 18
# Install build toolchain including clang, ld, make, autotools, ninja, and cmake
RUN apt-get update && \ RUN apt-get update && \
# Temporarily install to setup apt repositories apt-get install -y cmake ninja-build make autoconf autogen automake libtool && \
apt-get install -y curl gnupg && \
\
curl -sS https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor > /etc/apt/trusted.gpg.d/llvm.gpg && \
echo "deb [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \
echo "deb-src [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \
\
apt-get update && \
apt-get install -y clang-${LLVM_VERSION} lld-${LLVM_VERSION} cmake ninja-build make autoconf autogen automake libtool && \
apt-get autoremove -y curl gnupg && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
COPY --from=dist /wasi-sdk/share/wasi-sysroot/ /wasi-sysroot/ ADD dist/wasi-sdk-*.deb .
COPY --from=dist /wasi-sysroot-clang_rt/lib/wasi /usr/lib/llvm-${LLVM_VERSION}/lib/clang/${LLVM_VERSION}/lib/wasi RUN case `dpkg --print-architecture` in \
amd64) dpkg -i wasi-sdk-*-x86_64-linux.deb ;; \
ADD docker/wasi-sdk.cmake /usr/share/cmake/wasi-sdk.cmake arm64) dpkg -i wasi-sdk-*-arm64-linux.deb ;; \
ADD docker/wasi-sdk-pthread.cmake /usr/share/cmake/wasi-sdk-pthread.cmake *) exit 1 ;; \
ADD docker/wasi-sdk-p2.cmake /usr/share/cmake/wasi-sdk-p2.cmake esac && \
ENV CMAKE_TOOLCHAIN_FILE /usr/share/cmake/wasi-sdk.cmake rm wasi-sdk-*.deb
ADD cmake/Platform/WASI.cmake /usr/share/cmake/Modules/Platform/WASI.cmake
ENV CC /opt/wasi-sdk/bin/clang
ENV CC clang-${LLVM_VERSION} ENV CXX /opt/wasi-sdk/bin/clang++
ENV CXX clang++-${LLVM_VERSION} ENV LD /opt/wasi-sdk/bin/wasm-ld
ENV LD wasm-ld-${LLVM_VERSION} ENV AR /opt/wasi-sdk/bin/llvm-ar
ENV AR llvm-ar-${LLVM_VERSION} ENV RANLIB /opt/wasi-sdk/bin/llvm-ranlib
ENV RANLIB llvm-ranlib-${LLVM_VERSION}
ENV CFLAGS --target=wasm32-wasi --sysroot=/wasi-sysroot
ENV CXXFLAGS --target=wasm32-wasi --sysroot=/wasi-sysroot
ENV LDFLAGS --target=wasm32-wasi --sysroot=/wasi-sysroot

29
docker/wasi-sdk-p2.cmake
Unescape View File

@ -1,29 +0,0 @@
# Cmake toolchain description file for the wasi-sdk docker image
# This is arbitrary, AFAIK, for now.
cmake_minimum_required(VERSION 3.4.0)
# To make sure it recognizes the WASI platform
list(APPEND CMAKE_MODULE_PATH /usr/share/cmake/Modules)
set(CMAKE_SYSTEM_NAME WASI)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR wasm32)
set(triple wasm32-wasip2)
set(CMAKE_C_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_CXX_COMPILER /usr/bin/clang++-$ENV{LLVM_VERSION})
set(CMAKE_ASM_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_AR /usr/bin/llvm-ar-$ENV{LLVM_VERSION})
set(CMAKE_RANLIB /usr/bin/llvm-ranlib-$ENV{LLVM_VERSION})
set(CMAKE_C_COMPILER_TARGET ${triple})
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
SET(CMAKE_SYSROOT /wasi-sysroot)
# Don't look in the sysroot for executables to run during the build
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# Only look in the sysroot (not in the host paths) for the rest
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

36
docker/wasi-sdk-pthread.cmake
Unescape View File

@ -1,36 +0,0 @@
# Cmake toolchain description file for the wasi-sdk docker image
# This is arbitrary, AFAIK, for now.
cmake_minimum_required(VERSION 3.4.0)
# To make sure it recognizes the WASI platform
list(APPEND CMAKE_MODULE_PATH /usr/share/cmake/Modules)
set(CMAKE_SYSTEM_NAME WASI)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR wasm32)
set(triple wasm32-wasi-threads)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -pthread")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pthread")
# wasi-threads requires --import-memory.
# wasi requires --export-memory.
# (--export-memory is implicit unless --import-memory is given)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--import-memory")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--export-memory")
set(CMAKE_C_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_CXX_COMPILER /usr/bin/clang++-$ENV{LLVM_VERSION})
set(CMAKE_ASM_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_AR /usr/bin/llvm-ar-$ENV{LLVM_VERSION})
set(CMAKE_RANLIB /usr/bin/llvm-ranlib-$ENV{LLVM_VERSION})
set(CMAKE_C_COMPILER_TARGET ${triple})
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
SET(CMAKE_SYSROOT /wasi-sysroot)
# Don't look in the sysroot for executables to run during the build
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# Only look in the sysroot (not in the host paths) for the rest
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

29
docker/wasi-sdk.cmake
Unescape View File

@ -1,29 +0,0 @@
# Cmake toolchain description file for the wasi-sdk docker image
# This is arbitrary, AFAIK, for now.
cmake_minimum_required(VERSION 3.4.0)
# To make sure it recognizes the WASI platform
list(APPEND CMAKE_MODULE_PATH /usr/share/cmake/Modules)
set(CMAKE_SYSTEM_NAME WASI)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR wasm32)
set(triple wasm32-wasi)
set(CMAKE_C_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_CXX_COMPILER /usr/bin/clang++-$ENV{LLVM_VERSION})
set(CMAKE_ASM_COMPILER /usr/bin/clang-$ENV{LLVM_VERSION})
set(CMAKE_AR /usr/bin/llvm-ar-$ENV{LLVM_VERSION})
set(CMAKE_RANLIB /usr/bin/llvm-ranlib-$ENV{LLVM_VERSION})
set(CMAKE_C_COMPILER_TARGET ${triple})
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
SET(CMAKE_SYSROOT /wasi-sysroot)
# Don't look in the sysroot for executables to run during the build
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# Only look in the sysroot (not in the host paths) for the rest
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

18
docker_build.sh
Unescape View File

@ -1,18 +0,0 @@
#!/bin/sh
set -ex
echo "Building the docker image"
docker build \
--build-arg UID=$(id -u) --build-arg GID=$(id -g) \
-t wasi-sdk-builder:latest .
echo "Building the package in docker image"
mkdir -p ~/.ccache
docker run --rm \
--user $(id -u):$(id -g) \
-v "$PWD":/workspace:Z \
-v ~/.ccache:/home/builder/.ccache:Z \
-e NINJA_FLAGS=-v \
--tmpfs /tmp:exec \
wasi-sdk-builder:latest \
make package LLVM_CMAKE_FLAGS=-DLLVM_CCACHE_BUILD=ON

15
strip_symbols.sh
Unescape View File

@ -1,15 +0,0 @@
#!/usr/bin/env bash
set -e
DIRECTORY=${1:-/opt/wasi-sdk/bin}
if [[ "$OSTYPE" == "darwin"* ]] || [[ "$OSTYPE" == "freebsd"* ]]; then
# macos and freebsd find do not support -executable so we fall back on
# having a permission bit to execute:
EXECUTABLES=$(find ${DIRECTORY} -type f -perm +111)
else
EXECUTABLES=$(find ${DIRECTORY} -type f -executable)
fi
for e in ${EXECUTABLES}; do
echo "Stripping symbols: ${e}"
strip ${e} || echo "Failed to strip symbols for ${e}; continuing on."
done

51
tar_from_installation.sh
Unescape View File

@ -1,51 +0,0 @@
#!/usr/bin/env bash
set -ex
if [ -n "$1" ]; then
OUTDIR=$1
else
OUTDIR=$PWD/dist
fi
if [ -n "$2" ]; then
VERSION="$2"
else
VERSION=`./version.py`
fi
if [ -n "$3" ]; then
INSTALL_DIR="$3"
else
INSTALL_DIR=/opt/wasi-sdk
fi
case "$(uname -s)" in
Linux*) MACHINE=linux;;
Darwin*) MACHINE=macos;;
CYGWIN*) MACHINE=cygwin;;
MINGW*) MACHINE=mingw;;
MSYS*) MACHINE=msys;; #MSYS_NT-10.0-19043
*) MACHINE="UNKNOWN"
esac
if [ ! -d $INSTALL_DIR ] ; then
echo "Directory $INSTALL_DIR doesn't exist. Nothing to copy from."
exit 1
fi
PKGDIR=build/wasi-sdk-$VERSION
rm -rf $PKGDIR
if [ "$MACHINE" == "cygwin" ] || [ "$MACHINE" == "mingw" ] || [ "$MACHINE" == "msys" ]; then
# Copy with -L to avoid trying to create symlinks on Windows.
cp -R -L $INSTALL_DIR $PKGDIR
else
cp -R $INSTALL_DIR $PKGDIR
fi
cd build
tar czf $OUTDIR/wasi-sdk-$VERSION\-$MACHINE.tar.gz wasi-sdk-$VERSION
# As well as the full SDK package, also create archives of libclang_rt.builtins
# and the sysroot. These are made available for users who have an existing clang
# installation.
tar czf $OUTDIR/libclang_rt.builtins-wasm32-wasi-$VERSION.tar.gz -C compiler-rt lib/wasi
tar czf $OUTDIR/wasi-sysroot-$VERSION.tar.gz -C wasi-sdk-$VERSION/share wasi-sysroot

88
tests/CMakeLists.txt
Unescape View File

@ -0,0 +1,88 @@
# Support for running tests in the `tests/{compile-only,general}` folders
cmake_minimum_required(VERSION 3.22)
project(wasi-sdk-test)
include(CTest)
enable_testing()
set(CMAKE_EXECUTABLE_SUFFIX ".wasm")
add_compile_options(--sysroot=${wasi_sysroot})
add_link_options(--sysroot=${wasi_sysroot})
# Sanity check setup
if (NOT ${CMAKE_SYSTEM_NAME} STREQUAL WASI)
message(FATAL_ERROR "Wrong system name (${CMAKE_SYSTEM_NAME}), wrong toolchain file in use?")
endif()
if(NOT DEFINED WASI)
message(FATAL_ERROR "WASI is not set, platform file likely not loaded")
endif()
set(WASI_SDK_RUNWASI "wasmtime" CACHE STRING "Runner for tests")
# Test everything at O0, O2, and O2+LTO
set(opt_flags -O0 -O2 "-O2 -flto")
# Executes a single `test` specified.
#
# This will compile `test` for all the various targets and with various
# compiler options. If `runwasi` is non-empty then the test will be executed
# in that runner as well.
function(add_testcase runwasi test)
foreach(target IN LISTS WASI_SDK_TARGETS)
foreach(compile_flags IN LISTS opt_flags)
# Mangle the options into something appropriate for a CMake rule name
string(REGEX REPLACE " " "." target_name "${target}.${compile_flags}.${test}")
# Add a new test executable based on `test`
add_executable(${target_name} ${test})
# Configure all the compile options necessary. For example `--target` here
# if the target doesn't look like it's already in the name of the compiler
# as well.
if(NOT(CMAKE_C_COMPILER MATCHES ${target}))
target_compile_options(${target_name} PRIVATE --target=${target})
endif()
# Apply test-specific compile options and link flags.
if(test MATCHES "clocks.c$")
target_compile_options(${target_name} PRIVATE -D_WASI_EMULATED_PROCESS_CLOCKS)
target_link_options(${target_name} PRIVATE -lwasi-emulated-process-clocks)
elseif(test MATCHES "mmap.c$")
target_compile_options(${target_name} PRIVATE -D_WASI_EMULATED_MMAN)
target_link_options(${target_name} PRIVATE -lwasi-emulated-mman)
elseif(test MATCHES "(sigabrt|signals).c$")
target_compile_options(${target_name} PRIVATE -D_WASI_EMULATED_SIGNAL)
target_link_options(${target_name} PRIVATE -lwasi-emulated-signal)
elseif(test MATCHES "printf-long-double-enabled.c$")
target_link_options(${target_name} PRIVATE -lc-printscan-long-double)
endif()
# Apply language-specific options and dependencies.
if(test MATCHES "cc$")
target_compile_options(${target_name} PRIVATE -fno-exceptions)
add_dependencies(${target_name} libcxx-${target})
else()
add_dependencies(${target_name} wasi-libc-${target})
endif()
# Apply target-specific options.
if(target MATCHES threads)
target_compile_options(${target_name} PRIVATE -pthread)
endif()
if(runwasi)
add_test(
NAME test-${target_name}
COMMAND
bash ../testcase.sh
"${runwasi}"
${test}
$<TARGET_FILE:${target_name}>
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif()
endforeach()
endforeach()
endfunction()
add_subdirectory(compile-only)
add_subdirectory(general)

34
tests/cmake/CMakeLists.txt
Unescape View File

@ -1,34 +0,0 @@
cmake_minimum_required(VERSION 3.22)
project(wasi-sdk-test)
# Sanity check setup
if (NOT ${CMAKE_SYSTEM_NAME} STREQUAL WASI)
message(FATAL_ERROR "Wrong system name (${CMAKE_SYSTEM_NAME}), wrong toolchain file in use?")
endif()
if(NOT DEFINED WASI)
message(FATAL_ERROR "WASI is not set, platform file likely not loaded")
endif()
set(RUNWASI "" CACHE STRING "Path to or name of WASM runner")
# Test build a C and C++ target respectively
add_executable(void_main_c ../general/void_main.c)
add_executable(void_main_cc ../general/void_main.cc)
include(CTest)
enable_testing()
add_test(NAME void_main_c
COMMAND
${CMAKE_CURRENT_SOURCE_DIR}/test_driver.sh
${RUNWASI}
$<TARGET_FILE:void_main_c>
${CMAKE_CURRENT_SOURCE_DIR}/../general/void_main.c.stdout.expected)
add_test(NAME void_main_cc
COMMAND
${CMAKE_CURRENT_SOURCE_DIR}/test_driver.sh
${RUNWASI}
$<TARGET_FILE:void_main_cc>
${CMAKE_CURRENT_SOURCE_DIR}/../general/void_main.cc.stdout.expected)

17
tests/cmake/test_driver.sh
Unescape View File

@ -1,17 +0,0 @@
#!/bin/bash
# Simplified runner for cmake
set -ex
runwasi="$1"
target="$2"
stdout_expected="$3"
stderr_expected="/dev/null"
stdout_observed="$target.stdout.observed"
stderr_observed="$target.stderr.observed"
"$runwasi" "$target" > "$stdout_observed" 2> "$stderr_observed"
diff -u "$stderr_expected" "$stderr_observed"
diff -u "$stdout_expected" "$stdout_observed"

9
tests/compile-only/CMakeLists.txt
Unescape View File

@ -0,0 +1,9 @@
file(GLOB c_compile_tests RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} "*.c")
file(GLOB cxx_compile_tests RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} "*.cc")
set(compile_tests ${c_compile_tests} ${cxx_compile_tests})
foreach(test IN LISTS compile_tests)
add_testcase("" ${test})
endforeach()

8
tests/general/CMakeLists.txt
Unescape View File

@ -0,0 +1,8 @@
file(GLOB c_general_tests RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} "*.c")
file(GLOB cxx_general_tests RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} "*.cc")
set(general_tests ${c_general_tests} ${cxx_general_tests})
foreach(test IN LISTS general_tests)
add_testcase(${WASI_SDK_RUNWASI} ${test})
endforeach()

2
tests/general/abort.c.stderr.expected.filter
Unescape View File

@ -2,6 +2,6 @@
set -euo pipefail set -euo pipefail
cat \ cat \
| sed -e 's/main module `abort\.c\.[^`]*\.wasm`/main module `abort.c.---.wasm`/' \ | sed -e 's/main module `.*abort\.c\.wasm`/main module `abort.c.---.wasm`/' \
| sed -e 's/failed to invoke.*/failed to invoke ---/' \ | sed -e 's/failed to invoke.*/failed to invoke ---/' \
| sed -E '/0x[[:xdigit:]]+/d' | sed -E '/0x[[:xdigit:]]+/d'

4
tests/general/assert-fail.c
Unescape View File

@ -1,3 +1,7 @@
#ifdef NDEBUG
#undef NDEBUG
#endif
#include <assert.h> #include <assert.h>
#include <stdbool.h> #include <stdbool.h>

2
tests/general/assert-fail.c.stderr.expected
Unescape View File

@ -1,4 +1,4 @@
Assertion failed: false (assert-fail.c: main: 5) Assertion failed: false (assert-fail.c: main: 9)
Error: failed to run main module `assert-fail.c.---.wasm` Error: failed to run main module `assert-fail.c.---.wasm`
Caused by: Caused by:

3
tests/general/assert-fail.c.stderr.expected.filter
Unescape View File

@ -2,6 +2,7 @@
set -euo pipefail set -euo pipefail
cat \ cat \
| sed -e 's/main module `assert-fail\.c\.[^`]*\.wasm`/main module `assert-fail.c.---.wasm`/' \ | sed -e 's/main module `.*assert-fail\.c\.wasm`/main module `assert-fail.c.---.wasm`/' \
| sed -e 's/failed to invoke.*/failed to invoke ---/' \ | sed -e 's/failed to invoke.*/failed to invoke ---/' \
| sed -e 's/Assertion failed: false (.*assert-fail.c/Assertion failed: false (assert-fail.c/' \
| sed -E '/0x[[:xdigit:]]+/d' | sed -E '/0x[[:xdigit:]]+/d'

1
tests/general/clocks.c.options
Unescape View File

@ -1 +0,0 @@
-D_WASI_EMULATED_PROCESS_CLOCKS -lwasi-emulated-process-clocks

1
tests/general/iostream_main.cc.options
Unescape View File

@ -1 +0,0 @@
-fno-exceptions

1
tests/general/mmap.c.options
Unescape View File

@ -1 +0,0 @@
-D_WASI_EMULATED_MMAN -lwasi-emulated-mman

1
tests/general/printf-long-double-enabled.c.options
Unescape View File

@ -1 +0,0 @@
-lc-printscan-long-double

1
tests/general/sigabrt.c.options
Unescape View File

@ -1 +0,0 @@
-D_WASI_EMULATED_SIGNAL -lwasi-emulated-signal

2
tests/general/sigabrt.c.stderr.expected.filter
Unescape View File

@ -2,6 +2,6 @@
set -euo pipefail set -euo pipefail
cat \ cat \
| sed -e 's/main module `sigabrt\.c\.[^`]*\.wasm`/main module `sigabrt.c.---.wasm`/' \ | sed -e 's/main module `.*sigabrt\.c\.wasm`/main module `sigabrt.c.---.wasm`/' \
| sed -e 's/source location: @[[:xdigit:]]*$/source location: @----/' \ | sed -e 's/source location: @[[:xdigit:]]*$/source location: @----/' \
| head -n 6 | head -n 6

7
tests/general/signals.c
Unescape View File

@ -1,3 +1,7 @@
#ifdef NDEBUG
#undef NDEBUG
#endif
#include <signal.h> #include <signal.h>
#include <string.h> #include <string.h>
#include <assert.h> #include <assert.h>
@ -6,7 +10,10 @@
#include <errno.h> #include <errno.h>
// Make sure this exists. // Make sure this exists.
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-W#warnings"
#include <sys/signal.h> #include <sys/signal.h>
#pragma clang diagnostic pop
volatile sig_atomic_t flag = 0; volatile sig_atomic_t flag = 0;

1
tests/general/signals.c.options
Unescape View File

@ -1 +0,0 @@
-D_WASI_EMULATED_SIGNAL -lwasi-emulated-signal

89
tests/run.sh
Unescape View File

@ -1,89 +0,0 @@
#!/bin/bash
set -ueo pipefail
# Top-level test runner. Usage is "run.sh <path to wasi-sdk>" to run tests
# in compile-only mode, or "run.sh <path to wasi-sdk> <runwasi>" where
# <runwasi> is a WASI-capable runtime to run the tests in full compile and
# execute mode.
#
# The compiler used during testing is loaded from `<path to wasi-sdk>`.
if [ $# -lt 1 ]; then
echo "Path to WASI SDK is required"
exit 1
fi
wasi_sdk="$1"
# Determine the wasm runtime to use, if one is provided.
if [ $# -gt 1 ]; then
runwasi="$2"
else
runwasi=""
fi
testdir=$(dirname $0)
echo "SDK: $wasi_sdk"
# NB: all tests are run with the default `clang` and `clang++` executables
# but they're also executed with the target-prefixed `clang` executables to
# ensure that those work as well when the `--target` option is omitted.
for target in $TARGETS; do
echo "===== Testing target $target ====="
cd $testdir/compile-only
for options in -O0 -O2 "-O2 -flto"; do
echo "===== Testing compile-only with $options ====="
for file in *.c; do
echo "Testing compile-only $file..."
../testcase.sh "$target" "" "$wasi_sdk/bin/clang" "$options --target=$target" "$file"
../testcase.sh "$target" "" "$wasi_sdk/bin/$target-clang" "$options" "$file"
done
for file in *.cc; do
echo "Testing compile-only $file..."
../testcase.sh "$target" "" "$wasi_sdk/bin/clang++" "$options --target=$target -fno-exceptions" "$file"
../testcase.sh "$target" "" "$wasi_sdk/bin/$target-clang++" "$options -fno-exceptions" "$file"
done
done
cd - >/dev/null
cd $testdir/general
for options in -O0 -O2 "-O2 -flto"; do
echo "===== Testing with $options ====="
for file in *.c; do
echo "Testing $file..."
../testcase.sh "$target" "$runwasi" "$wasi_sdk/bin/clang" "$options --target=$target" "$file"
../testcase.sh "$target" "$runwasi" "$wasi_sdk/bin/$target-clang" "$options" "$file"
done
for file in *.cc; do
echo "Testing $file..."
../testcase.sh "$target" "$runwasi" "$wasi_sdk/bin/clang++" "$options --target=$target -fno-exceptions" "$file"
../testcase.sh "$target" "$runwasi" "$wasi_sdk/bin/$target-clang++" "$options -fno-exceptions" "$file"
done
done
cd - >/dev/null
done
# Test cmake build system for wasi-sdk
test_cmake() {
local option
for option in Debug Release; do
rm -rf "$testdir/cmake/build/$option"
mkdir -p "$testdir/cmake/build/$option"
cd "$testdir/cmake/build/$option"
cmake \
-G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE="$option" \
-DRUNWASI="$runwasi" \
-DWASI_SDK_PREFIX="$wasi_sdk" \
-DCMAKE_TOOLCHAIN_FILE="$wasi_sdk/share/cmake/wasi-sdk.cmake" \
../..
make
if [[ -n "$runwasi" ]]; then
ctest --output-on-failure
fi
cd - >/dev/null
done
}
test_cmake

52
tests/testcase.sh
Unescape View File

@ -6,39 +6,19 @@ set -ueo pipefail
# Command-line parsing; this script is meant to be run from a higher-level # Command-line parsing; this script is meant to be run from a higher-level
# script, so don't do anything fancy. # script, so don't do anything fancy.
target="$1" runwasi="$1"
runwasi="$2" input="$2"
compiler="$3" wasm="$3"
options="$4"
input="$5"
# Compile names for generated files. # Compile names for generated files.
wasm="$input.$options.wasm" stdout_observed="$wasm.stdout.observed"
stdout_observed="$input.$options.stdout.observed" stderr_observed="$wasm.stderr.observed"
stderr_observed="$input.$options.stderr.observed" exit_status_observed="$wasm.exit_status.observed"
exit_status_observed="$input.$options.exit_status.observed"
# Optionally load compiler options from a file. # Double-check that a runwasi command was specified since otherwise this script
if [ -e "$input.options" ]; then # was invoked with no arguments which isn't as intended.
file_options=$(cat "$input.options")
else
file_options=
fi
if echo "$target" | grep -q -- '-threads$'; then
pthread_options="-pthread"
else
pthread_options=
fi
echo "Testing $input..."
# Compile the testcase.
$compiler $pthread_options $options $file_options "$input" -o "$wasm"
# If we don't have a runwasi command, we're just doing compile-only testing.
if [ "$runwasi" == "" ]; then if [ "$runwasi" == "" ]; then
exit 0 exit 1
fi fi
# Determine the input file to write to stdin. # Determine the input file to write to stdin.
@ -75,11 +55,7 @@ echo $exit_status > "$exit_status_observed"
# Determine the reference files to compare with. # Determine the reference files to compare with.
if [ -e "$input.stdout.expected" ]; then if [ -e "$input.stdout.expected" ]; then
if [ -e "$input.$target.stdout.expected" ]; then
stdout_expected="$input.$target.stdout.expected"
else
stdout_expected="$input.stdout.expected" stdout_expected="$input.stdout.expected"
fi
# Apply output filters. # Apply output filters.
if [ -e "$input.stdout.expected.filter" ]; then if [ -e "$input.stdout.expected.filter" ]; then
@ -93,11 +69,7 @@ else
fi fi
if [ -e "$input.stderr.expected" ]; then if [ -e "$input.stderr.expected" ]; then
if [ -e "$input.$target.stderr.expected" ]; then
stderr_expected="$input.$target.stderr.expected"
else
stderr_expected="$input.stderr.expected" stderr_expected="$input.stderr.expected"
fi
# Apply output filters. # Apply output filters.
if [ -e "$input.stderr.expected.filter" ]; then if [ -e "$input.stderr.expected.filter" ]; then
@ -117,6 +89,6 @@ fi
# If there are any differences, diff will return a non-zero exit status, and # If there are any differences, diff will return a non-zero exit status, and
# since this script uses "set -e", it will return a non-zero exit status too. # since this script uses "set -e", it will return a non-zero exit status too.
diff -u "$stderr_expected" "$stderr_observed" diff --ignore-space-change -u "$stderr_expected" "$stderr_observed"
diff -u "$stdout_expected" "$stdout_observed" diff --ignore-space-change -u "$stdout_expected" "$stdout_observed"
diff -u "$exit_status_expected" "$exit_status_observed" diff --ignore-space-change -u "$exit_status_expected" "$exit_status_observed"

2
wasi-sdk-p2.cmake
Unescape View File

@ -1,7 +1,7 @@
# Cmake toolchain description file for the Makefile # Cmake toolchain description file for the Makefile
# This is arbitrary, AFAIK, for now. # This is arbitrary, AFAIK, for now.
cmake_minimum_required(VERSION 3.4.0) cmake_minimum_required(VERSION 3.5.0)
# Until Platform/WASI.cmake is upstream we need to inject the path to it # Until Platform/WASI.cmake is upstream we need to inject the path to it
# into CMAKE_MODULE_PATH. # into CMAKE_MODULE_PATH.

2
wasi-sdk.cmake
Unescape View File

@ -1,7 +1,7 @@
# Cmake toolchain description file for the Makefile # Cmake toolchain description file for the Makefile
# This is arbitrary, AFAIK, for now. # This is arbitrary, AFAIK, for now.
cmake_minimum_required(VERSION 3.4.0) cmake_minimum_required(VERSION 3.5.0)
# Until Platform/WASI.cmake is upstream we need to inject the path to it # Until Platform/WASI.cmake is upstream we need to inject the path to it
# into CMAKE_MODULE_PATH. # into CMAKE_MODULE_PATH.

4
wasi-sdk.control
Unescape View File

@ -1,6 +1,6 @@
Package: wasi-sdk Package: wasi-sdk
Version: VERSION Version: VERSION
Architecture: amd64 Architecture: ARCH
Priority: optional Priority: optional
Description: Clang toolchain with wasm32-wasi default target, and the wasi sysroot Description: Clang toolchain with wasm32-wasi default target, and the wasi sysroot
Maintainer: Pat Hickey <phickey@fastly.com> Maintainer: WASI SDK Maintainers <webassembly@users.noreply.github.com>

Write Preview
Loading…
Cancel
Save