This commit updates the release process of `wasi-sdk` to remove most
of the manual interaction and steps done. Instead now draft releases are
automatically created for tags made. This means that there's only two
steps necessary: (1) pushing a tag and (2) hitting publish on the
generated release. This commit also removes a number of the CI scripts
previously used to manage releases.
* Rewrite the build system with CMake
This commit is an attempt to provide a concrete path forward on
WebAssembly/wasi-sdk#425. I personally think it's pretty important to
get the ability to have more architectures here but at the same time I
also think it's important to to take this as an opportunity to refactor
and improve the build system of this repository. To that end this
represents my attempt to improve the status quo.
This removes the old `Makefile` and replaces it with a CMake-based
system to build all these projects. Overall this is intended to be a "no
functional change" intended sort of refactoring. Changing build systems
inevitably causes issues, however, so this change additionally has a
very high likelihood of needing follow-up fixes. At a high enough level
this commit introduces two major changes to how this repository is
built:
1. The `make`-based system (the root `Makefile`) is replaced with CMake.
This additionally updates tests to use CMake.
2. A single "build" is split into either building a toolchain or
building a sysroot. This enables builds to only build one or the
other as necessary.
The first change, using CMake, is due to the fact that using `make` on
Windows basically is not pleasant coupled with the fact that more
advanced logic, such as changing flags, compilers, etc, is much easier
with a CMake-based system. The second change is intended to cover the
use case of #425 in addition to refactoring the current build.
Throughout this change I have intentionally not tried to keep a 1:1
correspondance with behaviors in the old `Makefile` because much of this
PR is intended to address shortcomings in the old build system. A list
of changes, improvements, etc, made here are:
* CMake provides a much nicer portability story to Windows than `make`.
This is moving towards the direction of not needing `bash`, for
example, to build an SDK. Currently `wasi-libc` still requires this,
but that's now the only "hard" dependency.
* The set of targets built can now be configured for smaller builds
and/or debugging just a single target. All WASI targets are still
built by default but it's much easier to add/remove them.
* Different targets are now able to be built in parallel as opposed to
the unconditional serial-nature of the `Makefile`.
* Use of `ninja` is no longer required and separate build systems can be
used if desired.
* The sysroot and the toolchain can now be built with different CMake
build profiles. For example the `Makefile` hardcoded `MinSizeRel` and
`RelWithDebInfo` and this can now be much more easily customized by
the SDK builder.
* Tarballs are now more consistently produced and named. For a tarball
of the name `foo.tar.gz` it's guaranteed that there's a single folder
`foo` created when unpacking the tarball.
* The macOS binaries are no longer hybrid x64/arm64 binaries which
greatly inflates the size of the SDK. There's now a separate build for
each architecture.
* CI now produces arm64-linux binaries. The sysroot is not built on the
arm64-linux builder and the sysroot from the x86_64-linux builder is
used instead.
* Tests are almost ready to execute on Windows, there's just a few minor
issues related to exit statuses and probably line endings which need
to be worked out. Will require someone with a Windows checkout, however.
* Tests are now integrated into CMake. This means that the wasm binaries
are able to be built in parallel and the tests are additionally
executed in parallel with `ctest`. It is possible to build/run a
single test. Tests no longer place all of their output in the source
tree.
* Out-of-tree builds are now possible and the build/installation
directories can both be customized.
* CI configuration of Windows/macOS/Linux is much more uniform by having
everything in one build matrix instead of separate matrices.
* Linux builds are exclusively done in docker containers in CI now. CI
no longer produces two Linux builds only for one to be discarded when
artifacts are published.
* Windows 32-bit builds are no longer produced in CI since it's expected
that everyone actually wants the 64-bit ones instead.
* Use of `ccache` is now automatically enabled if it's detected on the
system.
* Many preexisting shell scripts are now translated to CMake one way or
another.
* There's no longer a separate build script for how to build wasi-sdk in
docker and outside of docker which needs to be kept in sync,
everything funnels through the same script.
* The `docker/Dockerfile` build of wasi-sdk now uses the actual
toolchain built from CI and additionally doesn't duplicate various
CMake-based configuration files.
Overall one thing I want to additionally point out is that I'm not CMake
expert. I suspect there's lots of little stylistic and such improvements
that can be made.
* Fix skipping tests on windows
* Fetch a full depth in the finalize workflow too
* Fix multi-arch docker build
* Fix assembling of sysroot
* Fix script syntax
* Clean up the merge script slightly
* Remove Pat's email
* Move configuration of CMAKE_EXECUTABLE_SUFFIX
* Remove redundant sysroot option
* Fix comment in testcase.sh
* Update new p2 cmake files
* Remove now-duplicate wasi-sdk-p2.cmake
My release attempt in #408 did not go well because the release artifacts
were not named correctly. This fixes an issue described by
actions/checkout#290 where checkouts of annotated tags overwrite the
annotation which breaks `git describe`. That means that version
detection is broken in CI during releases which causes artifacts to have
the wrong information.
This applies the workaround described in that issue to `git fetch --tags
--force` after the checkout step to undo the overwrite done in the
checkout step.
* Update LLVM to 18.1.2 and add wasm-component-ld
This commit has two intertwined changes within it. First the LLVM
submodule is updated to the 18.1.2 release branch. This alone would
cause CI and tests to fail due to differing behavior for the
`wasm32-wasip2` target. To fix these test failures the
`wasm-component-ld` tool is added to the build. This tool is a
Rust-written tool and installed via `cargo install` as part of the build
at a pinned version written in the `Makefile`. This linker, used for
components, is then used for the `wasm32-wasip2` target.
Tests and CI are then updated to skip the need to have `wasm-tools` or
the adapter for WASI when making components since that's now the job of
`wasm-component-ld`. This then necessitated some changes to tests too.
* Add back accidentally deleted CI
* Inherit tools on PATH on Windows
* Add Rust to docker build
* Put rust in a different location in docker
* Try to fix permissions
* Review comments
* Revert changes to test outputs
* Refactor installation of wasmtime/wasm-tools on CI
We've got official actions for installation Wasmtime and wasm-tools in
the bytecodealliance organization which should help make this
installation step a bit more readable.
* Change the directory of the adapter
* Update descriptions
* build and test `wasm32-wasi-preview2` target
This updates `wasi-libc` to include
https://github.com/WebAssembly/wasi-libc/pull/457, which adds preliminary
support for the new `wasm32-wasi-preview2` target.
It also adds support for testing the new target using Wasmtime 16.0.0 and
`wit-component`. Note that Wasmtime produces different output when reporting
errors for Preview 2 components than it does for Preview 1 modules, so I've
added a few .expected files specific to Preview 2.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* test all three targets
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
```
error: Server does not allow request for unadvertised object f992bcc08219edb283d2ab31dd3871a4a0e8220e
fatal: Fetched in submodule path 'src/config', but it did not contain f992bcc08219edb283d2ab31dd3871a4a0e8220e. Direct fetching of that commit failed.
```
This patch enables using latest version of wasmtime for testing. This
should also make it possible to running tests for wasm32-wasi-threads
in the future.
The virtual-environments documents are at: https://github.com/actions/virtual-environments .
Currently, the `windows-latest` == `windows-2022`
We currently are using `Microsoft Visual Studio\2019` installed with `windows-2019` and that not installed in `windows-2022`
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
When building wasi-sdk locally, we doesn't need so verbose output that not easily distinguish
the error command from the normal command. So refactoring -v option into workflow scripts
We still build with the `/opt/wasi-sdk` prefix but we don't actually
install into that directory. Instead we install into `build/install`.
This means you don't need to run `sudo make` and you don't end up with
a all the build output being owned by root.
The main different is that this requires the `--sysroot` flag when
running clang to build the testsuite.
* Make build work on Windows
* re-enable windows build workflow
* fix make check
* fix tar_from_installation.sh and dist target
* Review fixups
* No -j4 on Windows
* Get rid of -j4 everywhere in Makefile
* ci: add separate job to run docker build script
so that it doesn't bitrot
* github ci: try turning windows back on?
* workflow: run make in bash (for windows)
* windows doesnt have sudo
* github ci: windows is still broken, will have to fix later
* deb build: in wrong directory to mv build/wasi-sdk...deb
* ci: make filenames of uploaded artifacts more clear
* fix typo in deb creation
- Don't build pushes to non-main branches, only build the PR itself.
- Remove special windows llvm-nm installation
The SDK build process itself builds llvm so we don't need to download
it.
- Make the `name` the first item in each step. I peronally find
this more readable.
* build: put artifacts into dist/. workflow: upload contents of dist/.
* review feedback
* CI workflow: add comment explaining artifact name
* checkout action to v1
* typo fix
Alex Crichton
Changqing Jing
Joel Dice
YAMAMOTO Takashi
Cheng Shao
Anuraag Agrawal
Cheng Shao
Yonggang Luo
Pat Hickey
Vladimir Vukicevic
Sam Clegg
Dan Gohman