I forgot in #422 that by specifying `EXTRA_CFLAGS` to the wasi-libc
build that it would override the defaults of wasi-libc which is to pass
these two flags in. This passes them back in to ensure the default build
still has the same flags.
* Add DWARF debugging information to all artifacts by default
This commit adds DWARF debugging information with the `-g` compiler flag
to all WASI artifacts for wasi-sdk. The LLVM build itself does not have
debugging information, only the sysroot artifacts. This is intended to
assist with debugging. The main downside to this is binary size of
generated artifacts will, by default, be larger. Stripping debug
information from an artifact though involves removing custom sections
which is generally pretty easy to do through wasm tooling.
* Pass extra cflags to wasi-libc
* Fix tests from previous commit
* Update some expected error messages and remove some files with
duplicate error messages that are no longer needed.
* Remove undefined behavior in `stat.c` where padding bytes were being
compared.
This helps ensure the identically-named 32-bit tarfile doesn't overwrite the
64-bit one when we extract everything into a single directory.
I'll follow this up by making an updated wasi-sdk-22.0.m-mingw.tar.gz and adding
it to the wasi-sdk-22 release.
Fixes#326
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Recent changes have added various flags for customizing the
component-creation process in addition to supporting more flags being
forwarded to `wasm-ld`.
This commit fixes a minor mistake from #392 where the previous perl
script had a special case that if the "minor" version was zero then the
git hash information wouldn't be printed.
My release attempt in #408 did not go well because the release artifacts
were not named correctly. This fixes an issue described by
actions/checkout#290 where checkouts of annotated tags overwrite the
annotation which breaks `git describe`. That means that version
detection is broken in CI during releases which causes artifacts to have
the wrong information.
This applies the workaround described in that issue to `git fetch --tags
--force` after the checkout step to undo the overwrite done in the
checkout step.
* Update LLVM to 18.1.2 and add wasm-component-ld
This commit has two intertwined changes within it. First the LLVM
submodule is updated to the 18.1.2 release branch. This alone would
cause CI and tests to fail due to differing behavior for the
`wasm32-wasip2` target. To fix these test failures the
`wasm-component-ld` tool is added to the build. This tool is a
Rust-written tool and installed via `cargo install` as part of the build
at a pinned version written in the `Makefile`. This linker, used for
components, is then used for the `wasm32-wasip2` target.
Tests and CI are then updated to skip the need to have `wasm-tools` or
the adapter for WASI when making components since that's now the job of
`wasm-component-ld`. This then necessitated some changes to tests too.
* Add back accidentally deleted CI
* Inherit tools on PATH on Windows
* Add Rust to docker build
* Put rust in a different location in docker
* Try to fix permissions
* Review comments
* Revert changes to test outputs
This updates the `wasi-libc` submodule to point to `main` and also:
- update tests/testcase.sh to use new wasm32-wasip2 target name
- update various tests/**/*.expected files to match Wasmtime output
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
In #399, @glandium points out that newer versions of LLVM will place
their version information at a different path. This change adapts #399
to the new Python version script.
When running Git commands inside this Docker container (i.e., commands
that the `version.py` script needs for determining version information),
the Docker build would run into issues like:
```
fatal: detected dubious ownership in repository at '/workspace'
To add an exception for this directory, call:
git config --global --add safe.directory /workspace
```
This is due to an extra Git check that detects that the Docker user is
not the same one who owns the `.git` directory of this project. After
looking into this, the best solution the internet has to offer is to
thread the current user's UID and GID through the Docker image (i.e.,
the new `builder` user) and then `docker run --user ...`. This both
avoids the Git check but also seems to be considered a best practice in
some circles (?).
As noted in #372, some users may need to identify which version of
wasi-sdk they are using after it has been downloaded. There are many
ways to solve this, but the one I chose here is to dump the wasi-sdk
version into a `VERSION` file and follow that up with several dependency
versions that may be helpful to note when troubleshooting. Ideally
someone could paste the contents of that file when filing a bug. If we
adopt this approach, this fixes#372.
In the past, I've had trouble keeping the current set of version scripts
to correctly output the version, especially in light of how we append
Git suffixes for a non-tagged commit and dirty state. This change
replaces those scripts with a single Python one which, though much more
wordy than the previous one, may be easier for contributors to read and
modify. The original scripts relied on Perl; this one relies on Python,
which seems like a fair exchange. Having a single script also makes it
easier to solve #372.
This test looks to be asserting that `getrandom` never returns 256
consecutive zeros, but the way it's asserting that is summing up the
bytes and asserting the sum is nonzero. Due to this being a signed
addition, however, it's possible for the bytes to be nonzero and still
trigger the assert. Locally running this test in a loop took 30 or so
seconds before it triggered a failure.
I've updated the test to instead hunt for any entry which is not equal
to zero and then assert that something is not zero.
* Explicitly set LIBCXXABI_USE_LLVM_UNWINDER off
Clang trunk has recently turned it on, and also now fails the build when
libunwind is not built at the same time.
* Bump LLVM to 18.1.0
---------
Co-authored-by: Mike Hommey <mh@glandium.org>
Currently the `bin` directory installed by wasi-sdk is not currently
suitable for putting in `$PATH` in all cases because it can shadow a
system-installed `clang` executable which is intended for native
binaries. For Linux distributions with gcc-based cross-compilers I've
often seen the pattern where they're installed as `$target-gcc` and so
I've taken a leaf out of their books to do that here as well.
This commit adds, currently alongside the preexisting `clang`
executable, target-prefixed executables such as `wasm32-wasi-clang` and
`wasm32-wasi-clang++`. These executables are symlinks to the `clang`
executable itself in the same manner that `clang++` is a symlink to
`clang` itself.
I'll also note that this doesn't fix the problem of "add the wasi-sdk
bin dir to PATH" because `clang` and other prominent executables are
still there. My hope though is that this opens up a future refactoring
for doing so.
* Refactor installation of wasmtime/wasm-tools on CI
We've got official actions for installation Wasmtime and wasm-tools in
the bytecodealliance organization which should help make this
installation step a bit more readable.
* Change the directory of the adapter
* Update descriptions
* Start renaming preview1 to p1 and preview2 to p2
This commit is a reflection of WebAssembly/wasi-libc#478 into this
repository where a few changes are happening:
* A new `wasm32-wasip1` sysroot is prepared matching `wasm32-wasi`
* A new `wasm32-wasip1-threads` sysroot is prepared matching `wasm32-wasi-threads`
* The `wasm32-wasi-preview2` target is renamed `wasm32-wasip2`
I've done a bit of makefile refactoring to deduplicate things a bit now
that there's a number of targets being built.
The long-term goal would be to remove the `wasm32-wasi` and
`wasm32-wasip1-threads` targets, but that's not proposed just yet at
this time.
* Use $(CLANG_VERSION) instead of hardcoding
As of LLVM 17, which includes https://reviews.llvm.org/D152168, libcxx has
combined the old `_LIBCPP_HAS_NO_FILESYSTEM_LIBRARY` and
`_LIBCPP_HAS_NO_FSTREAM` preprocessor symbols into a single
`_LIBCPP_HAS_NO_FILESYSTEM` symbol, which means there's no longer any way to
enable `<fstream>` without enabling `<filesystem>`.
The solution is to set `-DLIBCXX_ENABLE_FILESYSTEM:BOOL=ON` and update
`wasi-libc`, which includes stubs for the functions required by libcxx's
`<filesystem>` implementation.
Fixes#373
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* build and test `wasm32-wasi-preview2` target
This updates `wasi-libc` to include
https://github.com/WebAssembly/wasi-libc/pull/457, which adds preliminary
support for the new `wasm32-wasi-preview2` target.
It also adds support for testing the new target using Wasmtime 16.0.0 and
`wit-component`. Note that Wasmtime produces different output when reporting
errors for Preview 2 components than it does for Preview 1 modules, so I've
added a few .expected files specific to Preview 2.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* test all three targets
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
```
error: Server does not allow request for unadvertised object f992bcc08219edb283d2ab31dd3871a4a0e8220e
fatal: Fetched in submodule path 'src/config', but it did not contain f992bcc08219edb283d2ab31dd3871a4a0e8220e. Direct fetching of that commit failed.
```
FreeBSD (like MacOS) does not support the `-executable` argument for
`find`. This commit checks if the OS is FreeBSD, and if so, uses the
same workaround as already in place on MacOS.
In #321, some OSX-specific `Makefile` additions to `LLVM_CMAKE_FLAGS`
were skipped unless `make` is run on a Darwin OS. This allowed building
wasi-sdk for aarch64. But, as reported in #336, this also broke
arm64/x86_64 universal binaries that are built during CI.
The reason for this is that CI's `main.yml` overrides `LLVM_CMAKE_FLAGS`
to add caching but `make` will not append to a variable set on the
command line. This changes uses the `override` keyword to append to
such a variable, as suggested [here].
[here]: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
This changes the front-page documentation to:
- use out-of-line links in more places
- mention the need for `libclang_rt.builtins-wasm32.a` when using
standard Clang
- mention how to use a `wasi-libc` sysroot
- explain the experimental status of `wasm32-wasi-threads
It looks like some macOS-specific CMake options to LLVM end up forcing
an x86_64 build of some blake3 intrinsics (or something like that) which
causes the build to fail on AArch64 Linux, for example. I found, though,
that when removing these options it was then possible to build a
toolchain for AArch64 Linux.
This patch enables using latest version of wasmtime for testing. This
should also make it possible to running tests for wasm32-wasi-threads
in the future.
Joel Dice
Anuraag (Rag) Agrawal
Alex Crichton
Philip Rying
Andrew Brown
Mike Hommey
Andy Li
Changqing Jing
Brett Cannon
YAMAMOTO Takashi
liang.he
Arvid Norlander
Kian Cross
Michael Lill
Casper Beyer
Cheng Shao