This backports https://github.com/llvm/llvm-project/pull/209282 to resolve the
remaining LLVM issues with Wasm exceptions and shared libraries.
I've updated `tests/CMakeLists.txt` to run all p2 and p3 tests as shared
libraries as well as normal executables. As usual my CMake skills are
underwhelming; very open to feedback.
Currently a few of the p3 tests are failing in shared library mode, apparently
due to TLS-related issues. I'll work on debugging those.
This commit creates a separate sysroot for coop threads for the
`wasm32-wasip3` target. This is explicitly labeled with `experimental`
in the name and isn't passed anywhere by default. The intention is to
assist with testing of this sysroot by making it reasonably easy to use
it and test that everything works. Using it requires passing `--sysroot`
to the folder generated here, and a simple "hello world" with threads
completes locally. This necessitated a minor update to an LLVM patch
which reflects the updated state of a PR upstream.
This is a work-in-progress to get things enabled. The main bulk of the
change here is updating LLVM itself to tip-of-tree as of yesterday
morning. This removes two `*.patch` files because they've landed, and
adds another `*.patch` file to work around a new issue that's cropped
up. I've also included a `*.patch` for
https://github.com/llvm/llvm-project/pull/200855, too.
This isn't ready yet to land as it's still failing to build/link/etc,
and I'll comment some more below.
This commit collects together some LLVM PRs, some changes in the build
configuration here, and some thoughts from #565 and related issues.
Specifically the changes here are:
* The patch for llvm/llvm-project#168449 is updated to its upstream
(unlanded) form.
* Patches for the (landed) llvm/llvm-project#185770 and
llvm/llvm-project#185775 are added.
* The `WASI_SDK_EXCEPTIONS` configuration is now either `ON`, `OFF`, or
`DUAL`. The default depends on the version of Clang in use, where
23.0.0+ (which isn't released officially yet) will be `DUAL` and
otherwise it's `OFF`. CI for our custom-built patched toolchain defaults
to `DUAL`.
* In `DUAL` mode libcxx is built twice into two different directories,
once with exceptions and once without. This is supported by LLVM patches
and means that Clang will select the right set of libraries based on
compiler flags.
The end result here is that the produced toolchain from this repository,
by default, supports C++ exceptions. Additionally if exceptions-related
flags are not passed then the final binary will not use C++ exceptions
nor require the wasm exception-handling proposal.
There's still follow-up work from #565, such as:
* Subjectively it feels wordy to pass `-fwasm-exceptions` vs
`-fexceptions`.
* Personally I think `-mllvm -wasm-use-legacy-eh=false` should become
the default upstream.
* Subjectively I don't think that `-lunwind` should be necessary and it
should be injected automatically with `-fwasm-exceptions` (or
`-fexceptions`).
* Shared libraries for exceptions remain disabled due to build errors I
do not personally know how to resolve.
I'll file follow-up issues for these once this has landed since they're
more minor compared to the main body of "anything works".
Closes#334Closes#565
---------
Co-authored-by: Joel Dice <joel.dice@akamai.com>
> **Note**: this PR is me weaving together the work of many others. For
example @yerzham's and @cpetig's work on #565 pretty much shaped this
PR. Thank you!
This commit adds initial configuration support for building wasi-sdk
with support for C++ exceptions in WebAssembly. A small test is included
which is exercised in CI at this time, but otherwise this does not
update CI to actually ship sdk builds with C++ exceptions enabled.
Instead the intention here is to make it easier to test builds with
support for C++ exceptions and centralize planning/development around
this.
The goal here is to get things compiling to the point that applications
can be compiled. I haven't thoroughly tested C++ exceptions and as
evident in this PR it still requires changes in LLVM. Some small logic
is added here to apply a `*.patch` file to LLVM to avoid needing a new
submodule fork or waiting for upstream changes. The intention is that
this `*.patch` is short-lived once the changes are officially merged
into LLVM itself.
The `*.patch` included here contains llvm/llvm-project#168449 as well as
another minor edit I found was necessary to get things compiling
locally. Given the discussion on that PR it looks like once LLVM is
updated with that merged the extra part of the `*.patch` won't be
necessary.
This PR notably does not enable shared libraries when exceptions are
enabled. I don't know enough about how things are supposed to work to be
able to fully diagnose the compilation errors I'm seeing if shared
libraries are enabled. This is something I'd hope would be fixed before
actually shipping exceptions support.
This PR then additionally folds in [this gist][gist] for various bits of
build logic to this repository itself.
Finally, this PR includes some documentation about the current status of
exceptions and links to various tracking issues too.
[gist]: https://gist.github.com/yerzham/302efcec6a2e82c1e8de4aed576ea29d
This updates to include WebAssembly/wasi-libc#709 and then additionally
configures wasi-libc with `-DWASI_SDK_VERSION=...` to ensure that
wasi-sdk builds will now have `__wasi_sdk_major__` defined along with
`__wasi_sdk_version__`.
ClosesWebAssembly/wasi-libc#688
This commit updates the build of the `wasm32-wasip2` target to always
pass `-fPIC` to compilation of wasi-libc. This notably enables using
`libc.a` in dynamic linking situations instead of being forced to use
`libc.so`. While many applications likely want to use `libc.so` I've
found it more flexible if objects are compatible with as many builds as
possible. This will enable, for example, building shared libraries in
Rust by default since Rust only ships `libc.a`, not `libc.so`, in the
pre-built sysroot. This behavior additionally matches the Rust
`wasm32-wasip2` target where `-fPIC` is enabled by default there.
This commit enables thread support in libc++ for all thread models,
enabling C++ applications that use threading APIs like `<atomic>` but do
not spawn threads (e.g. Clang) to be built with minimal changes.
Fixes#546.
Depends on WebAssembly/wasi-libc#602.
Update from LLVM 20.1.1 to 20.1.8. This release contains various bug
fixes and notably for wasi-sdk includes llvm/llvm-project#146574, which
fixes inline asm vector operands.
* Update to the latest wasi-libc.
This contains two significant changes:
- Implement a stub pthreads library for `THREAD_MODEL=single` (#518)
- Initial FTS support (#522)
* Update to the latest wasi-libc with build fix
This update includes a build fix for shared library builds.
---------
Co-authored-by: Dan Gohman <dev@sunfishcode.online>
To include https://github.com/WebAssembly/wasi-libc/pull/517,
which is necessary to avoid modifying host compiler's resource
directory.
At least on my environment (macOS if it matters), without the above
mentioned fix, gmake tries to create the directory for wasi compiler-rt
during wasi-libc build. I'm not sure why it didn't happen for others.
cf. https://github.com/WebAssembly/wasi-sdk/pull/445
* Add LTO build of wasi-libc
* Add LTO build of libc++
* Update wasi-libc (to include the LTO support)
* Add a comment about /llvm-lto/${llvm_version} convention
* Use separate targets for wasi-libc with and without lto
A downside: this makes four more copies of wasi-libc source tree.
* Make LTO build optional
* Add DWARF debugging information to all artifacts by default
This commit adds DWARF debugging information with the `-g` compiler flag
to all WASI artifacts for wasi-sdk. The LLVM build itself does not have
debugging information, only the sysroot artifacts. This is intended to
assist with debugging. The main downside to this is binary size of
generated artifacts will, by default, be larger. Stripping debug
information from an artifact though involves removing custom sections
which is generally pretty easy to do through wasm tooling.
* Pass extra cflags to wasi-libc
* Fix tests from previous commit
* Update some expected error messages and remove some files with
duplicate error messages that are no longer needed.
* Remove undefined behavior in `stat.c` where padding bytes were being
compared.
* Update LLVM to 18.1.2 and add wasm-component-ld
This commit has two intertwined changes within it. First the LLVM
submodule is updated to the 18.1.2 release branch. This alone would
cause CI and tests to fail due to differing behavior for the
`wasm32-wasip2` target. To fix these test failures the
`wasm-component-ld` tool is added to the build. This tool is a
Rust-written tool and installed via `cargo install` as part of the build
at a pinned version written in the `Makefile`. This linker, used for
components, is then used for the `wasm32-wasip2` target.
Tests and CI are then updated to skip the need to have `wasm-tools` or
the adapter for WASI when making components since that's now the job of
`wasm-component-ld`. This then necessitated some changes to tests too.
* Add back accidentally deleted CI
* Inherit tools on PATH on Windows
* Add Rust to docker build
* Put rust in a different location in docker
* Try to fix permissions
* Review comments
* Revert changes to test outputs
This updates the `wasi-libc` submodule to point to `main` and also:
- update tests/testcase.sh to use new wasm32-wasip2 target name
- update various tests/**/*.expected files to match Wasmtime output
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Explicitly set LIBCXXABI_USE_LLVM_UNWINDER off
Clang trunk has recently turned it on, and also now fails the build when
libunwind is not built at the same time.
* Bump LLVM to 18.1.0
---------
Co-authored-by: Mike Hommey <mh@glandium.org>
* Start renaming preview1 to p1 and preview2 to p2
This commit is a reflection of WebAssembly/wasi-libc#478 into this
repository where a few changes are happening:
* A new `wasm32-wasip1` sysroot is prepared matching `wasm32-wasi`
* A new `wasm32-wasip1-threads` sysroot is prepared matching `wasm32-wasi-threads`
* The `wasm32-wasi-preview2` target is renamed `wasm32-wasip2`
I've done a bit of makefile refactoring to deduplicate things a bit now
that there's a number of targets being built.
The long-term goal would be to remove the `wasm32-wasi` and
`wasm32-wasip1-threads` targets, but that's not proposed just yet at
this time.
* Use $(CLANG_VERSION) instead of hardcoding
As of LLVM 17, which includes https://reviews.llvm.org/D152168, libcxx has
combined the old `_LIBCPP_HAS_NO_FILESYSTEM_LIBRARY` and
`_LIBCPP_HAS_NO_FSTREAM` preprocessor symbols into a single
`_LIBCPP_HAS_NO_FILESYSTEM` symbol, which means there's no longer any way to
enable `<fstream>` without enabling `<filesystem>`.
The solution is to set `-DLIBCXX_ENABLE_FILESYSTEM:BOOL=ON` and update
`wasi-libc`, which includes stubs for the functions required by libcxx's
`<filesystem>` implementation.
Fixes#373
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* build and test `wasm32-wasi-preview2` target
This updates `wasi-libc` to include
https://github.com/WebAssembly/wasi-libc/pull/457, which adds preliminary
support for the new `wasm32-wasi-preview2` target.
It also adds support for testing the new target using Wasmtime 16.0.0 and
`wit-component`. Note that Wasmtime produces different output when reporting
errors for Preview 2 components than it does for Preview 1 modules, so I've
added a few .expected files specific to Preview 2.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* test all three targets
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
WebAssembly/wasi-libc#403 fixed an issue with `a_barrier` that should be
included in the next release of wasi-sdk. This change updates wasi-libc
to the latest `HEAD` of `main` to include it.
* llvm-project: update to 16.0.0 release
This changes updates the `src/llvm-project` submodule to the `HEAD` of
`release/16.x`, the same commit used to [release] the LLVM 16.0.0
binaries.
[release]: https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.0
* fix: use only Clang's major version in install prefix
Due to [a change] in LLVM, Clang will expect to find the
`libclang_rt.builtins-wasm32.a` file in a path that only contains the
major version (`16`) instead of the entire version (`16.0.0`) as was
previously the case. This change modifies the `CMAKE_INSTALL_PREFIX` to
use Clang's major version only.
[a change]: https://reviews.llvm.org/D125860
* review: only use `llvm_version_major.sh`
Since the `Makefile` can get by with only knowing Clang's major version,
this change removes `llvm_version.sh` and sets `CLANG_VERSION` to use
only the major part.
Joel Dice
Alex Crichton
Catherine
Dan Gohman
Yuta Saito
YAMAMOTO Takashi
Joel Dice
Andrew Brown