Recent changes have added various flags for customizing the
component-creation process in addition to supporting more flags being
forwarded to `wasm-ld`.
* Update LLVM to 18.1.2 and add wasm-component-ld
This commit has two intertwined changes within it. First the LLVM
submodule is updated to the 18.1.2 release branch. This alone would
cause CI and tests to fail due to differing behavior for the
`wasm32-wasip2` target. To fix these test failures the
`wasm-component-ld` tool is added to the build. This tool is a
Rust-written tool and installed via `cargo install` as part of the build
at a pinned version written in the `Makefile`. This linker, used for
components, is then used for the `wasm32-wasip2` target.
Tests and CI are then updated to skip the need to have `wasm-tools` or
the adapter for WASI when making components since that's now the job of
`wasm-component-ld`. This then necessitated some changes to tests too.
* Add back accidentally deleted CI
* Inherit tools on PATH on Windows
* Add Rust to docker build
* Put rust in a different location in docker
* Try to fix permissions
* Review comments
* Revert changes to test outputs
As noted in #372, some users may need to identify which version of
wasi-sdk they are using after it has been downloaded. There are many
ways to solve this, but the one I chose here is to dump the wasi-sdk
version into a `VERSION` file and follow that up with several dependency
versions that may be helpful to note when troubleshooting. Ideally
someone could paste the contents of that file when filing a bug. If we
adopt this approach, this fixes#372.
In the past, I've had trouble keeping the current set of version scripts
to correctly output the version, especially in light of how we append
Git suffixes for a non-tagged commit and dirty state. This change
replaces those scripts with a single Python one which, though much more
wordy than the previous one, may be easier for contributors to read and
modify. The original scripts relied on Perl; this one relies on Python,
which seems like a fair exchange. Having a single script also makes it
easier to solve #372.
* Explicitly set LIBCXXABI_USE_LLVM_UNWINDER off
Clang trunk has recently turned it on, and also now fails the build when
libunwind is not built at the same time.
* Bump LLVM to 18.1.0
---------
Co-authored-by: Mike Hommey <mh@glandium.org>
Currently the `bin` directory installed by wasi-sdk is not currently
suitable for putting in `$PATH` in all cases because it can shadow a
system-installed `clang` executable which is intended for native
binaries. For Linux distributions with gcc-based cross-compilers I've
often seen the pattern where they're installed as `$target-gcc` and so
I've taken a leaf out of their books to do that here as well.
This commit adds, currently alongside the preexisting `clang`
executable, target-prefixed executables such as `wasm32-wasi-clang` and
`wasm32-wasi-clang++`. These executables are symlinks to the `clang`
executable itself in the same manner that `clang++` is a symlink to
`clang` itself.
I'll also note that this doesn't fix the problem of "add the wasi-sdk
bin dir to PATH" because `clang` and other prominent executables are
still there. My hope though is that this opens up a future refactoring
for doing so.
* Start renaming preview1 to p1 and preview2 to p2
This commit is a reflection of WebAssembly/wasi-libc#478 into this
repository where a few changes are happening:
* A new `wasm32-wasip1` sysroot is prepared matching `wasm32-wasi`
* A new `wasm32-wasip1-threads` sysroot is prepared matching `wasm32-wasi-threads`
* The `wasm32-wasi-preview2` target is renamed `wasm32-wasip2`
I've done a bit of makefile refactoring to deduplicate things a bit now
that there's a number of targets being built.
The long-term goal would be to remove the `wasm32-wasi` and
`wasm32-wasip1-threads` targets, but that's not proposed just yet at
this time.
* Use $(CLANG_VERSION) instead of hardcoding
As of LLVM 17, which includes https://reviews.llvm.org/D152168, libcxx has
combined the old `_LIBCPP_HAS_NO_FILESYSTEM_LIBRARY` and
`_LIBCPP_HAS_NO_FSTREAM` preprocessor symbols into a single
`_LIBCPP_HAS_NO_FILESYSTEM` symbol, which means there's no longer any way to
enable `<fstream>` without enabling `<filesystem>`.
The solution is to set `-DLIBCXX_ENABLE_FILESYSTEM:BOOL=ON` and update
`wasi-libc`, which includes stubs for the functions required by libcxx's
`<filesystem>` implementation.
Fixes#373
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* build and test `wasm32-wasi-preview2` target
This updates `wasi-libc` to include
https://github.com/WebAssembly/wasi-libc/pull/457, which adds preliminary
support for the new `wasm32-wasi-preview2` target.
It also adds support for testing the new target using Wasmtime 16.0.0 and
`wit-component`. Note that Wasmtime produces different output when reporting
errors for Preview 2 components than it does for Preview 1 modules, so I've
added a few .expected files specific to Preview 2.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* test all three targets
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
In #321, some OSX-specific `Makefile` additions to `LLVM_CMAKE_FLAGS`
were skipped unless `make` is run on a Darwin OS. This allowed building
wasi-sdk for aarch64. But, as reported in #336, this also broke
arm64/x86_64 universal binaries that are built during CI.
The reason for this is that CI's `main.yml` overrides `LLVM_CMAKE_FLAGS`
to add caching but `make` will not append to a variable set on the
command line. This changes uses the `override` keyword to append to
such a variable, as suggested [here].
[here]: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
It looks like some macOS-specific CMake options to LLVM end up forcing
an x86_64 build of some blake3 intrinsics (or something like that) which
causes the build to fail on AArch64 Linux, for example. I found, though,
that when removing these options it was then possible to build a
toolchain for AArch64 Linux.
This commit passes through `EXTRA_CFLAGS` to the libcxx build, along
with `EXTRA_CXXFLAGS`. This models after the pattern in `wasi-libc` for
passing these flags to thread more flags through.
* llvm-project: update to 16.0.0 release
This changes updates the `src/llvm-project` submodule to the `HEAD` of
`release/16.x`, the same commit used to [release] the LLVM 16.0.0
binaries.
[release]: https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.0
* fix: use only Clang's major version in install prefix
Due to [a change] in LLVM, Clang will expect to find the
`libclang_rt.builtins-wasm32.a` file in a path that only contains the
major version (`16`) instead of the entire version (`16.0.0`) as was
previously the case. This change modifies the `CMAKE_INSTALL_PREFIX` to
use Clang's major version only.
[a change]: https://reviews.llvm.org/D125860
* review: only use `llvm_version_major.sh`
Since the `Makefile` can get by with only knowing Clang's major version,
this change removes `llvm_version.sh` and sets `CLANG_VERSION` to use
only the major part.
This change switches the MacOS build to target 10.12, an older version
than currently supported. This would close#127. @sbc100 mentions that
this is the setting used by Emscripten and related projects.
* Add a variation of wasi-sdk.cmake for pthread-using apps
* Install wasi-sdk-pthread.cmake
* Build wasi-libc with THREAD_MODEL=posix as well
* wasi-sdk-pthread.cmake: target rename to wasm32-wasi-threads
after https://github.com/WebAssembly/wasi-libc/pull/381
The version 11.0 is chosen because Apple seems providing
support for two previous releases.
(As of writing this, the latest is 13.)
REVISIT: as we don't need any brand-new OS features, it
might make more sense to specify even older version.
This might explain the following error i've seen with wasi-sdk-17.0
release binary on macos-11.0:
```
Run Build Command(s):/usr/bin/make -f Makefile cmTC_254ef/fast && /Applications/Xcode_13.2.1.app/Contents/Developer/usr/bin/make -f CMakeFiles/cmTC_254ef.dir/build.make CMakeFiles/cmTC_254ef.dir/build
Building C object CMakeFiles/cmTC_254ef.dir/testCCompiler.c.obj
/Users/runner/work/toywasm/toywasm/.wasi-sdk-17.0/bin/clang -o CMakeFiles/cmTC_254ef.dir/testCCompiler.c.obj -c /Users/runner/work/toywasm/toywasm/build.wasm/CMakeFiles/CMakeScratch/TryCompile-aWXHXM/testCCompiler.c
dyld: Symbol not found: __ZTTNSt3__114basic_ifstreamIcNS_11char_traitsIcEEEE
Referenced from: /Users/runner/work/toywasm/toywasm/.wasi-sdk-17.0/bin/clang (which was built for Mac OS X 12.0)
Expected in: /usr/lib/libc++.1.dylib
in /Users/runner/work/toywasm/toywasm/.wasi-sdk-17.0/bin/clang
make[1]: *** [CMakeFiles/cmTC_254ef.dir/testCCompiler.c.obj] Abort trap: 6
make: *** [cmTC_254ef/fast] Error 2
```
References:
https://cmake.org/cmake/help/latest/variable/CMAKE_OSX_DEPLOYMENT_TARGET.html
These dependencies are not needed in wasi-sdk, and if they're enabled,
the msys2 dll depenndencies will leak into build products. Also
statically links libstdc++, required for the windows build.
This is the new method to build runtimes as of LLVM 12, and the previous
standalone method is deprecated since LLVM 14 and unsupported since
LLVM 15 (6f17768e11)
Building libcxx and libcxxabi in one go simplifies the setup necessary
as:
- tests CMakeLists.txt have some wrong dependencies which would need
to be worked around.
- LIBCXXABI_LIBCXX_INCLUDES is not recognized when building separately
and passing -I paths via the command line yields msys path munging
problems on Windows.
This effectively drops support for LLVM 11 and older.
When building wasi-sdk locally, we doesn't need so verbose output that not easily distinguish
the error command from the normal command. So refactoring -v option into workflow scripts
With the following building commands:
```
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64
set MSYS2_PATH_TYPE=inherit
set MSYSTEM=MINGW32
set CC=cl
set CXX=cl
set "HOME=%CD%"
bash.exe --login -c "make package"
bash.exe --login -c "make check"
cmd /k
```
Fixes following error:
c:\wasi-sdk\bin\clang++.exe --target=wasm32-wasi -DNDEBUG -D_LIBCPP_BUILDING_LIBRARY -D_LIBCPP_DISABLE_NEW_DELETE_DEFINITIONS -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -Iinclude/c++build -fdebug-prefix-map=C:\work\wasi-sdk=wasisdk;\v13.5g73d30af917c4 --sysroot=c;C:\CI-Tools\msys64\wasi-sdk\share\wasi-sysroot -fPIC -DLIBCXX_BUILDING_LIBCXXABI -IC:/work/wasi-sdk/build/libcxx/include/c++/v1 -std=c++20 -MD -MT src/CMakeFiles/cxx_static.dir/regex.cpp.obj -MF src\CMakeFiles\cxx_static.dir\regex.cpp.obj.d -o src/CMakeFiles/cxx_static.dir/regex.cpp.obj -c C:/work/wasi-sdk/src/llvm-project/libcxx/src/regex.cpp
In file included from C:/work/wasi-sdk/src/llvm-project/libcxx/src/regex.cpp:9:
In file included from C:/work/wasi-sdk/build/libcxx/include/c++/v1\regex:766:
In file included from C:/work/wasi-sdk/build/libcxx/include/c++/v1\__debug:14:
In file included from C:/work/wasi-sdk/build/libcxx/include/c++/v1\iosfwd:98:
C:/work/wasi-sdk/build/libcxx/include/c++/v1\wchar.h:119:15: fatal error: 'wchar.h' file not found
#include_next <wchar.h>
^~~~~~~~~
This makes use of https://reviews.llvm.org/D76653, which is in clang 11
and newer, to fix#53. With this, clang can find the sysroot relative to
its own path.
Fixes#58.
* fix deb build:
change makefile and deb_from_installation to invoke it with the new
build directory (changed in #143).
deb_from_installation will exit 0 if dpkg-deb missing so we dont try
to run it as part of `make package` on non-deb systems, but other errors
should cause the makefile to fail, which should catch regressions like
this sooner in the future.
* thank you fgsch for style fix
* Update how wasi-sdk uses cmake
Change `CMAKE_SYSTEM_NAME` to `WASI`, use a `CMAKE_MODULE_PATH`,
remove `CACHE` usage, and fix the compiler autodetection code so that
the "compiler works" hacks are no longer needed, following the advice in
[this comment](https://gitlab.kitware.com/cmake/cmake/-/issues/19223#note_567327).
This is hopefully a step towards re-submitting WASI support to upstream cmake.
* Add --sysroot to compiler-rt's C flags.
This ensures that it finds the built sysroot when testing whether the C
compiler works.
* Fixes.
* Use the just-built ar rather than the host ar.
* Use -DCMAKE_C_COMPILER_WORKS=ON instead of stubbing out archive files.
* Use -DCMAKE_CXX_COMPILER_WORKS=ON too.
* Restore the `--sysroot` argument when building compiler-rt.
compiler-rt depends on some libc headers, so it needs a sysroot.
* Add sysroot arguments for libc++ and libc++abi too.
* Also set CMAKE_AR.
It appears with the removal of the AR checks cmake is no longer
autodetecting CMAKE_AR, so set it explicitly.
* Remove --no-threads from wasi-sdk.cmake.
It was added to work around this bug:
https://bugs.llvm.org/show_bug.cgi?id=41508
which was fixed in LLVM 9.0.
* Remove LLVM 8.0 workarounds.
We still build with the `/opt/wasi-sdk` prefix but we don't actually
install into that directory. Instead we install into `build/install`.
This means you don't need to run `sudo make` and you don't end up with
a all the build output being owned by root.
The main different is that this requires the `--sysroot` flag when
running clang to build the testsuite.
* Make build work on Windows
* re-enable windows build workflow
* fix make check
* fix tar_from_installation.sh and dist target
* Review fixups
* No -j4 on Windows
* Get rid of -j4 everywhere in Makefile
* build: put artifacts into dist/. workflow: upload contents of dist/.
* review feedback
* CI workflow: add comment explaining artifact name
* checkout action to v1
* typo fix
Alex Crichton
Andrew Brown
Changqing Jing
Joel Dice
Arvid Norlander
Catherine
YAMAMOTO Takashi
Cheng Shao
Dan Gohman
Mike Hommey
Vladislav Nepogodin
John Ericson
Yonggang Luo
Mike Hommey
Pat Hickey
Dan Gohman
Sam Clegg
Vladimir Vukicevic