Based on my experience with range-set-blaze, a data structure project, here are the decisions I recommend, described one at a time. To avoid wishy-washiness, I’ll express them as rules.
In 2019, Docker co-creator Solomon Hykes tweeted:
If WASM+WASI existed in 2008, we wouldn’t have needed to created Docker. That’s how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let’s hope WASI is up to the task.
Today, if you follow technology news, you’ll see optimistic headlines like these:
If WASM WASI were truly ready and useful, everyone would already be using it. The fact that we keep seeing these headlines suggests it’s not yet ready. In other words, they wouldn’t need to keep insisting that WASM WASI is ready if it really were.
As of WASI Preview 1, here is how things stand: You can access some file operations, environment variables, and have access to time and random number generation. However, there is no support for networking.
WASM WASI might be useful for certain AWS Lambda-style web services, but even that’s uncertain. Because wouldn’t you prefer to compile your Rust code natively and run twice as fast at half the cost compared to WASM WASI?
Maybe WASM WASI is useful for plug ins and extensions. In genomics, I have a Rust extension for Python, which I compile for 25 different combinations (5 versions of Python across 5 OS targets). Even with that, I don’t cover every possible OS and chip family. Could I replace those OS targets with WASM WASI? No, it would be too slow. Could I add WASM WASI as a sixth “catch-all” target? Maybe, but if I really need portability, I’m already required to support Python and should just use Python.
So, what is WASM WASI good for? Right now, its main value lies in being a step toward running code in the browser or on embedded systems.
In Rule 1, I mentioned “OS targets” in passing. Let’s look deeper into Rust targets — essential information not just for WASM WASI, but also for general Rust development.
On my Windows machine, I can compile a Rust project to run on Linux or macOS. Similarly, from a Linux machine, I can compile a Rust project to target Windows or macOS. Here are the commands I use to add and check the Linux target to a Windows machine:
rustup target add x86_64-unknown-linux-gnu
cargo check --target x86_64-unknown-linux-gnu
Aside: While
cargo checkverifies that the code compiles, building a fully functional executable requires additional tools. To cross-compile from Windows to Linux (GNU), you’ll also need to install the Linux GNU C/C++ compiler and the corresponding toolchain. That can be tricky. Fortunately, for the WASM targets we care about, the required toolchain is easy to install.
To see all the targets that Rust supports, use the command:
rustc --print target-list
It will list over 200 targets including x86_64-unknown-linux-gnu, wasm32-wasip1, and wasm32-unknown-unknown.
Target names contain up to four parts: CPU family, vendor, OS, and environment (for example, GNU vs LVMM):
Now that we understand something of targets, let’s go ahead and install the one we need for WASM WASI.
To run our Rust code on WASM outside of a browser, we need to target wasm32-wasip1 (32-bit WebAssembly with WASI Preview 1). We’ll also install WASMTIME, a runtime that allows us to run WebAssembly modules outside of the browser, using WASI.
rustup target add wasm32-wasip1
cargo install wasmtime-cli
To test our setup, let’s create a new “Hello, WebAssembly!” Rust project using cargo new. This initializes a new Rust package:
cargo new hello_wasi
cd hello_wasi
Edit src/main.rs to read:
fn main()
#[cfg(not(target_arch = "wasm32"))]
println!("Hello, world!");
#[cfg(target_arch = "wasm32")]
println!("Hello, WebAssembly!");
Aside: We’ll look deeper into the
#[cfg(...)]attribute, which enables conditional compilation, in Rule 4.
Now, run the project with cargo run, and you should see Hello, world! printed to the console.
Next, create a .cargo/config.toml file, which specifies how Rust should run and test the project when targeting WASM WASI.
[target.wasm32-wasip1]
runner = "wasmtime run --dir ."
Aside: This
.cargo/config.tomlfile is different from the mainCargo.tomlfile, which defines your project’s dependencies and metadata.
Now, if you say:
cargo run --target wasm32-wasip1
You should see Hello, WebAssembly!. Congratulations! You’ve just successfully run some Rust code in the container-like WASM WASI environment.
Now, let’s investigate #[cfg(...)]—an essential tool for conditionally compiling code in Rust. In Rule 3, we saw:
fn main()
#[cfg(not(target_arch = "wasm32"))]
println!("Hello, world!");
#[cfg(target_arch = "wasm32")]
println!("Hello, WebAssembly!");
The #[cfg(...)] lines tell the Rust compiler to include or exclude certain code items based on specific conditions. A “code item” refers to a unit of code such as a function, statement, or expression.
With #[cfg(…)] lines, you can conditionally compile your code. In other words, you can create different versions of your code for different situations. For example, when compiling for the wasm32 target, the compiler ignores the #[cfg(not(target_arch = "wasm32"))] block and only includes the following:
fn main()
println!("Hello, WebAssembly!");
You specify conditions via expressions, for example, target_arch = "wasm32". Supported keys include target_os and target_arch. See the Rust Reference for the full list of supported keys. You can also create expressions with Cargo features, which we will learn about in Rule 6.
You may combine expressions with the logical operators not, any, and all. Rust’s conditional compilation doesn’t use traditional if...then...else statements. Instead, you must use #[cfg(...)] and its negation to handle different cases:
#[cfg(not(target_arch = "wasm32"))]
...
#[cfg(target_arch = "wasm32")]
...
To conditionally compile an entire file, place #![cfg(...)] at the top of the file. (Notice the “!”). This is useful when a file is only relevant for a specific target or configuration.
You can also use cfg expressions in Cargo.toml to conditionally include dependencies. This allows you to tailor dependencies to different targets. For example, this says “depend on Criterion with Rayon when not targeting wasm32”.
[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
criterion = version = "0.5.1", features = ["rayon"]
Aside: For more information on using
cfgexpressions inCargo.toml, see my article: Nine Rust Cargo.toml Wats and Wat Nots: Master Cargo.toml formatting rules and avoid frustration | Towards Data Science (medium.com).
It’s time to try to run your project on WASM WASI. As described in Rule 3, create a .cargo/config.toml file for your project. It tells Cargo how to run and test your project on WASM WASI.
[target.wasm32-wasip1]
runner = "wasmtime run --dir ."
Next, your project — like all good code — should already contain tests. My range-set-blaze project includes, for example, this test:
#[test]
fn insert_255u8()
let range_set_blaze = RangeSetBlaze::<u8>::from_iter([255]);
assert!(range_set_blaze.to_string() == "255..=255");
Let’s now attempt to run your project’s tests on WASM WASI. Use the following command:
cargo test --target wasm32-wasip1
If this works, you may be done — but it probably won’t work. When I try this on range-set-blaze, I get this error message that complains about using Rayon on WASM.
error: Rayon cannot be used when targeting wasi32. Try disabling default features.
--> C:\Users\carlk\.cargo\registry\src\index.crates.io-6f17d22bba15001f\criterion-0.5.1\src\lib.rs:31:1
|
31 | compile_error!("Rayon cannot be used when targeting wasi32. Try disabling default features.");
To fix this error, we must first understand Cargo features.
To resolve issues like the Rayon error in Rule 5, it’s important to understand how Cargo features work.
In Cargo.toml, an optional [features] section allows you to define different configurations, or versions, of your project depending on which features are enabled or disabled. For example, here is a simplified part of the Cargo.toml file from the Criterion benchmarking project:
[features]
default = ["rayon", "plotters", "cargo_bench_support"]
rayon = ["dep:rayon"]
plotters = ["dep:plotters"]
html_reports = []
cargo_bench_support = [][dependencies]
#...
# Optional dependencies
rayon = version = "1.3", optional = true
plotters = version = "^0.3.1", optional = true, default-features = false, features = [
"svg_backend",
"area_series",
"line_series",
]
This defines four Cargo features: rayon, plotters, html_reports, and cargo_bench_support. Since each feature can be included or excluded, these four features create 16 possible configurations of the project. Note also the special default Cargo feature.
A Cargo feature can include other Cargo features. In the example, the special default Cargo feature includes three other Cargo features — rayon, plotters, and cargo_bench_support.
A Cargo feature can include a dependency. The rayon Cargo feature above includes the rayon crate as a dependent package.
Moreover, dependent packages may have their own Cargo features. For example, the plotters Cargo feature above includes the plotters dependent package with the following Cargo features enabled: svg_backend, area_series, and line_series.
You can specify which Cargo features to enable or disable when running cargo check, cargo build, cargo run, or cargo test. For instance, if you’re working on the Criterion project and want to check only the html_reports feature without any defaults, you can run:
cargo check --no-default-features --features html_reports
This command tells Cargo not to include any Cargo features by default but to specifically enable the html_reports Cargo feature.
Within your Rust code, you can include/exclude code items based on enabled Cargo features. The syntax uses #cfg(…), as per Rule 4:
#[cfg(feature = "html_reports")]
SOME_CODE_ITEM
With this understanding of Cargo features, we can now attempt to fix the Rayon error we encountered when running tests on WASM WASI.
When we tried running cargo test --target wasm32-wasip1, part of the error message stated: Criterion ... Rayon cannot be used when targeting wasi32. Try disabling default features. This suggests we should disable Criterion’s rayon Cargo feature when targeting WASM WASI.
To do this, we need to make two changes in our Cargo.toml. First, we need to disable the rayon feature from Criterion in the [dev-dependencies] section. So, this starting configuration:
[dev-dependencies]
criterion = version = "0.5.1", features = ["html_reports"]
becomes this, where we explicitly turn off the default features for Criterion and then enable all the Cargo features except rayon.
[dev-dependencies]
criterion = version = "0.5.1", features = [
"html_reports",
"plotters",
"cargo_bench_support"],
default-features = false
Next, to ensure rayon is still used for non-WASM targets, we add it back in with a conditional dependency in Cargo.toml as follows:
[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
criterion = version = "0.5.1", features = ["rayon"]
In general, when targeting WASM WASI, you may need to modify your dependencies and their Cargo features to ensure compatibility. Sometimes this process is straightforward, but other times it can be challenging — or even impossible, as we’ll discuss in Rule 8.
Aside: In the next article in this series — about WASM in the Browser — we’ll go deeper into strategies for fixing dependencies.
After running the tests again, we move past the previous error, only to encounter a new one, which is progress!
#[test]
fn test_demo_i32_len()
assert_eq!(demo_i32_len(i32::MIN..=i32::MAX), u32::MAX as usize + 1);
^^^^^^^^^^^^^^^^^^^^^ attempt to compute
`usize::MAX + 1_usize`, which would overflow
The compiler complains that u32::MAX as usize + 1 overflows. On 64-bit Windows the expression doesn’t overflow because usize is the same as u64 and can hold u32::MAX as usize + 1. WASM, however, is a 32-bit environment so usize is the same as u32 and the expression is one too big.
The fix here is to replace usize with u64, ensuring that the expression doesn’t overflow. More generally, the compiler won’t always catch these issues, so it’s important to review your use of usize and isize. If you’re referring to the size or index of a Rust data structure, usize is correct. However, if you’re dealing with values that exceed 32-bit limits, you should use u64 or i64.
Aside: In a 32-bit environment, a Rust array,
Vec,BTreeSet, etc., can only hold up to 2³²−1=4,294,967,295 elements.
So, we’ve fixed the dependency issue and addressed a usize overflow. But can we fix everything? Unfortunately, the answer is no.
WASM WASI Preview 1 (the current version) supports file access (within a specified directory), reading environment variables, and working with time and random numbers. However, its capabilities are limited compared to what you might expect from a full operating system.
If your project requires access to networking, asynchronous tasks with Tokio, or multithreading with Rayon, Unfortunately, these features aren’t supported in Preview 1.
Fortunately, WASM WASI Preview 2 is expected to improve upon these limitations, offering more features, including better support for networking and possibly asynchronous tasks.
So, your tests pass on WASM WASI, and your project runs successfully. Are you done? Not quite. Because, as I like to say:
If it’s not in CI, it doesn’t exist.
Continuous integration (CI) is a system that can automatically run your tests every time you update your code, ensuring that your code continues to work as expected. By adding WASM WASI to your CI, you can guarantee that future changes won’t break your project’s compatibility with the WASM WASI target.
In my case, my project is hosted on GitHub, and I use GitHub Actions as my CI system. Here’s the configuration I added to .github/workflows/ci.yml to test my project on WASM WASI:
test_wasip1:
name: Test WASI P1
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Rust
uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
targets: wasm32-wasip1
- name: Install Wasmtime
run: |
curl https://wasmtime.dev/install.sh -sSf | bash
echo "$HOME/.wasmtime/bin" >> $GITHUB_PATH
- name: Run WASI tests
run: cargo test --verbose --target wasm32-wasip1
By integrating WASM WASI into CI, I can confidently add new code to my project. CI will automatically test that all my code continues to support WASM WASI in the future.