☰

Previous example: <-- web-sys: Wasm in Web Worker

Wasm audio worklet

This is an example of using threads inside specific worklets with WebAssembly, Rust, and wasm-bindgen, culminating in an oscillator demo. This demo should complement the parallel-raytrace example by demonstrating an alternative approach using ES modules with on-the-fly module creation.
_ wasm-bindgen Guide{target="_blank"}

Parallel Raytracing{target="_blank"}

Building the demo

One of the major gotchas with threaded WebAssembly is that Rust does not ship a precompiled target (e.g. standard library) which has threading support enabled. This means that you'll need to recompile the standard library with the appropriate rustc flags, namely -C target-feature=+atomics,+bulk-memory,+mutable-globals. Note that this requires a nightly Rust toolchain.
_ see more detailed instructions of the parallel-raytrace example{target="_blank"}

rust-toolchain.toml

[toolchain]
channel = "nightly"
components = ["rust-src"]
targets = ["wasm32-unknown-unknown"]
profile = "minimal"

Caveats

This example shares most of its caveats with the parallel-raytrace example. However, it tries to encapsulate worklet creation in a Rust module, so the application developer does not need to maintain custom JS code.
_ wasm-bindgen Guide{target="_blank"}

Browser Requirements

This demo should work in the latest Chrome and Safari versions at this time. Firefox does not support imports in worklet modules, which are difficult to avoid in this example, as importScripts is unavailable in worklets. Note that this example requires HTTP headers to be set like in parallel-raytrace. _ wasm-bindgen Guide{target="_blank"}

setup the project

cargo new wasm-audio-worklet --lib
cd wasm-audio-worklet
mkdir -p www/js www/html
  • Edit Cargo.toml
[lib]
crate-type = ["cdylib"]

[dependencies]
console_log = "0.2.0"
js-sys = "0.3.66"
wasm-bindgen = "0.2.89"
wasm-bindgen-futures = "0.4.39"

[dependencies.web-sys]
version = "0.3.66"
features = [
  "AudioContext",
  "AudioDestinationNode",
  "AudioWorklet",
  "AudioWorkletNode",
  "AudioWorkletNodeOptions",
  "Blob",
  "BlobPropertyBag",
  "Document",
  "Event",
  "HtmlInputElement",
  "HtmlLabelElement",
  "Url",
  "Window",
]


# Replace command line  
# cargo build --target wasm32-unknown-unknown -Z build-std=panic_abort,std
# and 
# export RUSTFLAGS='-C target-feature=+atomics,+bulk-memory,+mutable-globals'
# with:
[unstable]
build-std = ['std', 'panic_abort']

[build]
target = "wasm32-unknown-unknown"
rustflags = '-Ctarget-feature=+atomics,+bulk-memory,+mutable-globals'

The code

  • index.html
<!DOCTYPE html>
<html>
    <head>
        <title>Wasm audio worklet</title>
    </head>
    <body>
        <script type="module">
            import init, {web_main} from "../pkg/wasm_audio_worklet.js";
            async function run() {
                await init();
                web_main();
            }
            run();
        </script>
    </body>
</html>
  • index.js

No index.js for this example we have it directly in the html file.

  • worklet.js
registerProcessor("WasmProcessor", class WasmProcessor extends AudioWorkletProcessor {
    constructor(options) {
        super();
        let [module, memory, handle] = options.processorOptions;
        bindgen.initSync(module, memory);
        this.processor = bindgen.WasmAudioProcessor.unpack(handle);
    }
    process(inputs, outputs) {
        return this.processor.process(outputs[0][0]);
    }
});
  • Rust side
  • lib.rs
#![allow(unused)]
fn main() {
// src/lib.rs
mod dependent_module;
mod gui;
mod oscillator;
mod wasm_audio;

use gui::create_gui;
use oscillator::{Oscillator, Params};
use wasm_audio::wasm_audio;
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub async fn web_main() {
    // On the application level, audio worklet internals are abstracted by wasm_audio:
    let params: &'static Params = Box::leak(Box::default());
    let mut osc = Oscillator::new(params);
    let ctx = wasm_audio(Box::new(move |buf| osc.process(buf)))
        .await
        .unwrap();
    create_gui(params, ctx);
}
}
  • dependent_module
#![allow(unused)]
fn main() {
use js_sys::{Array, JsString};
use wasm_bindgen::prelude::*;
use web_sys::{Blob, BlobPropertyBag, Url};

// This is a not-so-clean approach to get the current bindgen ES module URL
// in Rust. This will fail at run time on bindgen targets not using ES modules.
#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen]
    type ImportMeta;

    #[wasm_bindgen(method, getter)]
    fn url(this: &ImportMeta) -> JsString;

    #[wasm_bindgen(js_namespace = import, js_name = meta)]
    static IMPORT_META: ImportMeta;
}

pub fn on_the_fly(code: &str) -> Result<String, JsValue> {
    // Generate the import of the bindgen ES module, assuming `--target web`.
    let header = format!(
        "import init, * as bindgen from '{}';\n\n",
        IMPORT_META.url(),
    );

    Url::create_object_url_with_blob(&Blob::new_with_str_sequence_and_options(
        &Array::of2(&JsValue::from(header.as_str()), &JsValue::from(code)),
        BlobPropertyBag::new().type_("text/javascript"),
    )?)
}

// dependent_module! takes a local file name to a JS module as input and
// returns a URL to a slightly modified module in run time. This modified module
// has an additional import statement in the header that imports the current
// bindgen JS module under the `bindgen` alias, and the separate init function.
// How this URL is produced does not matter for the macro user. on_the_fly
// creates a blob URL in run time. A better, more sophisticated solution
// would add wasm_bindgen support to put such a module in pkg/ during build time
// and return a URL to this file instead (described in #3019).
#[macro_export]
macro_rules! dependent_module {
    ($file_name:expr) => {
        $crate::dependent_module::on_the_fly(include_str!($file_name))
    };
}
}
  • gui
#![allow(unused)]
fn main() {
use crate::oscillator::Params;
use wasm_bindgen::{closure::Closure, JsCast, JsValue};
use web_sys::{AudioContext, HtmlInputElement, HtmlLabelElement};

pub fn create_gui(params: &'static Params, ctx: AudioContext) {
    let window = web_sys::window().unwrap();
    let document = window.document().unwrap();
    let body = document.body().unwrap();

    let volume = add_slider(&document, &body, "Volume:").unwrap();
    let frequency = add_slider(&document, &body, "Frequency:").unwrap();
    volume.set_value("0");
    frequency.set_min("20");
    frequency.set_value("60");

    let listener = Closure::<dyn FnMut(_)>::new(move |_: web_sys::Event| {
        params.set_frequency(frequency.value().parse().unwrap());
        params.set_volume(volume.value().parse().unwrap());
        drop(ctx.resume().unwrap());
    })
    .into_js_value();

    body.add_event_listener_with_callback("input", listener.as_ref().unchecked_ref())
        .unwrap();
}

fn add_slider(
    document: &web_sys::Document,
    body: &web_sys::HtmlElement,
    name: &str,
) -> Result<HtmlInputElement, JsValue> {
    let input: HtmlInputElement = document.create_element("input")?.unchecked_into();
    let label: HtmlLabelElement = document.create_element("label")?.unchecked_into();
    input.set_type("range");
    label.set_text_content(Some(name));
    label.append_child(&input)?;
    body.append_child(&label)?;
    Ok(input)
}
}
  • oscillator
#![allow(unused)]
fn main() {
// Wasm audio processors can be implemented in Rust without knowing
// about audio worklets.

use std::sync::atomic::{AtomicU8, Ordering};

// Let's implement a simple sine oscillator with variable frequency and volume.
pub struct Oscillator {
    params: &'static Params,
    accumulator: u32,
}

impl Oscillator {
    pub fn new(params: &'static Params) -> Self {
        Self {
            params,
            accumulator: 0,
        }
    }
}

impl Oscillator {
    pub fn process(&mut self, output: &mut [f32]) -> bool {
        // This method is called in the audio process thread.
        // All imports are set, so host functionality available in worklets
        // (for example, logging) can be used:
        // `web_sys::console::log_1(&JsValue::from(output.len()));`
        // Note that currently TextEncoder and TextDecoder are stubs, so passing
        // strings may not work in this thread.
        for a in output {
            let frequency = self.params.frequency.load(Ordering::Relaxed);
            let volume = self.params.volume.load(Ordering::Relaxed);
            self.accumulator += u32::from(frequency);
            *a = (self.accumulator as f32 / 512.).sin() * (volume as f32 / 100.);
        }
        true
    }
}

#[derive(Default)]
pub struct Params {
    // Use atomics for parameters so they can be set in the main thread and
    // fetched by the audio process thread without further synchronization.
    frequency: AtomicU8,
    volume: AtomicU8,
}

impl Params {
    pub fn set_frequency(&self, frequency: u8) {
        self.frequency.store(frequency, Ordering::Relaxed);
    }
    pub fn set_volume(&self, volume: u8) {
        self.volume.store(volume, Ordering::Relaxed);
    }
}
}
  • wasm_audio
#![allow(unused)]
fn main() {
use crate::dependent_module;
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsValue;
use wasm_bindgen_futures::JsFuture;
use web_sys::{AudioContext, AudioWorkletNode, AudioWorkletNodeOptions};

#[wasm_bindgen]
pub struct WasmAudioProcessor(Box<dyn FnMut(&mut [f32]) -> bool>);

#[wasm_bindgen]
impl WasmAudioProcessor {
    pub fn process(&mut self, buf: &mut [f32]) -> bool {
        self.0(buf)
    }
    pub fn pack(self) -> usize {
        Box::into_raw(Box::new(self)) as usize
    }
    pub unsafe fn unpack(val: usize) -> Self {
        *Box::from_raw(val as *mut _)
    }
}

// Use wasm_audio if you have a single wasm audio processor in your application
// whose samples should be played directly. Ideally, call wasm_audio based on
// user interaction. Otherwise, resume the context on user interaction, so
// playback starts reliably on all browsers.
pub async fn wasm_audio(
    process: Box<dyn FnMut(&mut [f32]) -> bool>,
) -> Result<AudioContext, JsValue> {
    let ctx = AudioContext::new()?;
    prepare_wasm_audio(&ctx).await?;
    let node = wasm_audio_node(&ctx, process)?;
    node.connect_with_audio_node(&ctx.destination())?;
    Ok(ctx)
}

// wasm_audio_node creates an AudioWorkletNode running a wasm audio processor.
// Remember to call prepare_wasm_audio once on your context before calling
// this function.
pub fn wasm_audio_node(
    ctx: &AudioContext,
    process: Box<dyn FnMut(&mut [f32]) -> bool>,
) -> Result<AudioWorkletNode, JsValue> {
    AudioWorkletNode::new_with_options(
        ctx,
        "WasmProcessor",
        AudioWorkletNodeOptions::new().processor_options(Some(&js_sys::Array::of3(
            &wasm_bindgen::module(),
            &wasm_bindgen::memory(),
            &WasmAudioProcessor(process).pack().into(),
        ))),
    )
}

pub async fn prepare_wasm_audio(ctx: &AudioContext) -> Result<(), JsValue> {
    let mod_url = dependent_module!("worklet.js")?;
    JsFuture::from(ctx.audio_worklet()?.add_module(&mod_url)?).await?;
    Ok(())
}
}

Running the demo

"Currently it's required to use the --target no-modules or --target web flag with wasm-bindgen to run threaded code. This is because the WebAssembly file imports memory instead of exporting it, so we need to hook initialization of the wasm module at this time to provide the appropriate memory object. This demo uses --target no-modules, because Firefox does not support modules in workers.

With --target no-modules you'll be able to use importScripts inside of each web worker to import the shim JS generated by wasm-bindgen as well as calling the wasm_bindgen initialization function with the shared memory instance from the main thread. The expected usage is that WebAssembly on the main thread will post its memory object to all other threads to get instantiated with."
_ wasm-bindgen Guide{target="_blank"}

build and serve

This example requires to not create ES modules, therefore we pass the flag --target no-modules

wasm-pack build --target no-modules --no-typescript --out-dir www/pkg

http www

open index.html

firefox http://localhost:8000/html/

What's next?

Next example: TODO MVC using wasm-bingen and web-sys -->