treetime

Web2 app

The web app on the WebAssembly back end in a web worker.

Diese Seite ist noch nicht übersetzt
Sie wird auf Englisch angezeigt. Einstellungstexte, Hilfetexte und die Befehlszeilenreferenz bleiben auf allen Seiten englisch.

The web2 app (packages/app-web2) is the web app without a server: the interface of app-ui sends its API requests to the WebAssembly build of the API router (packages/app-wasm), which runs in a dedicated web worker of the page. kb/decisions/app-web2-wasm-worker-back-end.md describes the design.

TaskCommand
Build the single-threaded WebAssembly module into packages/app-wasm/pkg/st/just build-wasm [dev|release|prod]
Build the threaded WebAssembly module into packages/app-wasm/pkg/mt/just build-wasm-mt [dev|release|prod]
Clippy of the single-threaded and of the threaded modulejust lint-wasm, just lint-wasm-mt
Smoke test of the single-threaded module, in Bunjust test-wasm
Smoke test of the threaded module with 2 and 8 threads, in Nodejust test-wasm-mt
Start the web2 dev serverjust up-web2
Build web2 for production and serve itjust serve-web2
Production build of web2 into packages/app-web2/dist/just build-web2

just build-wasm builds app-wasm for wasm32-unknown-emscripten with cargo. rustc links the module through emcc of Emscripten, and emcc runs the wasm-bindgen CLI on the linked module, so the build writes one JavaScript module with the Emscripten runtime and the bindings, the WebAssembly binary, and the TypeScript declarations; the recipe copies them into the package @neherlab/app-wasm (packages/app-wasm/pkg/). Emscripten provides the C library of OpenBLAS and of the compression libraries, an in-memory file system for the runs, clocks, and entropy. The dev image installs Emscripten, a single-threaded and a threaded OpenBLAS that binarylandia/build_openblas builds with emcc for this Emscripten version, and the WebAssembly liblzma and libzstd of staticbourg, all from .config/mise.image.toml (the http:emscripten and *-wasm32-unknown-emscripten entries). Emscripten gives no ABI guarantee between versions, so the image build fails when an archive comes from another Emscripten version, and an Emscripten upgrade needs new build_openblas and staticbourg releases first. The prod mode uses the dist profile, which emcc optimizes at link time. just gen wasm-types writes the declarations packages/app-wasm/pkg/app_wasm.d.ts, the only committed file of the package, so the TypeScript checks run without a WebAssembly build. The version of the wasm-bindgen CLI in .config/mise.toml and the version of the wasm-bindgen crate in Cargo.toml must be equal, because the CLI accepts only modules of its own schema version. mise installs the CLI from binarylandia/appville (task tasks/wasm-bindgen): upstream wasm-bindgen with a patch that keeps the start function of the linker, which the threaded module needs to set up its shared memory. A new crate version needs a new appville release first.

The common link settings of both modules are in [target.wasm32-unknown-emscripten] of .cargo/config.toml, among them -sSTACK_OVERFLOW_CHECK=2 and -Wl,--stack-first, which stop the module with “stack overflow” instead of letting an overflow overwrite data. Each cargo call for the target adds the settings of its build with --config: dev/cargo/wasm-st.toml for the single-threaded module and dev/cargo/wasm-mt.toml for the threaded one. just build-wasm-mt builds on the nightly of dev/docker/files/wasm-threads-toolchain, which rebuilds the standard library with atomics (-Zbuild-std), in its own target folder, and links with -pthread and the threaded OpenBLAS, liblzma, and libzstd. The module starts a pool of jobs + 2 pthread workers with the module, because rayon waits for every thread of its pool and a new pthread worker starts only after its creator returns to the event loop. just build-wasm-mt fails when the threaded module declares other exports than pkg/app_wasm.d.ts. The package exports the single-threaded module as @neherlab/app-wasm and the threaded one as @neherlab/app-wasm/threads.

just lint-wasm and just lint-wasm-mt run clippy over the binary of app-wasm for the target, the second over the code that compiles only with atomics; just check runs both. The tests of app-wasm run natively, because a test harness needs unwinding and the module aborts on a panic. just test-wasm builds the single-threaded module and runs packages/app-wasm/smoke.ts in Bun: it creates a back-end instance and a job instance of the module and drives the job protocol without workers, with an ancestral run of data/flu/h3n2/20 and its files and archive, a request answered while a job computes, a cancelled run, and an input that fails its checksum. just test-wasm-mt runs the same smoke test on the threaded module with 2 and with 8 threads under Node, because Bun does not start the pthread workers of Emscripten, and Node stays alive during a threaded run only through a timer that the test holds; it is slow and outside every gate. just up-web2 builds both modules once; run just build-wasm and just build-wasm-mt again after a Rust change. TREETIME_WEB2_PORT and TREETIME_WEB2_SERVE_PORT override the ports of the dev server and of the served production build (just status prints them).

The page compiles the single-threaded module once and starts the back-end worker with it (WorkerConnection, packages/app-web2/src/backend/connection.ts). The worker answers each request through an exchange of the module and sends the response head and body chunks back through a message port (ExchangeServer, packages/app-web2/src/backend/serve.ts). The page uses the fetch of the desktop renderer for this port (createPortFetch() of @neherlab/app-ui/port-fetch), and transfers upload bodies instead of copying them. A back-end worker that fails to start stops the back end; a back-end worker that crashes restarts with the same module, and the requests in flight and every run of the page are lost. The content security policy of the web2 page allows 'wasm-unsafe-eval', which WebAssembly compilation needs, and nothing else that evaluates code.

Each run computes in a job worker of its own, which the back-end worker starts for the run (JobHost, packages/app-web2/src/backend/jobs.ts, and packages/app-web2/src/job/worker.ts), so several runs can compute at the same time. The back-end worker sends the job its input files, writes each progress event of the job into the run’s event log, so the page shows the progress live, and receives the output files. Stop terminates the job worker at once. A panic, an out-of-memory abort, or a stack overflow in a job fails only that run, with the stderr lines and the abort reason of the module as the error. The job worker runs the threaded module, 8 threads by default; the URL parameter jobs sets the number of threads of each run (?jobs=4), and ?jobs=1 selects the single-threaded module. Threads need a cross-origin isolated page in a secure context (HTTPS or localhost): the dev server and the preview server of just serve-web2 send Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp (packages/app-web2/vite.config.ts). Without isolation, without SharedArrayBuffer, or when the browser cannot create a shared memory of 4 GiB, and after a threaded job fails to start, the jobs run the single-threaded module, and the console says why.

The back end has the routes of treetime-server, without the routes of a local app (settings, workspace, examples download). The runs, the uploads, and the outputs live in the in-memory file system of the back-end worker and stay in the tab until it reloads or closes; the inputs section of the page says so, and web2 downloads run files and archives through the generated client. The request timeout and the keep-alive of the event streams exist only in the native builds. kb/decisions/app-web2-wasm-worker-back-end.md lists the open issues.