Development

Layout

clients/            one example client per language (see Client examples)
crates/
├── vcon/           the vCon library; today it reads and checks a vCon: form,
│                   gzip, uuid, critical (no I/O; the IETF examples are its fixtures)
└── vcon-server/
    ├── console/           the console's page, script and styles, built into the binary
    └── src/
        ├── main.rs        the command line, a thin front end
        ├── api.rs         HTTP routes, the request pipeline, problem documents
        ├── console.rs     serves the console's files with their headers
        ├── mcp.rs         the MCP server at /mcp; mcp/stdio.rs, the stdio command
        ├── access.rs      tokens, scopes, authentication
        ├── store.rs       one tenant's vCons: versions, deletes, purges, legal reads
        ├── tenants.rs     the registry: tenants, their keys, API keys, the admin log
        ├── envelope.rs    keys in locked memory; seal, wrap, HKDF
        ├── catalog.rs     SQLCipher catalogs
        ├── audit.rs       the audit log
        ├── retention.rs   calendar retention periods
        ├── config.rs      the configuration file
        ├── datadir.rs     who may use the data directory at once
        ├── serve.rs       the server and its purge sweep
        ├── fs.rs, clock.rs  the filesystem and the clock, behind interfaces for tests
        ├── ids.rs         tenant, version and key ids
        └── query.rs       query strings, read by one rule for every route
fuzz/               cargo fuzz targets for everything the server reads from outside

crates/vcon-server/openapi.json describes the HTTP API. It is written by hand; website/static/openapi.json must be the same bytes. The unit tests fail when a route, a status a test produces or a problem type is missing from it.

The binary does almost nothing itself: everything it runs is in the library, so tests drive the same code the binary does.

Build and test

cargo build
scripts/nextest.sh install
scripts/nextest.sh run --workspace
cargo test --workspace --doc

The tests run under cargo-nextest, which runs each test in a process of its own and ends a test still running at the deadline in .config/nextest.toml: a test that runs for more than 60 seconds is reported as SLOW by name, and after 3 minutes it is ended and fails as TIMEOUT. cargo test has no such deadline, so one hung test would hang the run. scripts/nextest.sh pins nextest's version; scripts/nextest.sh install downloads that release for x86_64 or aarch64 Linux, checks it against its pinned SHA-256 and puts it in cargo's bin directory, and the commit hook refuses to run another version. nextest does not run doc tests, so cargo test --doc runs those. Arguments after run are nextest's, which takes cargo's package and target options and a test-name filter, as cargo test does.

The tests that run the real binary (crates/vcon-server/tests/) start a server on a free port, with a temporary data directory and certificate, and check what a client and an operator see: stores and reads, memory locked, no core dumps, a second server refused, SIGTERM.

The console's browser tests, in console-tests/, drive headless Chromium with Playwright against the real server. scripts/test-console.sh builds the binary, starts it with a temporary data directory, certificate and operator key, runs the tests and removes everything it made. Install the pinned Playwright and its browser once:

npm ci --prefix console-tests
(cd console-tests && npx playwright install --only-shell chromium)
scripts/test-console.sh

Each test makes its own tenant and keys through the admin API, and fails on any content-security violation in the page.

The store's conformance suite runs on every backend. With the backend-tests feature it runs against a real S3-compatible service as well, with the S3 backend's own tests and serve keeping its objects in a bucket; CI's "storage backends (RustFS)" job runs them against RustFS, and "storage backends (MinIO)" against MinIO, whose binary it checks against a pinned SHA-256 (MinIO publishes no container image any more). The tests fail, never skip, when the service is not there. Start a RustFS that listens on the loopback address only, and give the tests its address and keys:

docker run -d --rm --name vcon-rustfs -p 127.0.0.1:9000:9000 \
  -e RUSTFS_ACCESS_KEY=vcontest -e RUSTFS_SECRET_KEY=vcon-local-only-secret \
  rustfs/rustfs@sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2
export VCON_TEST_S3_ENDPOINT=http://127.0.0.1:9000
export VCON_TEST_S3_ACCESS_KEY=vcontest VCON_TEST_S3_SECRET_KEY=vcon-local-only-secret
scripts/nextest.sh run -p vcon-server --features backend-tests --lib backend::
scripts/nextest.sh run -p vcon-server --features backend-tests --test serve s3::
docker stop vcon-rustfs

Each test makes a bucket of its own, so a run leaves buckets behind; docker stop removes the container and them.

Linking

Test and development builds on aarch64 Linux link with the Rust toolchain's own lld: .cargo/config.toml names scripts/cc-lld.sh as the linker for aarch64-unknown-linux-gnu, and it runs cc -fuse-ld=lld with the toolchain's lib/rustlib/<target>/bin/gcc-ld directory. Measured on 2026-11-02 on an aarch64 host, the 118 MB serve test binary linked in 2.1 to 2.6 seconds with lld and in 3.6 to 4.4 with GNU ld. A toolchain without that directory (a Linux distribution's own rustc, for example) links with plain cc and GNU ld, so nothing needs installing. On x86_64 Linux rustc links with its own lld already, and the config names no linker there. Release builds (scripts/reproducible-build.sh) set plain cc for aarch64, so the binaries released are linked as before.

How the code is tested

• Test first. Each behavior's test is written before the code and seen to fail for the right reason.
• Faults injected. After a change passes, its code is broken on purpose, one fault at a time (a condition inverted, a step skipped, a limit moved), and the tests must fail for each. A fault that survives means a missing test, which is then written.
• Crashes simulated. fs.rs has a test filesystem that can cut power at any step: what was not synced is lost, and the store must reopen whole.
• Races forced. Test hooks pause a request between its steps so another can run in the gap: two writers on one uuid, a purge during a restore, a removal during a request.
• Time controlled. Retention and the sweep run on a clock the tests set.
• Fuzzed. Every parser of outside input has a fuzz target; see below.

Fuzzing

Each target in fuzz/fuzz_targets/ feeds random input, grown from its seeds, to one thing the server reads from outside:

A target fails on a panic, on a failed check of what it reads, and on a sanitizer error. Fuzzing needs the nightly toolchain named in fuzz/rust-toolchain.toml and cargo-fuzz 0.13.2:

cargo install cargo-fuzz --version 0.13.2 --locked
scripts/fuzz.sh list                  # the targets
scripts/fuzz.sh run inspect 60        # run one target for 60 seconds
scripts/fuzz.sh plant inspect         # plant a panic, expect the target to find it

scripts/fuzz.sh run reads the seeds in fuzz/seeds/<target>/ and, for inspect and inspect_gzip, the IETF examples. What libFuzzer adds goes to fuzz/corpus/<target>/, and a crash input to fuzz/artifacts/<target>/; git ignores both. A crash is a defect: fix it with a regression test written first, in the crate the crash is in.

The seeds hold no personal data. The vCons among them are three of the synthetic vCons from vcon_faker, with every name, email address, phone number and URL replaced by a reserved value and every free text by neutral words.

scripts/fuzz.sh plant checks the targets themselves. For each line of fuzz/plants.txt it puts panic!("planted fault"); before one line of the code a target reads, runs that target (300 seconds at most, set by PLANT_SECONDS), and expects the crash. It then puts the file back and compares it with its copy, byte for byte. A target that misses its plant fails the check.

Test dependency from a fork

The tests use a fork of rusqlite beside rusqlite itself: https://github.com/NormB/rusqlite at commit a2936ca6190f54e7ab54375385877cc96eebebe5, version 0.41.0-sqlcipher.1, the dev-dependency rusqlite_fork of vcon-server.

• Why: rusqlite 0.41.0's pragma helpers (pragma_update, pragma_query_value and the others) refuse SQLCipher's own pragmas, such as cipher_log_level, with "Invalid pragma". They accept only names in SQLite's pragma list, and SQLCipher's are not in it. So catalog.rs sets SQLCipher's settings as plain SQL. The fork also accepts SQLCipher's names. The test sqlcipher_settings_read_the_same_through_the_pragma_helpers reads the settings back through the fork's helpers and checks that they equal what the plain SQL reads. The bug is not reported upstream yet.
• What uses it: only the tests. The server, from cargo build, a release or crates.io, uses crates.io rusqlite 0.41.0. A [patch] in the workspace Cargo.toml gives the fork crates.io's libsqlite3-sys 0.39.1, so both share one SQLite. A build fetches the fork all the same (see building without network access).
• When it goes: the test crates_io_rusqlite_refuses_sqlcipher_names fails once the workspace's rusqlite accepts SQLCipher's names, as in the dependency update that brings such a release. Then set the settings through the pragma helpers in catalog.rs, and remove the rusqlite_fork dev-dependency, the [patch] section, both tests and this section.

Code style

The Rust code follows the Google Rust Style Guide, the Rust Style Guide and, for the published crates' public API, the Rust API Guidelines.

• Format with scripts/rustfmt.sh, not cargo fmt. rustfmt.toml sets options only a nightly rustfmt reads, so the script runs the nightly it pins, for formatting only; scripts/rustfmt.sh install installs it. Builds, clippy and tests use the stable toolchain.
• scripts/check_style.py checks what rustfmt and clippy do not: glob imports (allowed: use super::* in an inline module, use Enum::* in a function, pub use sub::* in a non-leaf module), a reason above each #[rustfmt::skip], and a message on every assertion, tests included, saying why it holds. Until every assertion has one, style/assert-message-baseline.txt holds how many each file may still lack; a new file may lack none, and a file that gets better lowers its count with scripts/check_style.py --write-baseline, which never raises one.
• Clippy and rustc lints for the rest are set at deny in [workspace.lints] in Cargo.toml. Every package takes them, and check_style.py fails if one does not. A module that does not meet one yet carries #[expect(lint, reason = "...")] on its mod line in lib.rs; the expectation fails once the module is fixed, and is then removed. expect() is allowed: the Google guide prefers it to unwrap(), with the reason.

Gates

Enable the repository's hooks once per clone:

git config core.hooksPath .githooks

• pre-commit refuses personal data and AI-tool attribution in staged changes (scripts/scan_forbidden.py), and checks that git ignores every export of the private design documents (scripts/check-ignored-exports.sh). On a copy of exactly what is staged, it runs scripts/rustfmt.sh --check, scripts/check_style.py --walk, cargo clippy --workspace --all-targets --locked --quiet -- -D warnings, the tests with scripts/nextest.sh run --workspace --locked (each test under the deadline in .config/nextest.toml) and the doc tests with cargo test --workspace --locked --quiet --doc, then the same for the fuzz package (its format, clippy and its own tests). On aarch64 it links the test builds with the staged scripts/cc-lld.sh (see Linking). It also checks that every path .bestpractices.json links to exists (scripts/test_check_repo_links.py), and what each crate would publish to crates.io (scripts/check_package.py): it packages each published crate, refuses any file in a .crate other than the sources, the files they include, the README and the two licenses, and builds each crate from its .crate alone; a missing README or license; a dependency taken by git or by path, in the manifest or the lock file; a relative link in the README; and anything the scan above refuses.
• commit-msg refuses the same in the message, every line of it, including lines that start with #, which git commit -m keeps.
• pre-push refuses any outgoing commit that is not signed by an allowed signer, whose author and committer are not the pusher's configured identity, or whose whole message the same scan refuses (scripts/scan-commit-messages.sh), so a commit made with --no-verify is still caught. CI scans every commit message of a push or pull request the same way.

scripts/test-hooks.sh and scripts/test-pre-push.sh test the hooks themselves. Every commit on main is signed and shows as verified on GitHub.

Test data uses reserved values only: example.com addresses, 555-01xx numbers, RFC 5737 and RFC 3849 addresses. The one exception is the IETF draft's own examples, kept unchanged under crates/vcon/tests/fixtures/ietf/.

This website

The site is Zola 0.23.6, in website/. Preview it with zola serve from that directory.

scripts/check-site.sh   # builds the site and checks it, as the deploy does

The check fails on a broken internal link, on any inline script, inline style or event-handler attribute (the pages' content-security policy forbids them), and when the problem types or the configuration page disagree with the code. Once CI passes on a push to main, .github/workflows/pages.yml deploys the site of that commit to GitHub Pages; Cloudflare fronts it.

npm ci --prefix site-tests
npx --prefix site-tests playwright install --only-shell chromium
scripts/test-site.sh    # the built site in a browser

scripts/test-site.sh builds the site, serves it locally and runs the Playwright tests in site-tests/. They check every page with axe-core against WCAG 2.2 level AA, contrast and landmarks included, in light and dark at 360 and 1440 pixels wide. They also check the home page at 768 pixels, that it never scrolls sideways, that the browser logs no error (a refusal by the content-security policy is one), and that it shows the sections it is built from. On every page they check three WCAG 2.2 AA criteria axe does not: nothing scrolls sideways at 320 pixels (reflow), no keyboard focus stop is hidden under the pinned header, and a link to a place on a page shows it below the header. The search tests check that the search box is labelled and that what it finds is announced as a status. CI runs it in the website job.

The home page states the number of automated tests and the share of lines they cover. Nobody types these numbers. The deploy downloads the test output and the coverage summary from CI's run of the commit it deploys, derives the numbers with scripts/published_metrics.py, and writes them to website/data/metrics.toml, which git ignores. The page states exactly what that file says, or no number when it is absent; the site check fails otherwise, and also builds the site with a made-up file and requires the page to state its numbers.

The logo and the favicons are drawn from assets/brand/mark.svg by scripts/icons.py; see assets/brand/README.md. The check above also refuses an SVG that holds a script, a style, a link or an embedded object.

The how-to pages are tested

The install, getting started and how-to pages are run, not only read. scripts/docs_test.py reads a page from top to bottom, runs each sh block in one bash shell in a fresh container, and compares what each block prints with the text block that follows it on the page. In that text block, <words> stands for any text on its line, such as a token or a date, and a line holding only … stands for any number of lines. A block must exit 0, unless its marker says otherwise.

An HTML comment on the line before a block changes how it runs:

A step that works only from some release on, such as checking a signature that older releases do not have, is marked since, not skip. Until that release is published, the runner uses the newest published one, so the step does not run and the runner says so; from then on it runs. lint refuses a skip whose reason names a release, and a since block whose text does not name its release, so a reader on an older release is told to skip it.

A scenario names the pages it runs, the system it runs them on, and its tags. The scenarios include Debian 13 and Fedora 44 with the server in the foreground, Debian 13 under systemd, each install path on each distribution, an upgrade from the previous release to the server under test and back, and the container page with Docker on the runner. The how-to scenarios get the server built from the commit under test. The release scenarios download the release that the install page names. Until that release is published, they download the newest published release and say so.

python3 -I scripts/test_docs_test.py                    # the runner's own tests
python3 -I scripts/docs_test.py lint                    # the rules every tested page keeps
python3 -I scripts/docs_test.py list                    # the scenarios
cargo build -p vcon-server
python3 -I scripts/docs_test.py run foreground --binary target/debug/vcon-server

It needs Docker. lint fails on a block that no scenario runs, on a request to the server with no output shown after it, on a marker it does not know, and on an install step on the home page whose code is not a run of lines in a block of the install page. A run prints one line per block, and for a failure, the block, what the page says it prints, and what it printed.

Continuous integration

Every push to main and every pull request runs .github/workflows/ci.yml: format, clippy and the tests as the commit hook runs them (on a push to main it keeps the test output and the coverage summary for the deploy, and fails if the test output cannot be counted); the console's browser tests; the package check; the scan for AI attribution and personal data over every tracked file; the .bestpractices.json link check; scripts/check-site.sh; and scripts/icons.py --check. A coverage job runs the same tests under cargo llvm-cov and sends the result to Codecov. The how-to jobs run scripts/docs_test.py for each scenario it lists, against the server built from the commit (see the how-to pages are tested). One fuzz job per target runs it for 60 seconds, then scripts/fuzz.sh plant for it; the targets are read from fuzz/Cargo.toml, so a new one joins with no list to edit. A crash input is kept as a workflow artifact for 7 days. ClusterFuzzLite builds the same targets in its own image (.clusterfuzzlite/) and runs them when a pull request or a push to main changes crates/ or fuzz/ (five and ten minutes), and for an hour in all each Monday (.github/workflows/clusterfuzzlite.yml). Each action and each downloaded tool is pinned by commit or by sha256.

Contributing

• A bug or a feature request: open an issue at https://github.com/NormB/vcon-server/issues. For a bug, give the output of vcon-server --version, the commit you built, what you did, what you expected and what happened, and the log lines with every token removed.
• A change: send a pull request against main. Read CONTRIBUTING.md first, and sign the Contributor License Agreement before your first pull request merges.
• A security problem: report it privately, never in an issue; see reporting a vulnerability.

License

MIT OR Apache-2.0, at your option.

Source of this page: website/content/docs/development.md