Storing vCons

What is accepted

A vCon is sent as one of three forms, as the IETF vCon draft defines them:

• unsigned: a JSON object with at least uuid and created_at;
• signed: a JWS in general JSON serialization whose payload is the vCon, itself possibly signed again;
• encrypted: a JWE in general JSON serialization.

Each may be gzipped. The request's Content-Type is application/vcon, application/vcon+gzip or application/json (any case, any parameters), or absent; anything else is refused with unsupported-media-type. A body is treated as gzip when its media type says so or it starts with the gzip magic number.

What is checked

The checks are in the vcon crate, which reads a body without changing it. In order:

1. Gzip, if any, inflates cleanly (bad-gzip) and within limits.inflated (too-large). Every gzip layer, nested signed payloads included, shares that one limit.
2. The body is one JSON object (not-json) in which no object repeats a member name (duplicate-member) and no more arrays and objects are open at once than the fixed limit (too-deep). Parsers disagree on which duplicate wins, so a vCon with one could mean different things to different readers.
3. The form is clear: not both signed and encrypted (ambiguous-form), and one of the three (no-form).
4. The uuid is found: in an unsigned vCon, uuid in RFC 9562 text form (missing-uuid, bad-uuid) and created_at present (missing-created-at); in a signed one, inside the payload (bad-payload) of at most the fixed number of signed layers (too-deep), and every uuid a signature header names must agree (uuid-mismatch); in an encrypted one, from its unprotected header.
5. critical (or must_support) is a list of names (bad-critical), each one supported (unsupported-extension). An encrypted vCon's list cannot be read, so it is not checked.

vcon-server does not verify signatures and cannot decrypt an encrypted vCon: it holds no producer's keys. It stores them as received, so whoever does hold the keys can verify or decrypt them later.

Stored exactly as received

The stored body is the request body, byte for byte: never re-serialised, re-compressed or normalised. A read returns those bytes, with Content-Type: application/vcon+gzip if they were gzipped and application/vcon if not.

Versions

Every stored body is a version, named by a random 32-hex-digit id and returned as a strong ETag, such as "9f2c…". A uuid has one current version.

• Create (POST /vcons) stores a new uuid: 201 with Location and ETag. The same bytes again answer 200 with the same ETag, so a producer can retry without fear. Other bytes for a uuid that exists are conflict (409).
• Replace (PUT /vcons/{uuid}) needs If-Match naming the current version, with one strong tag; without it the answer is precondition-required (428), and with any other tag precondition-failed (412). The body's uuid must be the path's (path-uuid-mismatch). A signed or encrypted current version is never replaced (immutable): its signature would no longer hold for what is served. The old version is hidden and kept for legal reads until it is purged.
• Delete (DELETE /vcons/{uuid}) hides the uuid and all its versions at once: reads then answer gone (410), and so does a create with that uuid. Deleting again answers 204 again. The versions stay for legal reads until they are purged.

Two requests racing on one uuid are settled in the catalog: one wins, and the other gets the answer it would have had after it.

On disk

Each version is a file under the tenant's objects/ directory, named by its version id and sealed with AES-256-GCM under its own data key. The catalog row holds that key, wrapped by the tenant key. A write goes to a temporary file, is synced, renamed and the directory synced, before the catalog row commits; a crash at any point leaves either the whole version or nothing that start-up recovery does not remove. See security.

With the objects in an S3 bucket (see keep vCons in an S3 bucket), each version is an object named <prefix>tenants/<tenant id>/objects/<version id> instead, holding the same sealed bytes. The object is written with one PutObject before the catalog row commits. A store whose object cannot be written answers internal and stores nothing. An object written by a request that then fails before its commit is removed at once; one left by a crash, or by a commit whose outcome is unknown and whose row was not kept, is removed by start-up recovery at the next start, which logs how many objects it removed.

Source of this page: website/content/docs/storing-vcons.md