Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# The vendored corpus is compared byte for byte; no line-ending conversion.
conformance/** -text
43 changes: 24 additions & 19 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,18 +116,18 @@ namespace verified on the Central Portal, a signing key with its public half
on a public keyserver, a Central Portal user token, and the four secrets set
on this repository. Merging the prep follows this file's merge row, and
nothing else this file reserves for the operator changes. A prep is cut only
on a green full gate, which will include the conformance corpus check.
on a green full gate, which includes the conformance corpus check.

This repository keeps no decision records (`docs/adr`). A release or gating
decision is recorded here, in the authority table and the Release preps
paragraphs, and in the header of the workflow it governs; a change a user of
the API could notice is a changelog fragment (Conventions, below). The ID
format itself is not decided here: it is uxid-ex's wire-format spec,
`guides/wire-format.md` in https://github.com/riddler/uxid-ex, read at the
tag the vendored corpus names (`conformance/SOURCE.json`, once the corpus is
vendored). That this repository keeps no decision records was decided by the
conductor under a standing consent, 2026-10-21; a future API decision may
start a `docs/adr` directory.
tag the vendored corpus names (`conformance/SOURCE.json`). That this
repository keeps no decision records was decided by the conductor under a
standing consent, 2026-10-21; a future API decision may start a `docs/adr`
directory.

Widening this section is a decision for the operator to make and record here.
An agent may draft the change; it does not adopt it.
Expand Down Expand Up @@ -159,24 +159,27 @@ The tree on `main` today holds these agent instructions, the wurk manifest
(`.claude/wurk.json`), the tracker files under `.beads/`, the build
(`pom.xml`, `mise.toml` and `.mvn/jvm.config`), the module descriptor, the
package it exports, `LICENSE`, `CHANGELOG.md` with the fragment rule in
`changelog.d/README.md`, and the two workflows: CI
(`.github/workflows/ci.yml`) and the release workflow
`changelog.d/README.md`, the vendored conformance corpus under `conformance/`
(with `.gitattributes`, which keeps its line endings as committed), and the two
workflows: CI (`.github/workflows/ci.yml`) and the release workflow
(`.github/workflows/release.yml`). The package holds time-based and
deterministic generation (`Uxid.generate` and `UxidOptions`), decoding
(`Uxid.decode`, `DecodeOptions` and `DecodeResult`), the validity checks
(`Uxid.isValid` and `Uxid.isDeterministic`) and the monotonic generator
(`Monotonic`, with its `DrawSource`). The vendored corpus arrives with a later
change; until it does, the sentences in this file about the 1.0.0 package and
its corpus describe the target, not the current tree.
(`Monotonic`, with its `DrawSource`). The sentences in this file about the
1.0.0 package describe the target, not the current tree: it is not yet
published.

- **The package is `systems.riddler:uxid`.** Version 1.0.0 is its first
release on Maven Central.
- **The corpus is the arbiter.** `conformance/` will hold the corpus
vendored from uxid-ex at a release tag, its tag, commit and hash recorded in
`conformance/SOURCE.json`. The vendored files are never edited, and every
case passes with no skip list: a case this port cannot pass is reported
against uxid-ex with its evidence, never skipped or worked round here. An
answer the corpus does not fix follows uxid-ex's spec at that tag.
- **The corpus is the arbiter.** `conformance/` holds the corpus vendored from
uxid-ex at the release tag `v2.10.0` (case files, `manifest.json`, `schema/`
and `README.md`, byte for byte; not the generator `source/`), its tag,
commit and hash recorded in `conformance/SOURCE.json`. The vendored files
are never edited, and every case passes with no skip list: a case this port
cannot pass is reported against uxid-ex with its evidence, never skipped or
worked round here. An answer the corpus does not fix follows uxid-ex's spec
at that tag.

`README.md`, which a later change adds, is the user reference; `CHANGELOG.md`
records each release.
Expand All @@ -196,9 +199,11 @@ verify`: the enforcer rules (Maven 3.9 or later, every plugin version pinned,
no dependency outside test scope), the compile with every javac lint and Error
Prone, a warning failing it, the JUnit 5 tests, the jar and the sources and
javadoc jars, the Spotless format check (google-java-format) and the JaCoCo
check at 90 percent of lines; the conformance corpus check joins it once the
corpus is vendored. The inner loop, `mise run gate:loop`, is `mvn -B -ntp
test`: the compile and the tests.
check at 90 percent of lines. The conformance corpus check is part of the
tests: a test recomputes the corpus hash and each case count by the
reference's rule, and a runner (`CorpusRunnerTest`) runs every case of every
case file, with no skip list, rejecting a case key it does not know. The inner
loop, `mise run gate:loop`, is `mvn -B -ntp test`: the compile and the tests.

The gate runs on the JDK `mise.toml` pins (Temurin 25, with Maven 3.9). The
toolchain floor is declared separately: `maven.compiler.release` 17 in
Expand Down
124 changes: 124 additions & 0 deletions conformance/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# UXID conformance corpus

Test vectors for an implementation of UXID in another language. Each case is
one input and the answer this library gives for it, generated by calling the
library. A port vendors this directory at a release tag and runs every case
unchanged; a port that passes them all produces and reads the same strings as
this library for those inputs.

The rules the cases instance are in the wire format guide
([`guides/wire-format.md`](../guides/wire-format.md)). Where a case and the
guide disagree, the guide states the format and the case is a defect.

This directory is not part of the Hex package.

## Never edited by hand

Every case file and `manifest.json` are written by `mix uxid.corpus` from the
authored source under `source/`. The quality gate's Corpus stage runs
`mix uxid.corpus --check`, which builds the files in memory and fails on any
byte difference from the checked-in ones. A hand edit to a case file, or to
the hash, fails the gate; the fix is to change the source (or the library)
and regenerate:

mix uxid.corpus # write the case files and the manifest
mix uxid.corpus --check # build in memory and byte-compare; write nothing

The generator refuses to write anything when a case fails to build: the
library disagreeing with a golden vector (a case in `source/` that carries its
expected id), an encode case that does not decode back, an invalid case the
library does not refuse with the error the case names, a monotonic case whose
draws break the monotonic rule, or a duplicate id.

## Layout

| File | What it holds |
|------|---------------|
| `manifest.json` | The corpus version, the case files in order with their kind and case count, and the corpus hash |
| `encode.json` | Time-based ids: inputs to the id a producer emits |
| `deterministic.json` | Deterministic ids: inputs to the id a producer emits |
| `decode.json` | Strings to what a decoder reads from them |
| `invalid.json` | Strings a decoder refuses and options a producer refuses, to the error's name |
| `monotonic.json` | One monotonic generator's calls and draws to the ids it emits |
| `schema/` | One JSON Schema per case file (describing one line) and one for the manifest |
| `source/` | The authored source the generator reads; not needed to run the cases |

## The format

- A case file holds one JSON object per line, each line ended by a newline.
Object keys are sorted and there is no whitespace between tokens.
- Every case has a stable `id`, its kind and a slug (`encode/xl-standard-lower`),
its `input`, and either its `expected` answer or, in `invalid.json`, its
`expected_error`. A case id never changes meaning; a case that has to change
gets a new id.
- An error is a name from a closed set, never a message: the decode refusal
reasons and the generation errors the wire format guide lists ("Refusals"
and "Generation errors").
- Every integer is at most 2^53 - 1, so a JSON parser that reads numbers as
doubles reads each one exactly.
- Bytes are lowercase hex. Times are integer milliseconds since the Unix
epoch. Sizes are `xs`, `s`, `m`, `l` and `xl`. A missing prefix is `null`.
- The delimiter is `_` in every case.
- `manifest.json` is one line: `corpus_version` (1), `files` (each with
`kind`, `file` and `case_count`, in a fixed order) and `corpus_hash`, the
string `sha256:` followed by the lowercase hex SHA-256 of the case files'
bytes concatenated in the order of `files`.

## Running the cases

1. Read `manifest.json`. Check `corpus_version` is one your runner knows.
Concatenate the bytes of each file in `files`, in order, hash them with
SHA-256 and compare to `corpus_hash`; a mismatch means the vendored copy is
not the one the manifest describes.
2. For each file, split on newlines and parse each line as one case. Check the
line count against `case_count`.
3. Run each case by its kind:
- **encode**: build an id from `input` with the random field set to the
bytes of `rand` (not drawn from a random source) and compare it to
`expected`. `prefix` `""` and `null` are both no prefix.
- **deterministic**: build the deterministic id of `input.from` under
`input.prefix` at `input.size` in `input.case` and compare it to
`expected`.
- **decode**: decode `input` and compare the prefix, size, compact flag,
deterministic flag and time to `expected`. The random field is not read
back on decode, so no case carries one.
- **invalid**: for `input.decode`, decode the string and check it is
refused with the reason `expected_error` (the body length is counted in
bytes). For `input.generate`, make the request with those options (an
absent option takes its default) and check it is refused with the
generation error `expected_error`; how a port maps its own refusal to
the name is the port's.
- **monotonic**: run a monotonic generator, lower case, for `input.prefix`,
`input.size` and `input.compact`, with its random source replaced by the
case's draws, and call it once per entry of `input.times` with that time.
The first call draws `initial_seed`; each later call draws the next of
`steps`; the call whose step overflows the field also draws
`overflow_seed` (a case overflows at most once). Compare the ids to
`expected` in order. The draws are values, not a seed for a pseudorandom
generator, so no generator algorithm is part of the corpus. No time after
the first is later than the last id's time, so no other fresh value is
drawn.
4. Report each failing case by its `id`.

## Compact times and the decoder's clock

A compact body carries only the low 40 bits of its time; the decoder supplies
the rest from its own clock (the guide's "Decoding the time"). The compact
decode cases carry the full time this library recovers, and they all hold for
a decoder clock from 1699913600000 ms (2023-11-13T22:13:20.000Z) inclusive
to 2164577117183 ms (2038-08-04T23:25:17.183Z) exclusive:

- the cases at 1700000000000 ms are recovered by a clock no more than one day
(86400000 ms) behind them, which sets the start;
- `decode/s-compact-guard-last-valid` is the compact guard's last valid
millisecond (the largest compact time field, 1065151889407) from the window
before, and is recovered by the one-window subtraction only while
2^40 + 1065151889407 is more than one day ahead of the clock, which sets
the end.

A runner testing compact decode on a clock outside that range injects the
clock rather than reading the system time.

The compact xs and compact m cases decode to the standard reading the guide
names (their lengths are those of standard xs and m): the same size, `compact`
false, and a time read from the first 10 body characters as a standard time.
6 changes: 6 additions & 0 deletions conformance/SOURCE.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"repo": "riddler/uxid-ex",
"tag": "v2.10.0",
"sha": "57235777cb29c70f751552ce7aa20ab5fe71fb74",
"corpus_hash": "sha256:4f7572c8c1a60134b2955f02b34c82a839d0c3ab95d4218cc2df5a85eeb2bebb"
}
18 changes: 18 additions & 0 deletions conformance/decode.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"xl","time":1700000000000},"id":"decode/xl-standard","input":"usr_01hf7yat0004hmasw9nf6yy093"}
{"expected":{"compact":true,"deterministic":false,"prefix":"usr","size":"xl","time":1700000000000},"id":"decode/xl-compact","input":"usr_hf7yat0004hmasw9nf6yy093"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"l","time":1700000000000},"id":"decode/l-standard","input":"usr_01hf7yat0004hmasw9nf61"}
{"expected":{"compact":true,"deterministic":false,"prefix":"usr","size":"l","time":1700000000000},"id":"decode/l-compact","input":"usr_hf7yat0004hmasw9nf6yf"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"m","time":1700000000000},"id":"decode/m-standard","input":"usr_01hf7yat0004hmasw9"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"m","time":51950139736068},"id":"decode/m-compact-reads-as-standard","input":"usr_hf7yat0004hmasw9n3"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"s","time":1700000000000},"id":"decode/s-standard","input":"usr_01hf7yat0004h1"}
{"expected":{"compact":true,"deterministic":false,"prefix":"usr","size":"s","time":1700000000000},"id":"decode/s-compact","input":"usr_hf7yat0004hm5"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"xs","time":1700000000000},"id":"decode/xs-standard","input":"usr_01hf7yat00"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"xs","time":51950139736065},"id":"decode/xs-compact-reads-as-standard","input":"usr_hf7yat0001"}
{"expected":{"compact":false,"deterministic":false,"prefix":"usr","size":"xl","time":1700000000000},"id":"decode/xl-standard-upper","input":"usr_01HF7YAT0004HMASW9NF6YY093"}
{"expected":{"compact":false,"deterministic":false,"prefix":null,"size":"xl","time":1700000000000},"id":"decode/xl-standard-no-prefix","input":"01hf7yat0004hmasw9nf6yy093"}
{"expected":{"compact":false,"deterministic":false,"prefix":"a_b","size":"s","time":1700000000000},"id":"decode/s-standard-prefix-with-delimiter","input":"a_b_01hf7yat000001"}
{"expected":{"compact":true,"deterministic":false,"prefix":"usr","size":"s","time":1065151889407},"id":"decode/s-compact-guard-last-valid","input":"usr_yzzzzzzz04hm5"}
{"expected":{"compact":false,"deterministic":true,"prefix":"usr","size":"xl","time":null},"id":"decode/deterministic-usr-xl","input":"usr_zcvt7epac0t1ebcsjfyf7cwz25"}
{"expected":{"compact":false,"deterministic":true,"prefix":"cus","size":"s","time":null},"id":"decode/deterministic-cus-s","input":"cus_zt9vnyfd85bff0"}
{"expected":{"compact":false,"deterministic":true,"prefix":null,"size":"m","time":null},"id":"decode/deterministic-no-prefix-m","input":"zqz4cwzgyw8ks97869"}
{"expected":{"compact":false,"deterministic":true,"prefix":"usr","size":"xl","time":null},"id":"decode/deterministic-upper-marker","input":"usr_ZCVT7EPAC0T1EBCSJFYF7CWZ25"}
12 changes: 12 additions & 0 deletions conformance/deterministic.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{"expected":"usr_zcvt7epac0t1ebcsjfyf7cwz25","id":"deterministic/golden-usr-xl","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"xl"}}
{"expected":"cus_zt9vnyfd85bff0","id":"deterministic/golden-cus-s","input":{"case":"lower","from":"acme-corp","prefix":"cus","size":"s"}}
{"expected":"zqz4cwzgyw8ks97869","id":"deterministic/golden-no-prefix-m","input":{"case":"lower","from":"alice@example.com","prefix":null,"size":"m"}}
{"expected":"usr_z1yhw20vag","id":"deterministic/usr-xs","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"xs"}}
{"expected":"usr_z5rtsrkhx2h4dr","id":"deterministic/usr-s","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"s"}}
{"expected":"usr_zt8naccefxhwnrmscd","id":"deterministic/usr-m","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"m"}}
{"expected":"usr_zwbz6myvgz7ehe3nceap5n","id":"deterministic/usr-l","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"l"}}
{"expected":"usr_zcvt7epac0t1ebcsjfyf7cwz25","id":"deterministic/usr-xl","input":{"case":"lower","from":"alice@example.com","prefix":"usr","size":"xl"}}
{"expected":"z4krgegf6drn563pvqhya4tsx9","id":"deterministic/empty-prefix-xl","input":{"case":"lower","from":"alice@example.com","prefix":"","size":"xl"}}
{"expected":"z4krgegf6drn563pvqhya4tsx9","id":"deterministic/no-prefix-xl","input":{"case":"lower","from":"alice@example.com","prefix":null,"size":"xl"}}
{"expected":"usr_ZCVT7EPAC0T1EBCSJFYF7CWZ25","id":"deterministic/usr-xl-upper","input":{"case":"upper","from":"alice@example.com","prefix":"usr","size":"xl"}}
{"expected":"cus_zfsx9se75671f9kdjv","id":"deterministic/non-ascii-from-m","input":{"case":"lower","from":"café","prefix":"cus","size":"m"}}
Loading
Loading