T-017 — HERALD v0.2 Conformance Test Runner

T-017 — HERALD v0.2 Conformance Test Runner

A small TypeScript runner that loads the canonical test vectors produced by T-001 and exercises the reference TypeScript library produced by T-003, reporting deterministic pass/fail per vector and a per-category summary.

Output path declared in TODO.md line 32: tests/conformance/. This deliverable lives in the manthan sandbox at learning/herald-protocol/manthan/state/findings/T-017/ until promoted.

Run

cd learning/herald-protocol/manthan/state/findings/T-017
npm test          # node --test driver, 12 assertions
npm run run       # the runner CLI, human-readable report
npm run run:json  # the runner CLI, JSON report (CI-friendly)

Node 24 is required (--experimental-strip-types). No npm install is needed; this directory has no third-party dependencies.

What it does

1. Loads learning/herald-protocol/test-vectors/v0.2/vectors.json (overridable via --vectors-path or VECTORS_PATH).
2. Verifies the keypair derivation against the library: each declared keypair is re-derived from its seed; if the derived sigil or public-key bytes disagree with the vector, the runner aborts. This catches SPEC §2.1 lines 74-84 regressions in the library.
3. Dispatches each vector to a category through a small registry. The built-in envelope-core category covers the seven T-001 vectors; T-024 (cultural-attestation) and T-026 (reserved-sigil rejection) attach by calling registerCategory() from their own modules.
4. Reconstructs the wire envelope from the vector inputs and calls verifyEnvelope from the T-003 library. Replay vectors are reconstructed by following inputs.based_on; tampered vectors are reconstructed in their post-tamper form so the verifier exercises its id-mismatch path.
5. Compares observed verdict against expected, classifies failures, and emits a stable report.

Output formats

• human (default): table grouped by category with totals.
• json: stable, machine-readable with outcome, expected, observed, reasons, and known_interop_gap per vector. Exit code maps as below.

Exit codes

Files

• package.json — npm scripts and Node engine requirement.
• src/lib-adapter.ts — single import boundary into the T-003 reference library at learning/herald-protocol/manthan/state/findings/T-003/src/index.ts. If T-003 is later promoted to lib/, this is the only line to rewrite.
• src/loader.ts — reads vectors.json, re-derives keypairs, returns a LoadedVectors map.
• src/reconstruct.ts — turns a vector's stored inputs into an Envelope object the verifier can be handed. Handles the genesis/reply/tampered/replay shapes the T-001 vector schema uses.
• src/categories.ts — category registry + the built-in envelope-core category. New categories register via registerCategory().
• src/reporter.ts — summarise(), format() (human/json), and the exitCode() policy.
• src/runner.ts — CLI entry; can also be imported as run().
• test/runner.test.ts — node:test driver. Asserts the runner's reports against the seven T-001 vectors and tolerates the known interop gaps so the test exits 0 even when the library and the vectors disagree on stable, pre-existing divergences.

Extending (T-024, T-026)

Both follow-on tasks add new vector files alongside tests/conformance/. The extension shape:

import { registerCategory, type Category } from "./categories.ts";

const culturalAttestation: Category = {
  name: "cultural-attestation",
  description: "T-024 vectors exercising CHARTER-driven attestation handling.",
  matches: (v) => v.name.startsWith("cultural-"),
  run: (vector, ctx) => {
    // build an envelope, call verifier, return VectorReport
  },
};

registerCategory(culturalAttestation);

T-024 should land its vectors under tests/conformance/cultural/ (per TODO.md line 129) and import the runner's category API. T-026 lands under tests/conformance/reserved-sigils/ (per TODO.md line 131). Both should keep the current envelope-core category untouched.

Caveats

The runner reports two pre-existing divergences between T-001 and T-003 as known-interop-gap. These do not block the runner; they are the very thing a conformance runner exists to surface. Read NOTES.md before triaging any failure.