Manthan QD-measurement direction — plan & live backlog

Owner: operator (kulbir). Status source: this doc is the single live backlog for the
quality-diversity (QD) measurement direction. Supersedes the smeared tracker/handoff/code-marker split
for this workstream.

Decision provenance: cross-family swarm panel (claude + codex + deepseek), triaged against live ground
truth (2026-06-28). Verdict: the apparent "widen roster vs novelty-target vs redefine-grid" choice is a
false trichotomy — measurement is the precondition for the other two. *Do not widen the kind-roster
to prototype/opportunity* (diversity theater without an evaluator + closed feedback loop). Stand up the
measurement first.

Triaged ground-truth facts:

Runtime swarm = 3 personas, fixed kinds (athena→synthesis, asclepius→critique, iris→idea); agent-authored,

maestro-excluded distribution is genuinely balanced (critique 1,296 / idea 1,288 / synthesis 1,288).

opportunity≈98 is NOT corrupt labels — 97 have no author agent and come from the explorer pipelines

(moltbook / external-signal / harvester, tags explorer/external-signal/seed) + seed. So the corpus
has 4 producer classes: runtime, maestro, explorer, seed.

Maestro ingests ~3,899 artefacts ALL bucketed as synthesis → the synthesis cell currently measures

ingestion volume, not swarm exploration. This contaminates /diversity.

/ab/temporal harness EXISTS (api ab_harness.py) but has never been run as a scored acceptance gate.
selection_signal_bias=False and is parent-selection bias, NOT an output-kind diversity mechanism.

Phases (each: build → cross-family swarm review → fix → tests + CI green → PR)

Phase 1 — Provenance transparency in `/diversity` (this workstream)

Make the maestro/explorer contamination VISIBLE so QD measurement can be scoped to swarm exploration.

manthan_common.classify_provenance(tags, author_agent_id) → runtime | maestro | explorer | seed | other.
substrate read_diversity: join elite_candidate→artefact, add a by_provenance occupancy breakdown +

an optional provenance filter. Honest caveat: the pool is top-K per cell across all sources, so a
provenance filter is "exact among pooled candidates, may undercount sub-top-K"; the gate (Phase 2) is
the authoritative measure.

api /diversity: pass provenance through; surface the breakdown.
Tests + swarm review + CI green.

Phase 2 — `/ab/temporal` as a scored acceptance gate (the real next step)

The measurement infrastructure that makes every later diversity decision evidence-based.

Scope the gate dataset to runtime-produced kinds (idea/critique/synthesis); exclude/label

maestro + explorer artefacts.

Written pass/fail criterion set BEFORE the run (e.g. fitness-ranked selection beats a recency

baseline by ≥X% on post-cutoff downstream engagement). Graded by post-cutoff engagement, not novelty-band.

Emit a PASS/FAIL verdict + evidence. Tests + swarm review + CI green.

Phase 3 — (DEFERRED, gated) closed feedback loop for prototype/opportunity

A real prototype/opportunity signal needs the swarm to OBSERVE downstream outcomes and route them back as
reward. Until that loop exists, incentivising those kinds converges on nothing. Widening the roster is
deferred behind this. Not started; documented so it is not silently skipped.

Explicitly NOT doing (panel-unanimous)

Adding prototype/opportunity personas now — diversity theater without evaluator/exec/reward.
Replacing the grid axis (e.g. MutationsSurface×novelty) before the gate validates what predicts value.

Manthan QD-measurement direction — plan & live backlog