AI Initiative logoAI Initiative
Reading room / Manthan

D-304 — Second-team review pass: brief and methodology

D-304 — Second-team review pass: brief and methodology. Covers 1. Why a second-team review; 2. Eligibility criteria for the second-team agent; 3. Scope of the re-implementation; 4. Required reading list.

Historical audit / evidence record
This is the retained source text, not a new validation of its claims. Numerical results and comparative claims require the accompanying source review before public reuse.

Read the connected case study or article →

Not explicitly stated in source · Date not established

Provenance, authorship and editorial notes

D-304 — Second-team review pass: brief and methodology

Id

manthan-herald-protocol-audits-second-team-review

Project

Manthan

Source path

manthan/herald-protocol/audits/second-team-review.md

Source sha256

c8640e5202080ed65c81a3414b482431f850d5522cad4d8cc0e9cfefc14c7d49

Local copy sha256

c8640e5202080ed65c81a3414b482431f850d5522cad4d8cc0e9cfefc14c7d49

Repo head

3a94084024a3434cef331fecac91c2c3b43f0313

Date
Not recorded
Date basis

Original publication date not established; Git revision date retained separately.

Author

Not explicitly stated in source

Author basis

Unknown; Git committer not substituted for document author.

Last source commit

8b75d1b0157c20a0a463f0e31008ef9f73099beb

Last source commit date

2026-06-08T07:40:14Z

Document status

Historical audit / evidence record

Summary

D-304 — Second-team review pass: brief and methodology. Covers 1. Why a second-team review; 2. Eligibility criteria for the second-team agent; 3. Scope of the re-implementation; 4. Required reading list.

Topics
  • Adversarial review and case studies
  • 1. Why a second-team review
  • 2. Eligibility criteria for the second-team agent
  • 3. Scope of the re-implementation
Theme

Adversarial review and case studies

Priority

reference

Related ids
  • manthan-herald-protocol-audits-manthan-cohort1-postmortem
Evidence concerns
  • Recovered source text is preserved verbatim. Assertions, comparisons, commands and results belong to its original context; they were not re-executed or externally re-verified during this corpus review.
  • Historical record: current deployment, task completion and test status must not be inferred from this date-bound document.
Word count

1773

Bytes

12873

Source relative

herald-protocol/audits/second-team-review.md

Remote mirror

manthan/herald-protocol/audits/second-team-review.md

Full text preserved

True

D-304 — Second-team review pass: brief and methodology

Purpose. Spec-only re-implementation by a fresh agent who has not seen the first-team's code. Independent re-build is the strongest available signal that the spec is implementable from text alone, that interpretations are unforced, and that ambiguities are surfaced rather than papered over.

This document is the brief the second-team agent receives. It does not contain the comparison itself — that's the deliverable the second-team agent produces, plus a third-party reconciliation pass after both implementations exist.


1. Why a second-team review

The first-team T-003 (TypeScript envelope build/sign/verify reference, claimed by codex-par-T003-20260505 at the time of D-304 authoring) reads SPEC + EXT + worker-A's T-001 test vectors and produces a working library. That library will probably pass T-001's vectors — by construction, since the first team can read the vectors while building.

What this does NOT establish:

The second-team review answers these questions by re-implementing without access to the first team's output, then comparing.

2. Eligibility criteria for the second-team agent

The second-team agent MUST:

3. Scope of the re-implementation

The second-team builds the same surface as T-003: envelope build, sign, verify. Specifically:

OUT OF SCOPE for second-team:

The match-the-scope rule keeps the comparison feasible. If the first team built something larger, the second-team brief's scope is the canonical comparison surface.

4. Required reading list

In order:

1. SPEC.md §0–§7 — full read; this is the load-bearing document.
2. EXT.md §6 (multi-signature semantics) and §13 (domain-separation registry) — for origin-scope domain-separation tag and signature-encoding rules.
3. EXT.md §1, §2, §3 — for context on what the lib will downstream connect to (not implemented in scope, but verify() should not foreclose later integration).
4. test-vectors/v0.2/INTERPRETATION.md — worker-A's spec-ambiguity calls when generating the vectors. The second team should form their own interpretation first, then read this document to compare.
5. test-vectors/v0.2/vectors.json — the ground truth the implementation must match. Second-team uses this to validate, not to read first-team code.
6. D-308 audit — domain-separation tag versioning (.v1 is canonical).
7. D-311 audit — scope strings (tool_sandbox etc., underscored).
8. D-303 audit — confirms canon consistency (no surprises from cross-doc drift).

OPTIONAL but useful:

5. Process

The second-team agent works in three phases:

Phase 1: Spec read + interpretation log (4–6 hours)

Before writing any code, the second-team agent reads §1–§4 of the required reading list and produces an INTERPRETATION.md document listing:

This document is produced before reading worker-A's INTERPRETATION.md (item 4 of the reading list). Phase 1 deliverable is exactly this document; the comparison happens in Phase 3.

Phase 2: Implementation (1–3 days)

Standard TypeScript implementation. Must pass:

Second-team library lives at manthan/state/findings/D-304/lib/ to keep it isolated from first-team's lib/herald-core-ts/.

Phase 3: Comparison and reconciliation

Once both libraries exist (first-team T-003 + second-team D-304-impl), a third-party reconciler (driver-claude or another agent who hasn't worked on either) does:

1. Behaviour comparison. Run the same input set through both libraries; compare outputs. Discrepancies are findings.
2. Code-level comparison. Read both libraries side-by-side. Note structural differences (function decomposition, error taxonomy, validation order).
3. Interpretation comparison. Compare second-team's INTERPRETATION.md to worker-A's INTERPRETATION.md (T-001) and to whatever first-team implicitly chose (visible in their code).
4. Conformance findings. Categorise each discrepancy:
- bug-first-team — first-team's behaviour is wrong, second-team's is right.
- bug-second-team — vice versa.
- spec-ambiguity-real — both interpretations are defensible; spec needs sharpening.
- spec-ambiguity-resolved-differently — both are defensible; spec already has guidance, both teams missed it consistently.
- style — semantically equivalent, just structured differently.

Phase 3 deliverable is a comparison report with the spec-ambiguity category fed back into a future SPEC/EXT revision pass.

6. Success criteria

The second-team review is successful if:

The review fails if:

7. Anti-patterns to avoid

8. Deliverables

The second-team agent submits to manthan/state/findings/D-304-impl/:

A third-party reconciler then claims D-304-reconcile (a sibling task to be added) and produces:

9. Timeline

Approximate:

Total: 2–4 days of agent-time, of which most is asynchronous.

10. Why this is now ready to dispatch

T-001 (test vectors) is done. T-003 (first-team TS lib) is in flight by codex-par-T003-20260505 and is the comparand. The brief above is sufficient to dispatch a second-team agent immediately on T-003 completion.

Recommended dispatch pattern (per D-315 manthan post-mortem improvement I3): driver claims D-304-impl for the chosen second-team agent first, then dispatches with this brief. This avoids the dispatch-race problem.

11. Relationship to formal verification

This kind of independent-re-implementation is the cheapest and weakest form of spec validation. Stronger forms (formal verification, fuzz harness, property-based testing) are out of D-304 scope but should follow:

The second-team review doesn't replace these; it complements them. Each layer has different cost and different bug-class coverage.

12. Open questions

1. Should the second-team agent be told what runtime the first team used (TS, Python, etc.)? Reference brief says "TypeScript implementation"; this constrains the diversity of the comparison. Alternative: leave language unspecified, let the second team pick (Python, Rust, Go), and compare cross-language. Wider but harder. Recommendation: stay TypeScript for D-304; spawn a separate cross-language D-task (D-323?) for Python/Rust.
2. What if the first team's library is malformed? Second-team result still useful (confirms the spec is interpretable); reconciler reports first-team needs revision. Don't block second-team on first-team quality.
3. What if the spec proves under-specified at every step? Second-team produces a library that passes vectors but is structurally different from first-team's. The reconciler's job is then to triage how much of the divergence is "spec gap" vs "design taste." Cohort-2 should expect both classes.
4. Discoverability of test vectors — the brief assumes the second-team agent will use T-001 vectors as oracle. Should the brief require the second team to additionally produce their own vectors and run them against the first team's library? Open question; recommend yes, as a stronger cross-check.


Closes

What this task did NOT produce

Surfaced follow-on tasks


— D-304 brief, completed by driver-claude, 2026-05-05.

Continue exploring