beads-compliance-and-completion-verification
Made by JeffVerify every closed bead was actually implemented as specified. Use when "audit closed beads", "false-closed beads", "beads compliance audit", or "did we actually finish".
Works With
Quality and Utility come from canonical skill metadata. Community ratings appear separately as stars when subscriber reviews exist.
Subscribe to download
Public previews are available, but downloads, CLI sync, and premium content require an active subscription.
Sign In to Subscribe- Full skill content with deterministic versioning
- CLI sync + local search index
- Access to Jeffrey’s premium library
Save to your account
Saved skills live in your cloud list. Run jsm sync on each machine to pull them locally.
What's New with this Skill
3Updates, enhancements, and release notes for this skill.
Enhanced: beads-compliance, audit swarm recipes use flags that exist
The multi-repo and cross-skill swarm recipes invented their own spawn grammar: --agents, --weights, --pane-count, --panes-per-repo, --command-template. Both were rewritten against the live NTM binary, and a macOS trap got fixed along the way: --cc="$(wc -l < repos.txt)" fails at flag parse because BSD wc emits leading spaces and NTM's spec parser does not trim them.
- Portfolio audits spawn one labeled session sized by the repo list and dispatch per pane with send --pane=$i, 0-indexed with the tmux base-index assumption stated
- wc -l output pipes through tr -d ' ' before reaching --cc; the untrimmed form errors on macOS
- Cross-skill audit fan-out states its agent mix in real --cc/--cod/--agy counts instead of fractional weights
Update v1.2: beads-compliance, calibrated headlines, mandatory Phase 9.5 polish loop, no more 'idiomatic Rust = theater'
A real audit run on a Rust project with 844 beads flagged 153 as false-closed (18.1%). Manual review of the 15 lowest-scoring beads found all 15 were scoring false positives: real code, real tests, real fixes shipped. v1.2 reframes the headline before it can alarm, adds a bottom-N ground-truth spot-check before any remediation, mandates a 3-sweep polish loop on every new bead via br update + bv, and stops flagging idiomatic Rust #[cfg(test)] as theater.
- Calibration framing: every deterministic-only headline now ships with a 'plan for ~10–25% of the flag count, not all of it' prior; the raw 'X false-closed' line is preserved for tooling but framed in context above the fold
- New scripts/calibrate-bottom-n.sh: a read-only ground-truth spot-check that lists commits referencing each flagged bead, verifies cited files exist, prints first/last commit per file, and surfaces the scorecard's missing-items checklist
- Phase 9.5 polish loop is mandatory after Phase 9 writes beads. The verbatim user-mandated prompt is applied 3× to every new bead, with bv --robot-suggest consulted between sweeps; all edits route through br update / br comment, never hand-edits to JSONL/SQLite
- Polish prompt drift validator (validate-polish-prompt-consistency.py): single source of truth at assets/polish-prompt.txt; lints that the SKILL.md blockquote and the POLISH_PROMPT bash variable are character-identical; runs pre-commit / in CI
- Five Hard Rules at top of SKILL.md: never create/switch branches, audit dir lives INSIDE the project (no siblings), don't act on the deterministic-only headline before calibrating, Phase 9.5 mandatory, reframe alarming numbers
- Branch-drift detection: bootstrap-audit.sh records the project's entry branch as project_branch_at_pass_start in manifest.json; run-pass.sh warns loudly if the branch changed during the audit
- Sibling-dir guard: bootstrap-audit.sh refuses an AUDIT_DIR_OVERRIDE outside the project tree unless BCV_ALLOW_EXTERNAL_AUDIT_DIR=1 is explicitly set
- Theater-scan v1.2: idiomatic Rust #[cfg(test)] is no longer flagged as theater; only a runtime if cfg!(test) is. .beads/, beads_compliance_audit/, and .git/ paths are excluded from theater scans (they produced false BLOCKING findings on JSONL bead-store files)
- Two new deep-dive references: PHASE-9-5-POLISH-LOOP.md (worked example, bv interaction patterns, non-convergence handling) and CALIBRATION-FRAMING.md (the 15/15 case study, the 10–25% prior, decision tree for interpreting calibration.md outputs)
- New CASE-STUDIES.md entry #11: 'The 15/15 false-positive cohort (calibration evidence)', covering every root-cause cluster from the beads_rust audit run with the fix that landed for each
The deterministic baseline can over-flag beads when specs are prose-style, the LLM evidence-gatherer wasn't wired in, or theater patterns false-positive on idiomatic code. On the beads_rust audit, the 153 'false-closed' headline reduced to ~5–10 actually-broken beads after manual triage of the lowest-scoring band. The other ~143 were pipeline artifacts: idiomatic #[cfg(test)] mod tests flagged as conditional_skip_in_test_mode, bash return 0 in helper scripts flagged as hardcoded_return, runtime data files like .beads/issues.jsonl scanned as source code, and prose ACs the deterministi
beads-compliance-and-completion-verification: stop trusting a closed status; re-derive every closure from concrete evidence
Long-running multi-agent projects develop a failure mode where beads with detailed acceptance criteria get marked closed by an agent, but a careful audit reveals 30-60% of the substance was never built. The implementation is a stub; the conformance harness was never wired up; the fuzzer never runs in CI; the e2e test mocks the service. This skill makes the bead graph truthful again by treating every closed bead as guilty until an evidence pack proves it innocent.
- 10-phase loop runs end-to-end: inventory, spec extraction, evidence gather, compliance exec, anti-theater scan, test depth, synthesis, scoring, remediation, fresh eyes. Every phase emits machine-readable artifacts so the audit itself is auditable.
- Theater detection catches the four flavors of fake-passing tests: hardcoded short-circuit returns, mocks where the spec said no mocks, conformance harnesses never wired up, fuzzers that 'run' for 10s when the spec said 60s. Phase 5 runs mock-code-finder over the evidence.
- Tier routing by closed-bead count, hard-capped at 10 concurrent agents. Solo (<20), Pair (20-150), Squad (150-500), Battalion (500-1000), Swarm (1000-1500), Mega-swarm (1500+, prefer Sample). Past 10, Agent Mail thrash and cache fragmentation hurt throughput.
- Sample mode for 1500+ closed beads. A stratified 15-50 bead sample (5 keystones + 5 bottlenecks + 5-40 random recents) is ~100x cheaper than a comprehensive pass and keeps the headline signal.
- Audit dirs are sacred. The beads_compliance_audit/ subdirectory is its own git repo, auto-added to .gitignore so it never leaks into project history. Each pass commits one snapshot. Pass-over-pass diffs ARE the convergence signal. Never delete, never rebase.
- Remediation is graph maintenance, not implementation. Phase 9 either reopens originals (br reopen <id>) or creates new completion-debt beads that link back. The audit makes the graph truthful so the right work is visible. Fixing the code happens in a later session.
- Convergence requires two clean passes, not one. A single 'converged' pass might just mean nothing changed. Convergence demands score-delta + zero-new-false-closed + fresh-eyes rubric consistency, applied across two consecutive passes. Single-pass convergence is rejected.
- 10 modes route from one intake: Triage, Standard, Comprehensive, Tripwire (CI/cron, exits non-zero on regression), Single-bead (pre-merge gate), Re-verification, Onboarding, Red-team (rubric-evasion attacks), Spec-quality-gate, and Committee (multi-model triangulation).
- Phase 10 is the audit-of-the-audit. The fresh-eyes pass independently re-derives 5 random scorecards from evidence packs. If the scorer disagrees by more than 50 points, the rubric is ambiguous, the scorer is biased, or the operator pipeline drifted; flag for next pass.
- 23 subagent cards under subagents/. Each follows the operationalizing-expertise pattern: bead-spec-extractor, evidence-gatherer, compliance-verifier, theater-detector, scorer, fresh-eyes-rubric-auditor, red-team-adversary, closer-defender, remediator, plus 14 others.
The one rule. A bead's status field is a claim, not a fact. The audit treats every closed bead as guilty until the evidence pack proves it innocent. 'Proof' means concrete artifacts: test runs, coverage, file:line citations, fuzzer corpora, golden diffs, real-service logs. Never vibes. A bead scores 1000 only when every checklist item from its spec is satisfied with a citable artifact.