2026-04-24-summary.md
Content-Honesty Audit — 2026-04-24 Roll-Up
Triggered by: Adam brr directive (Session 14). First run of recurring content-honesty audit (urgent-todos #15).
Scope: 4 parallel multi-agent audits covering study guide Tier 1, wisdom rank-9+, blog topic tags, and topic cross-references.
Method: 4 Explore agents, each handed a concrete sample (inputs in audits/inputs-2026-04-24/), reading actual source post content and evaluating against specific criteria. No editorial rewriting — read-only evaluation.
Headline
The frozen-track archive is honest; derived-track classifiers have repeatable drift.
| Audit | Pass rate | Verdict |
|---|---|---|
| Study guide Tier 1 | 9/9 | S7 revision held. No action. |
| Wisdom rank-9+ | 14/20 | Text solid; rank and pillar errors. |
| Blog topic tags | 8/13 valid | 3 surface-match drifts + 2 empty-stub bugs. |
| Topic cross-refs | 1/4 justified, 4 under-tagged | Classifier under-represents real overlap. |
The three findings that matter
1. S7-style surface-match drift recurs in new pillars
Session 7 found information-theory pillar classifier matching the word "information". This audit found the same failure mode in:
bitcoin-protocolmatching "blockchain" (strategic-integration post is 95% Madoff fraud, tagged bitcoin-protocol on one keyword)identity-historymatching "history" (geographical-aspects-of-cybercrime drifts this way)
L29 is not fixed. The classifier needs structural rules (≥2 protocol references, Satoshi/identity argument presence, etc.) not keyword floors.
2. Wisdom top-rank drifts toward comprehensiveness, not distinctiveness
All 20 rank-9+ insights are verbatim or accurate paraphrases (cleanup held). But 6 are either rank-inflated (comprehensive regulatory material at rank 10) or mis-pillared (legal-argument content tagged bitcoin-protocol). Rank 10 should be reserved for Wright's distinctive thesis statements, not long-quotable passages.
3. Empty stubs still carry tags
sans-analyst-paper (0 words) and giac-cutting-hacking (0 words) are both tagged with 2-3 pillars each. Tagging pipeline should skip posts with word_count < 200.
Concrete follow-ups
(Not shipped this session — these are classifier + tagging-pipeline edits for a later session.)
1. Tagging pipeline fix. scripts/blog-categorise.py rule changes:
- Skip if
word_count < 200(no tags). bitcoin-protocolrequires ≥2 structural protocol references OR explicit thesis.identity-historyrequires Satoshi/identity argument OR pre-Bitcoin narrative.
2. Wisdom rank-10 tightening. Rank 10 criterion = "one-line thesis statement", not "long emphatic passage."
3. Bitcoin-protocol pillar cleanup. Audit all bitcoin-protocol-tagged insights for legal-argument drift (forking, tracing, EULA, signing-intent → migrate to law-governance).
4. Topic cross-ref fixes: 5 specific edits named in the topic-crossref audit — remove 1 drift cross-ref, add 4 missing ones.
Pairs with urgent-todos #14 (BASELINE.json)
These audits catch drift in derived views. BASELINE.json catches drift as metric regressions in committed data. They are complementary. Both remain unshipped as scripts.
Codification
scripts/audit-sample.py written this session — --target {tier1|wisdom|topics|blog-tags} samples inputs and produces the evaluation framing for each. The evaluation itself still requires agent/reader involvement (reading source posts and forming a judgment is not automatable — that's the whole point per L29). The script provides the deterministic sampling layer; agents (or humans) provide the content-reading layer.
Files
audits/2026-04-24-tier1.{json,md}— study guide descriptionsaudits/2026-04-24-wisdom-rank9plus.{json,md}— top-rank insight accuracyaudits/2026-04-24-blog-tags.{json,md}— blog post topic assignmentaudits/2026-04-24-topic-crossrefs.{json,md}— cross-reference justificationaudits/inputs-2026-04-24/— concrete samples the agents evaluatedscripts/audit-sample.py— recurring audit sampler
— Craig (/home/a/Projects/craig/)