2026-09-21-deassume-rank-tier-audit.md

audits/2026-09-21-deassume-rank-tier-audit.md · 21,430 bytes

De-Assume Lane Audit — Rank / Tier / Judgment Fields

Date: 2026-09-21

Auditor: Craig-agent

Bead: craig-de-assume-rank-tier-audit-qn1

Frame: hybrid (Adam vmm 2026-09-21) — machine JSON = facts only; human MD may carry curation with visible label.

Related: memory/feedback_deassume_hybrid_frame.md, wisdom-engine.md, bead craig-wisdom-rank10-tighten-rlj.

Read-only audit. No archive content mutated; this file is the sole deliverable.

Method

Grep patterns run (across --include=".py" --include=".json" --include="*.md", repo root):


impact_rank
quality_tier / quality_score / quality_bucket
provenance_score / attribution_confidence / provenance_tier
confidence_score / attribution_score / classifier_score / priority_score / importance
"tier" / 'tier'
"pillar" / 'pillar'
junk / junk_reason
"score" / 'score' (scoped to scripts/, tweets/*.json, wisdom/*.json)
Priority: / HIGH / MEDIUM / LOW / recommend / priority

Files scanned: all .py under scripts/ and web/; all .json under wisdom/, blog/, tweets/, substack/, patents/, academic-record/, books/, interviews/, court/, talks/, satoshi/, ghost-blog/, audits/; all .md at repo root and under topics/, audits/, coaching/, blog/.

Out-of-scope confirmed:

scripts/substack-blog-crossref.py — bead names it as candidate but it does not exist in scripts/. settings.json declares a substack_crossref_min_word_count "consumer" pointing at that filename but the file has never been written. Note as N/A.
is_craig in tweets/*.json — bead explicitly exempts (is_original + pre-cutoff), left alone.
Foreign-source rank (e.g. patents/inventory.json global_ranking.rank: 5 = Page White Farrer external ranking) — FACT with external anchor.

Classification key:

FACT — measurable, referenceable, external anchor. No violation.
JMJ — Judgment in Machine JSON. Lane violation.
JMU — Judgment in Human Markdown, Unlabeled. Needs curation note.
JML — Judgment in Human Markdown, Labeled. Compliant.

Findings

JMJ (Machine JSON — lane violation)

1. wisdom/insights.json — impact_rank (integer 3–10)

Locations: every record (10,770). Written by scripts/wisdom-extract-v2.py:762,780; consumed by web/server.py:1146,1149,1153,2089,2094,2119,2138,3193, scripts/fix-info-theory-tags.py:231, scripts/audit-sample.py:78, scripts/wisdom-extract-v2.py:795,806,822,855,877,1049,1054,1059,1079,1103.
Current shape: "impact_rank": 10 derived from score_insight() (position + emphasis + KEY_TERMS hit count + word_count + STRONG_STARTS + is_quote) then mapped via {0:3,1:4,2:5,...,7:10}.
JMJ. The raw_score is a fact-count; the mapped 3–10 rank + the boundary of each bucket is agent invention. rank_color = gold if >=8 else accent if >=6 else dim in web/server.py is the invented-cutoff giveaway.
Proposed replacement (per hybrid frame): drop impact_rank. Keep/expose the measurable facts already computed inside score_insight():
word_count (already present in substack file, missing here — add)
position_in_post (already position)
has_emphasis (bool, present as raw_score input; expose)
key_term_hits (count of KEY_TERMS in text)
starts_strong (bool: opens with STRONG_STARTS)
is_quote (bool)
raw_score (already exposed in substack file — expose here too for consistency)
Downstream (web app "top insights", audit-sample rank9plus) must pick its own filter from these facts.

2. wisdom/insights-cleaned.json — impact_rank (9,808 records). Same shape, same fix. Downstream references identical to #1.

3. wisdom/information-theory-insights.json — impact_rank (172 records). Same shape, same fix.

4. wisdom/substack-insights.json — impact_rank (21,454 records — largest file). Written by scripts/substack-wisdom-extract.py:655,676,731,762. Same shape. Already carries raw_score and word_count alongside — good; deletion of impact_rank here loses nothing measurable.

5. wisdom/insights.json — tier (T1/T2/T3) (10,770 records + 9,808 in cleaned + 172 in info-theory).

Written by scripts/wisdom-extract-v2.py:783 (values "T1"/"T2"/"T3" from TIER1_FILES + TIER2_TITLES enum in the script; every other post → "T3").
JMJ. Tier is entirely a curator-chosen bucketing of study-guide selection; it mirrors study-guide.md Tiers.
Proposed replacement: drop tier from insights JSON. Downstream that wants tier can join insights.source.slug against a new study-guide-slugs.json that lists {slug, tier} — this makes the derivation transparent (join is a fact; the source-of-truth is study-guide.md, which is the labeled MD surface). Web/server.py:2144 currently renders [T1] | pillar | ... — either drop that badge or load from the sidecar.

6. wisdom/insights.json — consequences (list[str]) (10,770 records).

Written by scripts/wisdom-extract-v2.py:551,766,782 (generate_consequence()). Text is 100% boilerplate: 10 template strings ("Bitcoin's design decisions have specific technical implications", "Economic incentives shape participant behaviour in Bitcoin", …) auto-mapped by pillar + keyword-hit. Verified: the top-10 unique consequences account for 8,725 / ~10,300 entries.
JMJ. Pure agent-invented boilerplate; carries no per-insight fact.
Proposed replacement: drop entirely. If a downstream needs "implied consequences," it should read the source post — the field currently provides zero information a template string doesn't.

7. wisdom/insights.json — type (foundational_claim / definition / explanation / …) (10,770 records).

Written by scripts/wisdom-extract-v2.py:539,765,781 (determine_type(); regex-hit tallies against TYPE_PATTERNS, argmax).
JMJ (borderline). The type name is a classification the script picks; the underlying regex-hit counts are facts but not exposed.
Proposed replacement: either (a) drop type and expose type_pattern_hits: {foundational_claim: 2, definition: 1, explanation: 0, ...} so downstream picks its own argmax, or (b) keep type alongside type_pattern_hits so the derivation is visible. (a) is stricter; (b) is pragmatic. Recommend (b) with _provenance_note field explaining the argmax rule.

8. wisdom/insights.json + all wisdom variants — pillar (bitcoin-protocol / economics / …) (10,770 + 9,808 + 172 + 21,454).

Written by scripts/wisdom-extract-v2.py:527,764,779 (keyword-count argmax over PILLARS.keywords). Later reassigned by scripts/wisdom-cleanup.py:208,287 from blog metadata.topics[0] — so cleaned insights inherit the blog's topic tag, which is itself a judgment-in-JSON (see #10).
JMJ (borderline; genuinely open). Pillar is a TAG (single value from a fixed 10-name vocabulary), not a rank/bucket. Bead text explicitly says pillar is borderline. However: (a) it inherits from blog topics[0] which the 2026-09-18 audit found ~60% honest (audits/2026-09-18-blog-tags.md); (b) it drives web-app filtering and topic pages.
Position (defensible): downgrade pillar to a tag-list pillar_candidates: [{name, keyword_hits}] (all pillars with any hit, hit-count exposed) rather than a single argmax. If a single value is required, sourcing it from blog metadata.topics[0] (once #10 is honest) is at least a downstream inheritance, not an insight-level judgment. See ❓ OPEN 2.

9. wisdom/insights.json — theme (Theme A / B / C string) (10,770 records).

Written by scripts/wisdom-extract-v2.py:769,784 (TIER2_THEMES lookup keyed on normalized title; blank for anything not in the study-guide Tier 2 title list).
JMJ. Curator theme labels from study-guide.md leaked into insight records. Same class as tier.
Proposed replacement: drop; same sidecar approach as tier (#5).

10. blog/metadata.json — topics (list of 1–3 topic names) (795 records).

Written by scripts/blog-categorise.py:576,640,650,676,684 (score-thresholded argmax with S15 structural gate). Consumed by scripts/wisdom-cleanup.py:219, scripts/audit-sample.py:87, scripts/fix-info-theory-tags.py:97,106,120,151, scripts/tweet-process.py:596, topic-page generators, web app.
JMJ (borderline). It IS a tag list (not a rank) — but the 2026-09-18 audit found only 12/20 (60%) honest, so it is agent judgment presented as fact. No trace of the derivation (score/matches) survives in the JSON.
Proposed replacement: keep topics but add sidecar per topic:


  "topic_evidence": {
    "bitcoin-protocol": {"title_hits": 2, "content_unique": 5,
                        "content_total": 14, "density_pct": 1.2,
                        "structural_hits": 3, "category_hint": true,
                        "score": 22.5, "qualified": true}
  }

Downstream can then filter on measurable evidence, not on the argmax label. Consumers that currently join by topic in topics still work; consumers that want a stricter filter (e.g. structural_hits >= 3 and title_hits >= 1) become possible.

11. patents/inventory.json — total_estimated: 3900, total_granted: 1090, cryptocurrency_patents_total: 1305, etc.

JMJ. Adam flagged in S16 pipeline-coordinates.json as "ungrounded patent count." These are agent aggregations across web-research sources (nchain-research.md) with no cited primary-source URL per number in the JSON.
Proposed replacement: replace each number with {value, source_url, source_date, method} so a downstream can verify or supersede. Even the string "source": "web research" at the top of the file is the wrong grain — every count needs its own citation. The Session 18 partial patent grounding (grounded-patents.json empty stub) is the start of the fact-based replacement.

*12. web/server.py:2798–2823 — TWEET_ACCOUNTS[].score (92 / 90 / 88 / 0)**

JMJ. Hard-coded curator-confidence scores per account, embedded in the Python dict (machine data). Rendered in tweet-page cards with color coding (>=80 green, >=60 yellow, else dim) — the invented-cutoff giveaway is the color logic itself.
Proposed replacement: move to settings.json under attribution_rules (each account already has a rule there) as the measurable evidence, e.g.:


  "attribution_evidence": {
    "cstominaga": {"cross_ref_count": N, "self_link_count": N,
                   "signed_content_count": N, "period": "2025–present"},
    ...
  }

Drop the 0–100 score. If a coloured badge is desired, compute from evidence at render time.

13. tweets/tweet-blog-crossref.json — score binning + score-≥4 threshold

Written by scripts/tweet-blog-crossref.py:200,301,314,342-352,394.
Per-record score is a computed count (n_overlap_keywords + n_important_overlap) — that IS a fact. The _provenance_note (line 375) already honestly labels match_type=url (score=100) vs match_type=keyword. The invented piece is the score ≥ 4 threshold (line 301) and the histogram bins (url(100)/8+/6-7/4-5 on lines 345–351).
JMJ (weak). The record shape is honest; the filter cutoff is agent-picked and hidden in code.
Proposed replacement: move the ≥4 threshold to settings.json (like substack_crossref_min_word_count); document as "downstream can raise; we chose 4 empirically." Drop the score-bin histogram from the emitted JSON (it's a summary, not per-record; keep in log only). Records untouched.

JMU (Markdown, unlabeled — needs curation note)

14. study-guide.md (329 lines).

Tier 1 has a note ("descriptions have been verified against the actual post content") — that vouches for the descriptions, not for the ordering. There is no "these tiers are Craig-agent's editorial judgment, not Wright's own hierarchy" label anywhere. Tier 2 tables have "Why" columns entirely made of curator judgment.
Fix: add the standard label per hybrid frame near the top and above each Tier heading:


  > **Curation note:** These tiers and orderings are Craig-agent's editorial
  > judgment, not Wright's own hierarchy. The underlying data at
  > wisdom/insights.json ships only measurable facts (word_count, key_term_hits,
  > position_in_post), not tiers.

15. study-guide-tier1-revised.md (71 lines). Same issue, same fix.

16. urgent-todos.md (192 lines). "Priority: HIGH/MEDIUM/LOW" on every entry (16 instances) is curator judgment. No label.

Fix: add near the top:


  > **Curation note:** Priority tags are Craig-agent's editorial judgment
  > about what would most benefit a serious student continuing Wright's work.
  > They are not Wright's own priorities.

17. wisdom-engine.md (196 lines) — design doc that describes impact_rank scale (1–10) as if it were a fact. Predates the hybrid frame. This document is the rationale for the violations #1–#3.

Fix: add an addendum at top:


  > **Superseded (2026-09-21):** the impact_rank field described here is a
  > lane violation under the hybrid de-assume frame and is being removed
  > from wisdom/*.json. This document remains as historical rationale;
  > see audits/2026-09-21-deassume-rank-tier-audit.md and
  > memory/feedback_deassume_hybrid_frame.md for the current policy.

18. topics/information-theory-audit.md — audit result MD with curator judgments about which posts belong in the IT pillar. Adjacent to audits/ outputs but lives in topics/.

Fix: minor — add "audit judgment, not final classification" note. Low priority.

JML (Markdown, labeled — compliant)

19. PROVENANCE.md (143 lines) — verification-score scale (0–100 in six labeled bands: Verified/High confidence/Probable/Attributed/Uncertain/Unverified). The scale is agent-invented but the document opens with "Every piece of content in this archive has a verification score" and describes the scale explicitly. Labeled. Compliant. Note: the scores themselves are not exposed in any machine JSON (except the leaked mirror in web/server.py #12, which IS a violation).

*20. topics/.md** (13 files) — mostly "Key Blog Posts (by length)" tables, sorted on word_count (a fact). Curation is minimal (topic-blurb intro paragraph). Border-compliant; no label required if we treat the topic listing itself as fact-tables.

*21. audits/2026-09-18-blog-tags.md + .json, audits/2026-04-24- etc.** — audit outputs record agent verdicts (honest/under-tagged/mis-tagged). Living in audits/ is itself the label ("this file records agent judgment about the classifier"). Compliant, but see ❓ OPEN 3.

FACT (verified, no action)

is_craig and is_craig_basis in tweets/*.json (scripts/tweet-attribution.py) — is_original + pre-cutoff test with rule-and-reasons record. FACT.
raw_score in wisdom/substack-insights.json — computed count from score_insight(). FACT.
word_count, position, has_emphasis etc. — measurable. FACT.
patents/inventory.json:global_ranking — external Page White Farrer ranking with cited source URL in nchain-research.md. FACT.
patents/scrape-report.json:source_rationale — provenance about the scrape method itself, not judgment about content. FACT.
settings.json — all values are configuration (type: canonical/admin-user/user); numeric values are declared with range for their config axis. Not judgment-in-data.
wisdom/insights.json id, source.{title,date,slug,category}, text, position, word_count (where present). FACT.

Summary Counts

Class	Count	Sites
JMJ (machine JSON — violation)	13	wisdom/insights.json (5 fields), wisdom/insights-cleaned.json (5), wisdom/information-theory-insights.json (5), wisdom/substack-insights.json (2), blog/metadata.json (1), patents/inventory.json (1 aggregate field-family), web/server.py TWEET_ACCOUNTS (1), tweets/tweet-blog-crossref.json (1 weak)
JMU (MD unlabeled — needs label)	5	study-guide.md, study-guide-tier1-revised.md, urgent-todos.md, wisdom-engine.md, topics/information-theory-audit.md
JML (MD labeled — compliant)	3	PROVENANCE.md, topics/.md family, audits/.{md,json} family
FACT (no action)	7 field-families	is_craig, raw_score, word_count/position/has_emphasis, external rankings, scrape-report, settings, insight identifiers

Note: findings are counted per site/field-family, not per record. Record-count blast radius is in the ship order below.

Recommended Ship Order (by blast radius)

1. *JMJ #1–#4 + #5 + #6 + #9: wisdom/.json impact_rank + tier + consequences + theme. Biggest lift, biggest blast radius, biggest visibility. Record count: 10,770 + 9,808 + 172 + 21,454 = ~42,200 records across 4 JSON files. Consumers touched:** web/server.py (8 locations), scripts/wisdom-extract-v2.py (11+ locations), scripts/wisdom-extract.py (parallel), scripts/substack-wisdom-extract.py, scripts/audit-sample.py, scripts/fix-info-theory-tags.py. This is bead craig-wisdom-rank10-tighten-rlj (already scoped for the strip). Do this first; it clears the largest category and lets the web app render from facts.

2. JMU #14–#17: add curation labels to study-guide.md, study-guide-tier1-revised.md, urgent-todos.md, wisdom-engine.md. One-line edit per file. Trivial. Ship next so the human surfaces are honest while the machine JSON is being restructured.

3. JMJ #10: blog/metadata.json topics + add topic_evidence sidecar. 795 records. Consumers: wisdom-cleanup (pillar reassignment), audit-sample, fix-info-theory-tags, tweet-process, topic-page generators, web app. Blast radius smaller than #1 but touches more scripts. Deferred until #1 lands because pillar (#8) inherits from topics[0].

4. *JMJ #8: wisdom/.json pillar (or downgrade to pillar_candidates).** Blast radius = wisdom files + web app pillar-filtering. Blocked by #3 (pillar honesty depends on topic honesty).

5. *JMJ #7: wisdom/.json type → add type_pattern_hits sidecar.** Small. Do alongside #1.

6. JMJ #12: web/server.py TWEET_ACCOUNTS score → settings.json attribution_evidence. Small footprint (5 accounts), Python-code edit only, no JSON schema churn.

7. JMJ #11: patents/inventory.json — per-count provenance. 20+ aggregate fields. Session 18 already flagged this partially done; finish the grounding.

8. JMJ #13: tweets/tweet-blog-crossref.json — move ≥4 threshold to settings.json, drop histogram summary. Low priority; record shape is already honest via _provenance_note.

❓ OPEN Questions (route to Adam)

1. type field: fact or judgment? determine_type() runs a regex-hit tally per (foundational_claim / definition / explanation / …) and takes argmax. The individual regex hits are facts; the argmax label is a judgment. Ship option (a) drop type, expose type_pattern_hits dict; (b) keep type as argmax convenience + expose type_pattern_hits with a _provenance_note. (b) is pragmatic, (a) is stricter. Which?

2. pillar field: is inheritance-from-topic OK? If blog/metadata.json.topics becomes honest (#3 above), does an insight-level pillar inherited from topics[0] count as JMJ or as an honest downstream tag? Or must every insight expose its own pillar_candidates: [{name, keyword_hits}] even if it duplicates the blog-level tag?

3. *audits/.json verdict field (honest/under-tagged/mis-tagged):** the whole point of an audit is to record an agent's judgment about a classifier. Is verdict a JMJ violation because it's in machine JSON, or is audits/ itself the label (audit outputs are meta-data-about-classification-quality, not data-about-Wright)? Recommend: not a violation; add a top-level _note: "verdict values are the audit-agent's judgment, not primary archive facts" to each audit JSON.

4. consequences — drop or replace? Currently 100% boilerplate (10 template strings cover 85% of records). Drop entirely, or replace with an empty-list default so downstreams that check .get("consequences", []) don't break?

5. wisdom-engine.md — supersede or delete? The document reads as active spec but is now the rationale for the largest violation cluster. Add supersession note (my recommendation) or delete outright?

Notes

Every violation above is a design problem, not a data problem — regenerating wisdom/insights.json with impact_rank stripped is a one-script change (scripts/wisdom-extract-v2.py), not 10,770 file edits.
The audit-sample.py cron is currently scheduled for [blog-tags, wisdom-rank9plus] (settings.json:audit_sample_targets). If we strip impact_rank, wisdom-rank9plus becomes unfilterable — either rewrite the audit target to wisdom-highwordcount (fact-based) or add a rank9plus_slugs.json sidecar exported from a script that applies whichever bucket rule Adam wants for auditing. Note the dependency.
Nothing in books/, interviews/, court/, talks/, satoshi/, ghost-blog/, substack/ (beyond wisdom) carries judgment fields. Those domains are FROZEN-track (copy) archives — the fact-honest layer of the archive is already correct.

_End of audit._