Tuning skills, subagents, and their context layout

This skill is about making skills and subagents cheaper per use and more correct, and about not getting fooled by plausible-but-wrong intuitions while doing it. The guidance here is empirically grounded — the headline verdicts below came from A/B tests, and a few of them overturned the "obvious" design. When you apply this skill, you measure before you presume.

The one mental model: hot vs cold

Every piece of context lives in one of two states. Internalize this — every decision flows from it.

The whole game is putting the right things in the right state. Hot is a standing tax; cold is pay-per-use. Optimizing is mostly moving rarely-needed detail from hot to cold, and keeping hot to {what's needed almost every time + what must never be skipped}.

Hard-won verdicts (measured, often counter-intuitive)

Lead with these. Several contradict the natural guess — that's exactly why they're worth stating.

1. **A grep/anchor index over a small, well-structured doc gives ~no retrieval benefit.**

Markdown headers are already grep targets; an agent greps ## 7. or a keyword on its own. Adding <!-- tag --> anchors + a token index measured worse (more lines read, same tool calls) on docs in the hundreds of lines. Don't add index machinery by reflex. → references/file-indexes.md

1. An index inside HOT content is pointless. If the whole file is already in context, there

is nothing to "grep to" — the agent has every line. Anchors in a hot file are dead weight.

1. The real token win is the TRIM, not the index. Moving mid-depth prose out of an

always-loaded file (CLAUDE.md / agent body) into a cold reference is what actually saves tokens. The index that often accompanies it is usually ceremony. → references/hot-vs-cold.md

1. **Cold references are read reluctantly.** On a neutral prompt, a subagent will answer from

its hot context + training and not open a relevant cold reference — even one engineered to be needed. It reads when: the fact is clearly project-specific or version-sensitive (a ground truth it knows it lacks), when it senses it can't recall a precise value, or when the prompt induces it. → references/ab-test-harness.md

1. The silent-skip failure is the dangerous one. If a critical fact lives only in a cold

reference and the model's training is stale/wrong, the agent never looks and answers confidently wrong — silently. Fixing this is a rules problem, and the fix is layered and measurable. → references/subagent-verification-rules.md

1. Provenance forcing is the cheapest robust fix. Requiring an agent to state where a

specific claim came from converts a silent confident-wrong answer into a visibly-flagged estimate, even when it still doesn't read the reference. It fired reliably across tests where the read-trigger only fired sometimes. → references/subagent-verification-rules.md

1. Measure, don't presume — and clean up after yourself. The A/B harness below is how every

verdict above was earned. Agent memory contaminates repeat tests; clean it between runs. → references/ab-test-harness.md, references/memory-hygiene.md

When to reach for what

Core workflow

Whatever the specific ask, the shape is the same: characterize → hypothesize → A/B → keep what wins.

1. Characterize. Read the target (skill, agent .md, CLAUDE.md, the doc). For every chunk,

ask: hot or cold? Used almost-every-time or occasionally? Load-bearing (must-never-skip) or optional depth? This classification is most of the analysis.

1. Hypothesize a change, and predict its effect in hot/cold terms. "Move §X to a reference"

→ saves hot tokens. "Add an anchor index" → predict ~no retrieval gain on a small doc (verdict 1); say so. "Add a verify rule" → predict it fires for project/version-specific facts.

1. A/B test it when the effect is non-obvious or the user wants proof. Same prompt, change

only the one variable (the rule, the skill, the doc layout) → clean causal attribution. Instrument via the transcript, not self-report: count tool calls, lines read (input proxy), correctness, and provenance honesty (claimed source vs actual tool call). Full protocol in references/ab-test-harness.md.

1. Keep what wins, revert what doesn't, and say what you measured. Don't ship ceremony. If

the index didn't help, drop it; if the trim saved tokens, keep it; if a rule fired only partially, report the limit honestly rather than overclaiming.

1. Clean up. If you ran subagent tests, scrub the memory they generated

(references/memory-hygiene.md) so it can't contaminate later work.

Anti-patterns this skill exists to stop

• Index-by-reflex. Adding anchors/TOC/grep-tokens to every doc "for navigability." Measure

first; on small docs it's cost without benefit (verdict 1).

• Hoarding hot. Letting CLAUDE.md / an agent body accrete mid-depth prose that's needed 5% of

the time. That prose is a per-turn tax. Push it cold.

• Burying must-apply rules cold. A non-negotiable, divergent-from-default, or version-pinned

fact placed only in a reference will be silently skipped (verdict 5). Critical → hot, or guard it with a verify/provenance rule.

• Presuming instead of measuring. "This is obviously better" is how the grep-index almost

shipped as dogma. Run the A/B; let the transcript decide.

• Leaving test memory behind. Subagent runs write agent-memory + reflection_store +

index entries that re-inject into later runs. Always clean.