Brain Ingest Gate — Resolve and Dedup Before Anything Enters the Brain

Convention: see [conventions/brain-first.md](../conventions/brain-first.md) —
the lookup chain (gbrain entity → search → query → get) is the same
chain this gate runs before every write.

Convention: see [brain-filing-rules.md](../brain-filing-rules.md) —
when the gate's verdict is "write", the primary subject picks the directory.

Convention: skills/conventions/quality.md owns the cross-cutting page
rules (citations, Iron Law back-linking, notability) — every page the gate
lets through follows them. Gate-specific delta: the gate only decides
write/link/skip; the admitting skill applies the quality rules on write.

The Rule

No content enters the brain without passing this gate. A raw cp or mv into the brain repo is a bug.

One insight, one place. If it already exists, link to it — don't clone it. Before any new page is written (file migration, bulk import, manual gbrain put, subagent output), two checks run in order:

1. Named-Entity Resolution Gate — is this about a named thing that

already has a page under its chosen name?

1. Dedup Gate — does the brain already state this insight somewhere?

Scope honesty: this gate is a routing convention — the harness resolves it into context when an ingest-shaped intent matches, and a well-behaved agent follows it. Nothing in the gbrain runtime mechanically blocks an unenriched or duplicate write if the skill never loads.

Why gbrain needs this gate

The native pipeline does NOT do semantic dedup for you:

• gbrain import / gbrain sync skip only matching frontmatter IDs.

Identical content under a different slug or ID indexes twice — every duplicate becomes a second search hit competing with the canonical page.

• gbrain capture's dedup is a 24-hour exact content-hash — it catches

re-captures of identical bytes, not the same insight reworded.

• The remember verb dedupes facts, not pages.

Semantic dedup and named-entity resolution are this skill's job, in full.

When This Gate Fires

1. File migration — moving files already in the workspace into the brain

repo ("move this to brain").

1. Bulk imports — batch moves of any kind into brain directories, BEFORE

gbrain sync or gbrain import indexes them. For batches, also read [conventions/test-before-bulk.md](../conventions/test-before-bulk.md): gate 3-5 items and inspect the decisions before running the rest.

1. Manual writes — gbrain put or gbrain capture of rich content, or

direct file writes into the brain repo.

1. Subagent output — background agents writing notes or pages into the

brain.

What This Gate Owns vs Delegates

This skill is a gate, not a pipeline. It owns the pre-write checks below. Everything downstream of a "write" verdict is delegated to shipped skills — do not restate their steps here or inline:

Named-Entity Resolution Gate (runs FIRST)

Fires whenever the content is about a NAMED project, place, company, person, or anything someone "wants to build / found / make."

Vector similarity alone cannot be trusted to catch named-entity dupes: a page stored under its chosen NAME will not embed close to the generic English phrase someone happens to describe it with. The classic failure: a search for a descriptive phrase scores the canonical named page below the prose floor, so a duplicate stub gets written on top of a years-old page. Stored by named meaning; retrieval attempted by literal generic phrase.

The rules

1. Resolve registry-first, not by the generic phrase. gbrain's native

registry is the entity surface:

``bash gbrain entity "<name>" # zero-LLM card: page, aka list, near-miss suggestions ``

A card hit means the page exists — STOP, link, don't clone. On a miss (or for concept-shaped nouns), fall through to gbrain query "<name>" --limit 3. If the brain also keeps an explicit index of named initiatives (e.g. a page under concepts/), read it before concluding anything is new.

1. Expand through aliases before searching. Named pages should carry an

aliases: frontmatter list (generic label + chosen name + any nickname + signature phrase). Search EACH alias and the generic label, not just the phrase the user happened to say.

1. A vector score is a floor for prose, NEVER a gate for named things.

If there is ANY plausible named match, open and read the candidate page (gbrain get <slug>) before concluding it doesn't exist. A named page can be the right answer at a score that would be a clear miss for prose.

1. When a NEW named thing appears, bake its aliases in the same write.

Create the page with the full aliases: list so every future synonym resolves through gbrain entity. One frontmatter list covers all future phrasings — O(1), not a per-instance reminder.

Why a gate and not a memory note

A memory reminder ("query the real name, not the generic phrase") is a per-instance sticky note: it only works if it happens to be in hot context that turn, doesn't generalize to the next named entity, and rots. This skill loads when an ingest-shaped task routes here. Process rules belong in the triggered gate, not in hot memory.

Dedup Gate (runs SECOND)

Before writing ANY new page (for named things, the resolution gate above runs first and takes precedence):

1. Extract the core claim — 1-2 sentences capturing what's novel about the

new content.

1. Search for it:

``bash gbrain search "<core claim>" --limit 5 ``

1. OPEN AND READ the top hit (gbrain get <slug>). Never band on the

score alone. Donor systems publish cosine cutoffs for this step — do NOT port them: gbrain search returns fused hybrid rank scores, not cosine similarity, and no numeric threshold maps across. The band comes from reading, not from the number.

1. Assign a band:

Decision tree

New content to write
  ├─ Named thing? → Named-Entity Resolution Gate first
  │    (entity card → alias-expanded search → READ the candidate)
  ├─ Extract core claim (1-2 sentences)
  ├─ gbrain search "<core claim>" --limit 5
  └─ OPEN AND READ the top hit (gbrain get <slug>)
      ├─ clear-dup     → STOP. Link to existing. Report "duplicate".
      ├─ plausible-dup → Read both. Same insight?
      │    ├─ yes → STOP. Link to existing. Report "duplicate".
      │    └─ no  → Write with cross-link. Report "new angle".
      └─ clear         → Write via enrichment skills. Report "unique".

When to skip dedup

• Operational/state files — time-series records, not knowledge.
• Meeting transcripts — each meeting is unique by definition (entities

INSIDE it still go through the named-entity gate via the delegated skills).

• Timeline entries on existing pages — back-links are additive, not

duplicative.

• Media files — dedup by filename/hash, not semantic similarity.

Verification

After the batch, verify the gate's output holds:

gbrain check-backlinks check            # mentioned entities link back (fix with: check-backlinks fix)
gbrain backlinks <new-slug>             # each new page has inbound links
gbrain search "<core claim>" --limit 3  # the insight has exactly ONE home

If check-backlinks check reports gaps on pages the gate just admitted, the enrichment delegation was skipped — route back through [enrich](../enrich/SKILL.md) before declaring the ingest done.

Contract

This skill guarantees:

• No new page enters the brain through this skill's flows without the

named-entity resolution check and the dedup check running first.

• Every "duplicate" verdict names the matched slug and produces a link or

timeline entry instead of a clone.

• New named-entity pages carry an aliases: frontmatter list in the same

write that creates them.

• Dedup bands are assigned by READING the top hit, never by score alone; no

numeric similarity thresholds are used against gbrain's fused scores.

• Enrichment is delegated to shipped skills (ingest, enrich, signal-detector,

concept-synthesis) — never restated or reimplemented inline.

• Batches end with a gbrain check-backlinks check verification pass.
• Routing matches the canonical triggers in the frontmatter.
• Output written under the directories listed in writes_to:.
• Privacy contract preserved: no real names, no fork-specific filesystem path

literals, no upstream-fork references.

The full behavior contract is documented in the body sections above; this section exists for the conformance test.

Output Format

One decision line per item checked, then the verification result:

Ingest gate — 3 item(s) checked

| item | entity resolution | band | action |
|---|---|---|---|
| notes-on-widget-co.md | resolved: companies/widget-co | clear-dup | linked (timeline entry on companies/widget-co) |
| pricing-thesis.md | n/a (prose) | plausible-dup | new angle — written to concepts/ with cross-link to concepts/pricing-power |
| charlie-example-intro.md | miss (near-miss: people/charlie-example) | — | read near-miss; same person → linked, no new page |

Verification: check-backlinks check → 0 gaps on admitted pages

Every "linked" or "duplicate" row MUST name the matched slug. If any row says "written", the enrichment delegation (which skill handled it) should be recoverable from the conversation.

Anti-Patterns

• ❌ cp file.md <brain-repo>/concepts/ — raw copy, no gate, no enrichment.
• ❌ Bulk mv of a folder into the brain repo, then gbrain sync — sync

happily indexes every duplicate; matching-ID skip will not save you.

• ❌ Trusting a low vector score as proof a named thing has no page — named

pages don't embed near generic descriptions of them.

• ❌ Banding on the search score without opening the top hit.
• ❌ Porting numeric dedup thresholds from other systems onto gbrain's fused

scores.

• ❌ Writing a new named page without its aliases: list — the next synonym

creates the next duplicate.

• ❌ Reimplementing entity detection, backlinking, or concept linking inline

instead of delegating to the shipped skills.

• ❌ Skipping the gate because the write is "just one page" via gbrain put —

single manual writes are where duplicate stubs come from.

Dedup (sharp boundaries)

• [capture](../capture/SKILL.md) — the quick-save front door; its dedup is

a 24h exact content-hash on identical bytes. This gate is the SEMANTIC + named-entity layer for content entering the brain as real pages (migrations, bulk imports, inbox graduation). "capture this thought" → capture; "migrate these files into the brain" → this gate.

• [ingest](../ingest/SKILL.md) — the router for NEW external content

(meetings, articles, media) and its enrichment pipeline. ingest decides what to DO with content; this gate decides whether a page should EXIST at all. The gate fires before the write; ingest and its specialized skills handle everything after a "write" verdict.

• [enrich](../enrich/SKILL.md) — page creation/update mechanics (tiers,

citations, timelines, backlinks) AFTER this gate says "write" or "link".

• [concept-synthesis](../concept-synthesis/SKILL.md) — retroactive,

at-scale dedup of concept stubs that already slipped in. This gate is prevention at write time; concept-synthesis is the cleanup pass. "dedupe my existing concepts" → concept-synthesis.

• frontmatter-guard (host-side) — the same standalone-gate pattern on an

orthogonal axis: structural validity of what's written vs (here) semantic novelty of whether to write.

• [bulk-ingestion](../bulk-ingestion/SKILL.md) — the bulk sibling. Its

pipeline dedup key (source + source_id) only makes RE-RUNS idempotent; it does not catch cross-source duplicates or resolve named entities. This gate is the semantic + named-entity layer bulk-ingestion runs on its Phase 3 trial items and bakes into the codified pipeline (its Phase 1d/6). "Build a large-corpus pipeline" → bulk-ingestion; "does this page already exist before I write it" → this gate.

• [data-loss-gate](../data-loss-gate/SKILL.md) — the inverse gate: it

stops data LEAVING the brain without confirmation; this gate stops data ENTERING without resolution + dedup.