Skip to content

Curated canon

The canon is the curated, governed context an answer is grounded in: the semantic model, the governed definitions, and the certified-definition pages a lens selects. Its opposite is dumping — pointing the model at the catalog, the auto-profiled data dictionary, everything you have. Dumping feels safer. It measures worse.

In our internal runs, adding a raw auto-profiled data dictionary made the model worse than no context at all on definitional questions; replacing it with curated definitions recovered the loss and beat baseline.

The mechanism behind the V: a dictionary teaches the model enough to compute a plausible answer, and nothing about which computation is sanctioned — context without definitions trades guessing for unauthorized deciding. More context is not more correctness; selected context is.

How the finding shaped the machinery

  • The semantic model is injected whole — no retrieval gamble on the facts the SQL must bind to. The scale lever is upstream scoping (the lens selects few entities), not truncation (services/runtime/generator.py).
  • Certified-definition pages are selected, not dumped: usage_mode: auto pages are always in context, search pages only when token-overlap relevance says so (services/certdefs/__init__.py). This knob exists because of the measurement above.
  • Prose context is retrieved top-6 from the lens's vector namespace, and the certified page is always the first context block, ahead of every retrieved chunk (services/runtime/assembly.py).

How canon is authored

The entity fields are exactly the ones that teach judgment, not just structure (services/contracts/shared_semantic.py): grain ("one row per closed deal"), use_cases — when to use and when to avoid, common_questions, join cardinality, and metric filters that are ANDed into the WHERE whenever the metric computes.

# semantic/entities/deals.yaml (excerpt)
grain: one row per closed deal
use_cases:
  - Use for bookings, deal counts, deal sizes, and new-vs-existing business questions.
  - Avoid for commission or payout questions - payouts carries what reps earned.

Definitions (semantic/definitions/<term>.md) are frontmatter + prose, optionally an enforceable sql_expr, and — for contested terms — status: ambiguous with the possible mappings, which turns guessing into asking (see Clarification & refusal). Per-metric certified-definition pages carry the verified end of the canon. Skills add terms and instructions to a lens but never override its governed definitions (services/contracts/skill.py).

How canon is selected

The lens selects; nothing flows in by default. select.definitions is deliberately empty until you name terms — every selected term becomes a routing anchor, so shared vocabulary must not fan into every lens automatically (services/project/compile.py).

In live-warehouse spot checks the failure mode of ungoverned access was never an obvious error: direct warehouse answers were plausible-but-wrong — test accounts included, invoice eras un-unioned, row versions double-counted. Canon and certified answers exist to close exactly that gap.