Connect a warehouse¶

dst ships five warehouse connectors (services/lenses/connections.py:25):

Type	Read-only mechanism	Runaway-query cap
`duckdb`	every connection opened `read_only=True`	— (local file)
`postgres`	session forced `default_transaction_read_only=on`	statement timeout
`mysql`	autocommit read-only session	`MAX_EXECUTION_TIME`
`bigquery`	read-scoped credential	`maximum_bytes_billed` cost cap
`snowflake`	read-scoped role	statement timeout

Read-only is layered, not assumed: the SQL guard is the first line — generated SQL is SELECT-only by construction — and the read-only credential/session is the backstop (services/runtime/sql_guard.py). There is no write path for callers.

Declaring a connection¶

Connections are declared in dst.yaml, never created in the UI (services/project/schema.py:22):

connections:
  finance_wh:
    type: snowflake
    config: { account: acme-eu, warehouse: ANALYTICS_WH, database: FINANCE }
    secret_env: DST_API_KEY_FINANCE_WH

config holds the non-secret settings; the credential lives in .env under the env var secret_env names (convention: DST_API_KEY_<NAME>). For BigQuery, point the env var at the service-account JSON with the @/path/to/key.json idiom instead of pasting the blob, and scope a wide project with datasets: [finance_marts, product_marts] — a real layer usually reads several datasets, and the pin scopes introspection and dst probe alike (dataset: and the generic schema: spelling are accepted too). A config key the connector doesn't read warns at apply and on dst introspect instead of sitting there silently. On dst apply, every declaration is probed — connect plus one read — before it lands; a dead credential never replaces a working one, the error names the env ref to fix, and the probed connection reports its capabilities on the apply output (read ✓ · query ✓ · query history ✗ — grant …), so a missing grant is a visible degradation at deploy time, not a silent nightly skip. See Project files for the full file model.

To try dst without a warehouse, dst init --warehouse demo uses a bundled DuckDB fixture that ships inside the package.

What dst needs from the warehouse¶

A read-only credential that can SELECT the tables your lenses expose and read the catalog (schema introspection).
Sampling reads during profiling: after a connection is created, a background chain profiles it — catalog, then value sampling, then column descriptions (services/api/mgmt_connections.py). Sampling runs through the connector's own read-only query path.
Optionally, the history catalog — Snowflake ACCOUNT_USAGE, BigQuery INFORMATION_SCHEMA.JOBS — to bootstrap from history. The mining SQL reads statements and metadata only, never row data.

What dst does not need¶

No write or DDL rights. Grant none; nothing in the serving path writes.
No agents or extensions installed in the warehouse. dst connects like any read-only client.
No copy of your tables. dst's own state — semantic assets, lens versions, traces, embeddings — lives in its own Postgres. Query results flow through to the caller; profiling stores sampled values and descriptions, not table extracts.

Where the server runs¶

One process (API + dashboard) plus one Postgres. dst dev runs both on a laptop (compose Postgres → migrate → serve on :8000); deploy/docker-compose.yml is the self-host recipe — a pgvector Postgres and the app container, with idempotent migrations on start. Postgres is the only stateful dependency.