hindsight-gate

An energy-aware retain queue that sits in front of Hindsight's memory API. It is a transparent HTTP reverse proxy with one special case: retain is accepted instantly, coalesced per document into a durable queue, and drained to Hindsight one op at a time, only during configured time-of-day windows — so GPU-heavy fact extraction is deferred off-peak and never fights interactive inference for the GPU. No memory writes are lost; each document is extracted once per window instead of once per turn.

Why it exists

Every "retain" drives the GPU: Open WebUI re-sends the entire accumulated chat document every exchange (append), and Claude Code re-sends the full session transcript every few turns (replace). Both POST async:true, so Hindsight starts extraction immediately on its own worker — there is no pause/throttle endpoint. The gate is the missing throttle.

How it works

A transparent reverse proxy to the upstream (hindsight:8888) with one special case:

• Retain — POST /v1/{ns}/banks/{bank}/memories → accepted instantly, coalesced into a durable per-document queue, a synthetic RetainResponse returned immediately (producers read only operation_id). Nothing is forwarded now; it is drained later.
• Everything else (/recall, /config, /health, /operations, /openapi.json, …) → forwarded verbatim in real time. Recall stays low-latency (one local hop), so a single base-URL swap on a producer is safe.

Coalescing

coalesce_key = (namespace, bank, document_id), at most one pending op per key. An incoming replace drops the pending op and supersedes it; two appends merge as pending + "\n" + incoming — byte-identical to two sequential server-side appends. So an N-exchange Open WebUI chat collapses to one queued append, and an N-turn Claude Code session to one queued replace. Newest call's tags/metadata win. (Pure logic in app/reducer.py.)

Draining

A single worker drains one op at a time (gate-side concurrency = 1), only during the windows in config.yaml — the gate is the sole GPU pacing authority for memory. It submits each op with async:true, persists the returned operation_id on the inflight row, then polls the namespaced /v1/{ns}/banks/{bank}/operations/{id} until terminal before claiming the next row. Per-op runtime is bounded by drain.op_deadline_seconds (default 3600s); a too-slow op is demoted and retried next window (idempotent per (document_id, content_hash)). A restart resumes polling a persisted op id instead of re-submitting; an inflight row with no id (crashed before submit) demotes to pending. The Hindsight operations-status contract is encoded in one place: app/scheduler.py::_parse_operation_status.

Durable store

SQLite + WAL at $HINDSIGHT_GATE_DATA/queue.db. The enqueue commits before the synthetic op id is returned (no loss on crash). Row states pending → inflight → deleted on terminal success; on restart app/queue.py migrates the schema and recovers inflight rows.

Configuration

config.yaml is baked into the image as a default; bind-mount your own at /app/config.yaml to override:

Key	Meaning
timezone	TZ for evaluating windows (e.g. America/Phoenix)
hindsight_base_url	upstream Hindsight (e.g. http://hindsight:8888)
windows	list of {start, end} HH:MM ranges (may wrap midnight)
drain.spacing_seconds	GPU breathing room between ops
drain.submit_timeout_seconds	httpx timeout for the submit POST + each poll
drain.op_poll_interval_seconds	poll cadence while an op runs
drain.op_deadline_seconds	per-op ceiling before demote/retry
drain.poll_interval_seconds	idle re-check cadence
proxy_timeout_seconds	passthrough (recall/config/health) timeout

Optional env overrides are documented in .env.example (HINDSIGHT_GATE_{UPSTREAM,TZ,CONFIG,DATA}). No secrets — the service is internal and unauthenticated.

Endpoints

• GET /healthz — liveness (stays 200 even if Hindsight is down — the gate still queues).
• GET /metrics — Prometheus text. Series: hindsight_gate_retain_received_total, _coalesced_total, _submitted_total, _drained_total, _drain_failed_total, _drain_timeout_total, _queue_depth, _queue_pending, _queue_inflight, _last_drain_timestamp_seconds, _paused, _in_window.
• POST /admin/flush — drain now until empty, ignoring window + pause.
• POST /admin/pause / POST /admin/resume — halt/resume draining.
• Retain intercept: POST /v1/{ns}/banks/{bank}/memories.
• Everything else: transparent passthrough to the upstream.

Develop

uv sync                              # install deps (incl. dev: pytest, ruff)
uv run pytest -q                     # offline unit tests (no live Hindsight)
uv run ruff check app tests          # lint
uvicorn app.main:app --reload        # local run; set HINDSIGHT_GATE_UPSTREAM

Single worker only — never pass --workers. The one-pending-per-key invariant and single-drain pacing assume one event loop over one SQLite file.

Releasing

Semver lives in pyproject.toml [project].version. Every PR must bump it (and re-run uv lock) or the version-check Gitea workflow fails. On merge to main, ci.yaml's release job builds + pushes the image to the Gitea registry and pushes an annotated vX.Y.Z tag:

docker pull 192.168.10.32:3000/jhonnold/hindsight-gate:<version>   # or :latest

Deployment

Consumed by the homelab LLM server (VM 215, llm.home.arpa) via the llm-server repo's compose stack, which pins the image by digest and runs it as the hindsight-gate service published on :8889. See that repo for deploy mechanics.