- Python 86.7%
- JavaScript 7%
- CSS 2.6%
- HTML 1.8%
- Shell 0.9%
- Other 1%
Jay Honnold
28877ca52b
Fixes #10 **Defect 1: Worker dies on transient DB error** The `except` handler in `_drain` called `jobs.fail()` unwrapped. If `jobconn` was dead, the exception escaped and killed the worker, leaving the job stuck `running`. **Fix:** Wrap `fail()` in its own try/except — the heartbeat reaper recovers the orphan. **Defect 2: No retry backoff; head-of-line blocking** A persistently failing job burns all attempts back-to-back, blocking every other queued job. **Fix:** Exponential backoff (`base^attempt`, capped at `max`) between failed attempts. Configurable via `AI_WIKI_RETRY_BACKOFF_BASE` (default 5s) and `AI_WIKI_RETRY_BACKOFF_MAX` (default 300s). Reviewed-on: https://gitea.home.arpa/jhonnold/ai-wiki-service/pulls/68 Co-authored-by: Jay Honnold <jayhonnold@gmail.com> Co-committed-by: Jay Honnold <jayhonnold@gmail.com> |
||
|---|---|---|
| .gitea/workflows | ||
| deploy | ||
| docs/plans | ||
| scripts | ||
| src/ai_wiki_service | ||
| tests | ||
| .dockerignore | ||
| .env.example | ||
| .gitignore | ||
| .schema.yaml.example | ||
| CLAUDE.md | ||
| Dockerfile | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
ai-wiki-service
A self-synthesizing AI knowledge wiki (Karpathy LLM-wiki pattern). The successor to
ai-wiki-mcp: instead of the calling agent doing synthesis, this service does it itself.
Three parts:
- UI — vanilla JS/HTML, served by the server (ingest box, file tree, wikilink graph).
- Server — FastAPI HTTP API for the UI + a read-only FastMCP surface for external agents to
trace the wiki. Writes only
raw/. - Worker — a LangGraph agentic loop that, on each ingested raw document, runs the import + synthesis process against a configured model (via the LiteLLM proxy) and commits the result.
State (job queue + LangGraph checkpoints) lives in Postgres. The wiki itself is a git repo of markdown — one commit per import/synthesis run. A periodic lint pass is the correctness safety net; there is no human review gate.
See docs/plans/ for the implementation plan.
Commands
uv sync # install deps
uv run pytest # run tests (offline; DB tests need AIWIKI_TEST_DSN)
uv run ruff check src tests # lint
For the full local dev loop (rootless Postgres, env, running the server/worker, and the
manual end-to-end spikes), see CLAUDE.md.
Deployment (llm-server / VM 215)
Built as one image with two entrypoints; docker-compose runs it as three services
(ai-wiki-service-{db,server,worker}) on the existing app-net bridge.
# 1. Image is built + pushed to the Gitea registry automatically by CI on merge to main
# (.gitea/workflows/ci.yaml; bump pyproject version per PR). To build by hand instead:
# docker build -t 192.168.10.32:3000/jhonnold/ai-wiki-service:<ver> . && docker push …
# 2. One-time host provisioning (creates + git-inits the wiki tree, seeds from live ai-wiki):
bash deploy/bootstrap.sh # run on VM 215
# 3. Add the three services from deploy/docker-compose.snippet.yaml into
# ~/projects/llm-server/docker-compose.yaml, add the AIWIKI_* vars to its .env, then:
docker compose up -d ai-wiki-service-db ai-wiki-service-server ai-wiki-service-worker
The UI + read-only MCP are then at http://192.168.10.40:3011/app/ and …/mcp. The
worker auto-synthesizes each ingested source and runs a lint pass after every
AI_WIKI_LINT_AFTER_N syntheses. See .env.example for all settings.