zzstoatzz.io
Commits
dogfooding zigman to write an HTTP server surfaced two things; both fixed within
the existing `zigman <query>` surface:
- queries < 3 chars that aren't an exact section go straight to no-match instead of
substring-matching unrelated titles ("Io" -> "Introduction"/"Test-Declarations").
exact 2-char sections like `if` still resolve.
- a no-match now states what zigman *is* — "(the langref covers the Zig language,
not std)" — so a std-ish miss reads as out-of-scope, not a broken tool. it does
NOT instruct how to find std (zig env / std-source grep): that would make the
binary editorialize about the rest of the toolchain, and it's wrong on typos.
the std-lookup how-to stays in the guide/skill where it belongs.
11/11 tests, fmt clean, binaries rebuilt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a 401/403 from a hosted provider's /models means the endpoint is there; only a
connection failure means unreachable. lets hosted providers pass the reachability
check (surfaced wiring z.ai, where Pi otherwise masks a billing-error-200 as empty
content + retries).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the monolithic run.py (messy run_loop) becomes a src-layout uv project mirroring
does-it-tool: tasks.py (corpus + compiler oracle), config.py (resolution, no
Path.home — model via arg/env, zigman via PATH, skill by walking up), pi.py (Pi
runner + ArmResult), loop.py (ablate -> Report, render, save), cli.py (the
zigman-eval entry point). pure stdlib; pytest validates the oracles.
uv run zigman-eval selftest
uv run zigman-eval run --model <id> # or ZIGMAN_EVAL_MODEL
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docs/guide.md: how to use zigman in practice — fuzzy lookups, discovery,
everyday workflows, composition/piping, and using zigman from an AI agent via
the skill. rendered to www/guide.html by md2html (same path as the README),
linked from the landing page.
- rebuild www: binaries now carry the resolution improvements (optional->Optionals,
error union->Error Union Type, no multi-word dead-ends).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the loop is now one stdlib-only file (evals/run.py) orchestrating two CLIs:
Pi (@earendil-works/pi-coding-agent, headless `-p --mode json`, native --skill)
as the agent, and `zig test` as the objective oracle. gemma-4 via mlx_vlm.server
returns real structured tool_calls (the thing ollama+gemma3/qwen never did), so a
competent local agent finally drives the loop end-to-end.
removes eval.py (pydantic-ai), opencode_runner.py, prove.py — opencode's overhead
was what broke weak models; Pi (lighter) covers every role we used it for.
DESIGN.md updated to the Pi/gemma-4/MLX stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- zigman arm is now a plain pydantic-ai function tool + a one-line nudge, not the
SkillsToolset preamble that destabilized weak-model tool formatting. confirms
tools-via-pydantic-ai work: gemma3-tools writes code via write_file and runs
zig_test (sentinel-slice passes).
- opencode_runner.py: headless opencode runner (model-agnostic), parses JSON
events for the trace, scores with the same oracle. works mechanically end-to-end
(hello.txt); local models bottleneck on ollama tool-calling.
finding: organic local-model validation is gated by per-model ollama tool-calling
quirks, not the harness — gemma3-tools tool-calls but is too weak at zig; qwen2.5
writes good zig but ollama doesn't parse its bare-JSON tool calls.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
loop-derived (a not-too-smart agent types natural queries, not exact anchors):
- primaryMatch: when several sections match, open the one whose normalized
title/id equals the query (modulo plural) or uniquely prefixes it — so
`optional`->Optionals, `error union`->Error Union Type, `slice`->Slices,
instead of dumping a candidate list. genuinely ambiguous queries (`sentinel`)
still list.
- collectWords: a multi-word query that matches nothing as a phrase retries on
individual words, so `labeled block` surfaces Blocks + Labeled-switch/while/for
instead of "no section matching". no dead ends.
both stay within the existing `zigman <query>` surface (no new flags). +regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- provider registry maps provider -> (base_url, api_key_env); one OpenAI-compatible
code path drives dumb local and smart cloud models. roles are independent specs:
--agent-model (under test) and --judge-model (evaluator), bare name => ollama
- 7 reference-dependent tasks (intCast single-arg, for-index, enum switch,
labeled block, error-union try, sentinel slice/len); oracles force RUNTIME
inputs so `zig test` executes the code, not just compiles it. selftest 7/7
- zigman arm now loads the real SKILL.md via SkillsToolset (the product), not a
hardcoded nudge — and the finding is that this overhead re-breaks gemma's tool
formatting (documented in DESIGN.md)
- DESIGN.md: model-agnostic design, the yak-shave framing, sample_report.json fixture
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the eval agent now gets real tools (write_file, zig_test, zigman) and does the
task agentically — code flows through a tool arg (no fenced-prose parsing
failures), it can compile/read errors/iterate, and an llm judge scores the
trace. proven end-to-end on local gemma3-tools (writes file, compiles, oracle
passes, judge returns structured PASS).
DESIGN.md frames the goal (a self-improvement loop) and the open questions for
review. prove.py demonstrates the tool-call + serving-layer findings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- skills/zigman/SKILL.md: cross-client (agentskills.io) skill — read the langref
with zigman instead of guessing zig semantics
- evals/eval.py: ablation harness in does-it-tool's style (pydantic-ai over any
OpenAI-compatible endpoint). validator is the zig compiler, not an LLM judge.
selftest validates every oracle against a reference solution before spending
tokens; run does the bare-vs-zigman ablation and dumps JSON evidence
- evals/AXES.md: the dimensions the quality loop scores (api sensibility,
discoverability, output quality, ergonomics, portability, perf, robustness,
maintainability, aesthetics, distribution)
first baseline (gemma3-tools, ollama): bare 3/5, zigman 2/5 — and the model
never actually called zigman. signal: tasks need a real knowledge gap, and the
model must be induced to use the tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
reuses the same query resolution as showing: a unique match opens
https://ziglang.org/documentation/<ver>/#<anchor>, an ambiguous one lists the
candidates, and `zigman -o` with no query opens the reference index. uses
$BROWSER, else the platform opener (open / xdg-open).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the previous commit piped `zig build release` through head, which SIGPIPE'd the
page-generation step and committed a stale www/index.html.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- remove the "single static binary: no browser, no runtime, no services" line
- keep intro + install + a few usage examples visible; tuck build-from-source,
completion, and options into <details> blocks (scannable, punchline-first)
- md2html passes <details>/<summary> through verbatim and styles them
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the old README told users to write _zigman into ~/.zfunc (a dir that may not
exist and isn't on fpath). switch to `eval "$(zigman --completions <shell>)"`,
which registers in the current shell with nothing to create. the zsh script now
ends with `compdef` so it works when eval'd rather than autoloaded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- langref version flag is now -V/--version <ver> (was --zig); "version" reads
naturally as "which docs version" in a reference tool
- replace the confusing `sentinel` example with `error`/`pointer`
- rewrite README for clarity; generate www/index.html from it at build time via
a tiny in-repo md->html tool (tools/md2html.zig) so page and README never drift
- drop the prominent trafilatura credit (all code is our own; only a generic
"preserve structure" idea was borrowed) — keep a short note in render.zig
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
drop the show/search/list verbs for a single positional query, modeled on
rg/man/tldr. a query that uniquely names a section opens it; an ambiguous one
lists candidates (the old "search", surfaced only when needed); -l forces the
list. removes the two-step "search to find the anchor, then show it" dance.
- pager on a tty ($PAGER, default `less -FRX`), raw when piped
- shell completions: zigman --completions zsh|bash|fish
- rename langref version flag -v/--version -> --zig; free --version for the tool version
- data on stdout, diagnostics (match-count hint) on stderr
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a single static Zig binary that fetches the langref (one static HTML page with
stable id anchors), caches it, and prints any section as clean markdown — code,
tables, and lists preserved. no browser, no runtime, no external services.
- src/{main,root,render}.zig — CLI + library facade + html→markdown renderer
- search / list / show / version-pinned / XDG-cached
- `zig build release` cross-compiles macos+linux (aarch64/x86_64) into www/
- served as a tangled Git site; install via curl | sh
renderer structure-preservation inspired by trafilatura (not ported; see CREDITS.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dogfooding zigman to write an HTTP server surfaced two things; both fixed within
the existing `zigman <query>` surface:
- queries < 3 chars that aren't an exact section go straight to no-match instead of
substring-matching unrelated titles ("Io" -> "Introduction"/"Test-Declarations").
exact 2-char sections like `if` still resolve.
- a no-match now states what zigman *is* — "(the langref covers the Zig language,
not std)" — so a std-ish miss reads as out-of-scope, not a broken tool. it does
NOT instruct how to find std (zig env / std-source grep): that would make the
binary editorialize about the rest of the toolchain, and it's wrong on typos.
the std-lookup how-to stays in the guide/skill where it belongs.
11/11 tests, fmt clean, binaries rebuilt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a 401/403 from a hosted provider's /models means the endpoint is there; only a
connection failure means unreachable. lets hosted providers pass the reachability
check (surfaced wiring z.ai, where Pi otherwise masks a billing-error-200 as empty
content + retries).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the monolithic run.py (messy run_loop) becomes a src-layout uv project mirroring
does-it-tool: tasks.py (corpus + compiler oracle), config.py (resolution, no
Path.home — model via arg/env, zigman via PATH, skill by walking up), pi.py (Pi
runner + ArmResult), loop.py (ablate -> Report, render, save), cli.py (the
zigman-eval entry point). pure stdlib; pytest validates the oracles.
uv run zigman-eval selftest
uv run zigman-eval run --model <id> # or ZIGMAN_EVAL_MODEL
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docs/guide.md: how to use zigman in practice — fuzzy lookups, discovery,
everyday workflows, composition/piping, and using zigman from an AI agent via
the skill. rendered to www/guide.html by md2html (same path as the README),
linked from the landing page.
- rebuild www: binaries now carry the resolution improvements (optional->Optionals,
error union->Error Union Type, no multi-word dead-ends).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the loop is now one stdlib-only file (evals/run.py) orchestrating two CLIs:
Pi (@earendil-works/pi-coding-agent, headless `-p --mode json`, native --skill)
as the agent, and `zig test` as the objective oracle. gemma-4 via mlx_vlm.server
returns real structured tool_calls (the thing ollama+gemma3/qwen never did), so a
competent local agent finally drives the loop end-to-end.
removes eval.py (pydantic-ai), opencode_runner.py, prove.py — opencode's overhead
was what broke weak models; Pi (lighter) covers every role we used it for.
DESIGN.md updated to the Pi/gemma-4/MLX stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- zigman arm is now a plain pydantic-ai function tool + a one-line nudge, not the
SkillsToolset preamble that destabilized weak-model tool formatting. confirms
tools-via-pydantic-ai work: gemma3-tools writes code via write_file and runs
zig_test (sentinel-slice passes).
- opencode_runner.py: headless opencode runner (model-agnostic), parses JSON
events for the trace, scores with the same oracle. works mechanically end-to-end
(hello.txt); local models bottleneck on ollama tool-calling.
finding: organic local-model validation is gated by per-model ollama tool-calling
quirks, not the harness — gemma3-tools tool-calls but is too weak at zig; qwen2.5
writes good zig but ollama doesn't parse its bare-JSON tool calls.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
loop-derived (a not-too-smart agent types natural queries, not exact anchors):
- primaryMatch: when several sections match, open the one whose normalized
title/id equals the query (modulo plural) or uniquely prefixes it — so
`optional`->Optionals, `error union`->Error Union Type, `slice`->Slices,
instead of dumping a candidate list. genuinely ambiguous queries (`sentinel`)
still list.
- collectWords: a multi-word query that matches nothing as a phrase retries on
individual words, so `labeled block` surfaces Blocks + Labeled-switch/while/for
instead of "no section matching". no dead ends.
both stay within the existing `zigman <query>` surface (no new flags). +regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- provider registry maps provider -> (base_url, api_key_env); one OpenAI-compatible
code path drives dumb local and smart cloud models. roles are independent specs:
--agent-model (under test) and --judge-model (evaluator), bare name => ollama
- 7 reference-dependent tasks (intCast single-arg, for-index, enum switch,
labeled block, error-union try, sentinel slice/len); oracles force RUNTIME
inputs so `zig test` executes the code, not just compiles it. selftest 7/7
- zigman arm now loads the real SKILL.md via SkillsToolset (the product), not a
hardcoded nudge — and the finding is that this overhead re-breaks gemma's tool
formatting (documented in DESIGN.md)
- DESIGN.md: model-agnostic design, the yak-shave framing, sample_report.json fixture
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the eval agent now gets real tools (write_file, zig_test, zigman) and does the
task agentically — code flows through a tool arg (no fenced-prose parsing
failures), it can compile/read errors/iterate, and an llm judge scores the
trace. proven end-to-end on local gemma3-tools (writes file, compiles, oracle
passes, judge returns structured PASS).
DESIGN.md frames the goal (a self-improvement loop) and the open questions for
review. prove.py demonstrates the tool-call + serving-layer findings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- skills/zigman/SKILL.md: cross-client (agentskills.io) skill — read the langref
with zigman instead of guessing zig semantics
- evals/eval.py: ablation harness in does-it-tool's style (pydantic-ai over any
OpenAI-compatible endpoint). validator is the zig compiler, not an LLM judge.
selftest validates every oracle against a reference solution before spending
tokens; run does the bare-vs-zigman ablation and dumps JSON evidence
- evals/AXES.md: the dimensions the quality loop scores (api sensibility,
discoverability, output quality, ergonomics, portability, perf, robustness,
maintainability, aesthetics, distribution)
first baseline (gemma3-tools, ollama): bare 3/5, zigman 2/5 — and the model
never actually called zigman. signal: tasks need a real knowledge gap, and the
model must be induced to use the tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
reuses the same query resolution as showing: a unique match opens
https://ziglang.org/documentation/<ver>/#<anchor>, an ambiguous one lists the
candidates, and `zigman -o` with no query opens the reference index. uses
$BROWSER, else the platform opener (open / xdg-open).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- remove the "single static binary: no browser, no runtime, no services" line
- keep intro + install + a few usage examples visible; tuck build-from-source,
completion, and options into <details> blocks (scannable, punchline-first)
- md2html passes <details>/<summary> through verbatim and styles them
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the old README told users to write _zigman into ~/.zfunc (a dir that may not
exist and isn't on fpath). switch to `eval "$(zigman --completions <shell>)"`,
which registers in the current shell with nothing to create. the zsh script now
ends with `compdef` so it works when eval'd rather than autoloaded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- langref version flag is now -V/--version <ver> (was --zig); "version" reads
naturally as "which docs version" in a reference tool
- replace the confusing `sentinel` example with `error`/`pointer`
- rewrite README for clarity; generate www/index.html from it at build time via
a tiny in-repo md->html tool (tools/md2html.zig) so page and README never drift
- drop the prominent trafilatura credit (all code is our own; only a generic
"preserve structure" idea was borrowed) — keep a short note in render.zig
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
drop the show/search/list verbs for a single positional query, modeled on
rg/man/tldr. a query that uniquely names a section opens it; an ambiguous one
lists candidates (the old "search", surfaced only when needed); -l forces the
list. removes the two-step "search to find the anchor, then show it" dance.
- pager on a tty ($PAGER, default `less -FRX`), raw when piped
- shell completions: zigman --completions zsh|bash|fish
- rename langref version flag -v/--version -> --zig; free --version for the tool version
- data on stdout, diagnostics (match-count hint) on stderr
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a single static Zig binary that fetches the langref (one static HTML page with
stable id anchors), caches it, and prints any section as clean markdown — code,
tables, and lists preserved. no browser, no runtime, no external services.
- src/{main,root,render}.zig — CLI + library facade + html→markdown renderer
- search / list / show / version-pinned / XDG-cached
- `zig build release` cross-compiles macos+linux (aarch64/x86_64) into www/
- served as a tangled Git site; install via curl | sh
renderer structure-preservation inspired by trafilatura (not ported; see CREDITS.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>