odd — design#

Keep per-switch HTML documentation continuously up to date by hooking into opam's session lifecycle and rebuilding only the packages each session actually changed, using odd-driver (from the odoc driver, opam mode) as the per-package generator.

Goals#

• After any successful opam install / opam upgrade / opam remove, the switch's HTML docs reflect the new state, with no manual step.
• Incremental: only packages (re)built or removed in the session are reprocessed.
• Per-switch: all state and output live under the switch prefix and vanish with opam switch remove.
• Fail-safe: a documentation failure never causes the opam invocation to fail.

Non-goals#

• Multi-universe handling (docs.ocaml.org's problem). A switch is a single coherent universe: exactly one version of each package, all built together — which is why opam mode drops voodoo mode's universe/blessed machinery and uses the flat <package>/ layout.
• Parallel doc building across packages (possible later; see Open questions).

The generator: odd-driver#

A variant of odoc_driver (opam mode) designed for exactly this job. It is vendored into this repo under driver/ (the whole odoc_driver_lib plus the opam-mode entry point, built and installed as the odd-driver binary alongside odd) — a temporary measure pending upstreaming of the changes into odoc-driver; until then odd carries its own copy so it is self-contained. Per invocation it documents one package installed in the current switch:

• Package file lists come from opam's own record, <prefix>/.opam-switch/install/<pkg>.changes — no prepared input tree, no file copying; installed artifacts are read in place.
• The switch is discovered by shelling out to opam switch show / opam var prefix (both read-only operations).
• --odoc-dir, --odocl-dir and --html-dir all default to <switch-prefix>/odoc. Intermediate .odoc/.odocl files and HTML share that tree, with each package's output directly under odoc/<package>/ (flat layout, same as plain odoc_driver). This is an odd-owned tree, kept separate from opam's own <prefix>/doc, where packages install their own documentation.
• Previously built dependencies are discovered by scanning --odoc-dir for the .odoc_pkg_marker / .odoc_lib_marker files written on earlier runs — this is what makes incremental, package-at-a-time building work.
• Like the voodoo driver, it must be run package-by-package, with each package's dependencies already compiled in --odoc-dir. Ordering is our job (see Ordering).

So the wrapper tool's responsibilities reduce to: ordering the work and invoking the driver once per stale package (recording what changed and cleaning up removals are done directly by the coreutils hooks, see below).

Components#

1. opam hooks (configured in ~/.opam/config, shipped via opamrc)#

Three wrapper fields drive everything. The two per-package hooks are plain coreutils — crucially they invoke no odd binary — so they keep working even in a switch where odd isn't installed (or is mid-upgrade, or where a global hook fires for a switch that never had it). All real work happens once per session, in the one hook that does need the binary.

post-install-commands: [
  [ "sh" "-c" "mkdir -p \"$1/odoc/$2\" && touch \"$1/odoc/$2/.odd-stale\""
    "--" "%{prefix}%" "%{name}%" ] { error-code = 0 }
]
post-remove-commands: [
  [ "rm" "-rf" "%{prefix}%/odoc/%{name}%" ] { error-code = 0 }
]
post-session-commands: [
  [ "%{hooks}%/odd" "sync" "--prefix" "%{prefix}%" ] { success }
]

• post-install marks the package stale by touch-ing odoc/<pkg>/.odd-stale (after mkdir -p-ing the dir). It fires for every install action including same-version rebuilds — which matters, because a dependency-triggered rebuild changes the .cmtis but is invisible to session-level %{new}%/%{removed}%. A marker is a single empty file per package, so opam's parallel package actions can't corrupt anything (distinct files; touch is idempotent). The set of markers is the work list — no ordering or content is implied (order is recomputed at sync time, see Ordering below).
• post-remove deletes the package's doc subtree outright. Removal needs no marker and no deferral: the deletion is the action, done immediately and binary-free. (This also clears any stale marker that lived in that subtree.)
• post-session runs odd sync, the one hook that needs the binary. It always exits 0: doc failures are logged, never propagated, because opam aborts the invocation with a configuration error if a session hook fails (opamSolution.ml, post-session handling). If odd is absent, this hook simply doesn't run; the markers persist and are drained by the next session that does have it — deferred, never lost.

2. State and layout (all under $OPAM_SWITCH_PREFIX)#

var/cache/odd/
  log              # sync output, since hooks must stay quiet
  lock             # guards a manual sync against a hook-invoked one
odoc/
  <package>/
    .odd-stale  # marker: post-install touched it, sync rebuilds
    ...                # per-package odoc, odocl and HTML (driver defaults)
  index.html       # switch-wide landing page (ours)
  odoc-search/...  # driver support files, search assets

The driver's defaults are taken as-is: one odoc/ tree per switch holding both intermediates and HTML, separate from opam's own <prefix>/doc. The work list lives in that tree too — a .odd-stale marker file inside each stale package's dir — so the post-install hook can write it without odd. Our only other state is the log and lock.

3. The sync step#

Runs once at session end, under opam's switch lock (post-session executes inside the opam invocation), so no extra locking is needed against concurrent opam sessions. Take a private lock file anyway to guard against a manually-invoked sync racing a hook-invoked one. The hook environment is the switch environment, so the driver's opam switch show resolves to the right switch; being read-only, the nested opam calls don't contend with the lock the surrounding session holds.

1. Collect the stale set: the package names whose odoc/<pkg>/ holds a .odd-stale marker, intersected with the installed set. (A marker for a no-longer-installed package can only survive an install+remove in the same session — the remove hook's rm -rf normally clears it — so tidy it away.) Removals need no handling here: the post-remove hook already deleted the docs.
2. Compute dependency order over the stale set (see Ordering).
3. For each stale package, in order: delete odoc/<pkg>/ (a clean build — the flat layout has no version in the path, so this prevents files from modules that no longer exist surviving from a previous version, and it removes the marker), then run odd-driver <pkg> --actions all (all directory options left at their switch defaults). On success the marker stays gone; on failure re-create it (mkdir -p + touch) so the next session retries.
4. Regenerate the top-level index every session: a landing page listing all installed packages with built docs (directory listing of odoc/, filtered). Doing this unconditionally is what lets a session that only removed packages drop them from the page — there's no removal marker to trigger on. The write is skipped when the content is unchanged, so idle sessions stay cheap. The per-package pages and redirects are the driver's job; only this one page is ours.

4. Ordering#

odd-driver must see a package's dependencies already compiled in --odoc-dir, so stale packages are processed in dependency order. Two facts make this tractable:

• The stale set is closed under reverse dependencies. If A's rebuild could affect B's docs, opam rebuilt B too (that is what triggers recompilation), so B was marked stale by the post-install hook. We never need to add packages to the work list ourselves.
• Order is computed at sync time, never inferred from the markers. The markers are an unordered set (filesystem entries with no meaningful timestamps to trust), so order is always recomputed from package metadata — necessary anyway, since a marker left over from a failed earlier session can coexist with a freshly touched one for its dependency.

The graph is built from opam's installed-package metadata: $OPAM_SWITCH_PREFIX/.opam-switch/packages/<pkg>.<ver>/opam — the opam file as installed, correct for pins and local packages. Parse with the opam-format library (never by hand):

• collect every package name in depends: via OpamFormula.fold_left, ignoring version constraints and disjunction structure;
• exclude dependencies marked {post} — they are allowed to be circular and opam itself excludes them from build order;
• include depopts: — an installed optional dependency influenced the build, hence the docs;
• intersect the collected names with the installed set (the directory listing of .opam-switch/packages/ is that set). Whichever disjunct or optional dep is installed is the one that was used. Spurious edges from installed with-test/with-doc deps only constrain order; they cannot make it wrong.

Topologically sort stale against this graph (edges to non-stale packages are dropped — their docs are already current in --odoc-dir). A cycle (shouldn't happen with {post} excluded) is logged and broken arbitrarily rather than failing the run.

5. The compiler / stdlib#

The compiler's libraries (stdlib, compiler-libs, threads…) live under lib/ocaml and everything depends on them, so they must be documented first whenever they change. In our favour, they are installed by an ordinary package (ocaml-base-compiler / ocaml-variants / ocaml-system) whose .changes file covers lib/ocaml, so odd-driver's normal discovery path applies — no synthetic package needed, just the ordinary ordering edge, and a compiler change stales the whole world (correct: opam rebuilds the world anyway). Needs verification early in implementation: that the driver's library discovery (META-less lib-dir scanning) handles the lib/ocaml layout, and how the ocaml wrapper package (which installs almost nothing itself) behaves — possibly just a no-op build to skip.

Tool shape#

One OCaml executable, odd, with subcommands:

• odd sync — the session worker described above (the post-session hook).
• odd rebuild [--all | <pkg>...] — manual escape hatch: mark packages (or everything installed) stale and run sync. --all works on pre-existing switches because the installed-package metadata is always present, whether or not hooks were configured at install time.
• odd setup — write the three wrapper fields into ~/.opam/config with opam option --global (idempotent).

There is deliberately no record subcommand: recording a change (a touch) and cleaning up a removal (an rm -rf) are plain coreutils in the hooks, so they never depend on odd being installed.

Dependencies: opam-format (opam file parsing), bos, fpath, cmdliner. The vendored driver (driver/) pulls in its own cone (eio, progress, yojson, ocamlfind, logs, sexplib, ppx_sexp_conv, and the odoc / odoc-md / sherlodoc binaries it drives); odd sync still invokes odd-driver as a subprocess, so the driver's runtime cost stays in a separate process. odd-driver is built and installed by this package, so it is always on PATH next to odd (overridable with --driver / $ODD_DRIVER).

Distribution: an opamrc adding the three *-commands fields for opam init --config; odd setup for retrofitting existing roots.

Failure handling summary#

Open questions#

• Parallelism: stale-set topo order admits parallel builds of independent packages; odd-driver already parallelises units internally (nb_workers), which is probably enough for session-sized work lists.
• Search/occurrences: the driver emits per-package occurrence files and json search indexes; merging them into a switch-wide search index is a later sync step.
• Driver self-staleness: when the switch's odoc/driver itself is upgraded, every package's intermediates were produced by the old odoc. The closure property covers packages opam rebuilt, but odoc version changes may warrant a full rebuild --all — detectable by recording the driver version used per run.