ocaml-observe/test/ at main · gazagnaire.org/ocaml-observe · Tangled

gazagnaire.org / ocaml-observe

0

Logging, runtime tracing, and the obs performance profiler for OCaml

0

ocaml-observe / test

9 folders 4 files

at main

Thomas Gazagnaire obs: profile native CPU with xctrace 5d ago

obs: profile native CPU with xctrace Replace sample(1)'s blocked-thread snapshots with scheduled Time Profiler samples on macOS. Capture all processes once, retain process identities, and report the target alongside external CPU consumers. Save the native sidecar in every default bundle, load it from obs report, and handshake benchmark triggers only after recording is active.

5 days ago

obs: split the observe library from the obs CLI One opam package shipped both, so a program that wanted Cmdliner logging terms declared a dependency on sqlite, hdr, uuidm, cstruct, eio_main and monitor. Those belong to the profiler. obs now carries the CLI, its collector and its trace adapters; observe carries the library. observe also drops nox-tty, since Tty.Display now installs the log-suspending reporter wrapper itself. Dune ties a public_name to its package, so every observe.* sublibrary stays in observe, and the CLI's four libraries take public names under obs: obs.model, obs.fxt, obs.memtrace, obs.offline. They had to become public because dune forbids a public library depending on a private one, and being public brought their modules under merlint's untested-module rule. So they are tested now: 18 cases over the Fuchsia trace codec and 24 over the collector, its adapters and the CTF reader, each with a hostile case the public API must reject. observe.opam still declares nox-protobuf, nox-varint, bytesrw and vsock through observe.producer, and nox-json and nox-catapult through observe.capture. Neither is the logging library's business; both are addressable by moving those sublibraries into packages of their own.

2 weeks ago

obs: trigger native sampling at the measured phase

6 days ago

observe: sample allocations on every domain The vendored multicore trace format was already domain-complete -- per-domain writers over one mutex-serialised fd, atomic id chunks, a domain id on every event -- but the tracer's callback gates dropped every non-main-domain sample, so an Executor_pool workload's allocation profile read as empty (a 100 GB run yielded a 2 KB boot-only trace, which cost the vz campaign its ring diagnosis). Non-main domains now write through their own per-domain writer under a plain mutex, which is sound where the vendored Lock is not: they have no synchronous users, and memprof never re-enters a callback on the domain running one. Tracked objects carry their writer so promotion and collection land in the right stream, orphaned events are counted and dropped rather than crossing domains, and stop flushes every writer before the fd closes. The per-domain lookup is an atomic slot array over a mutexed registry: Domain.DLS.get can grow its backing array, which segfaults inside a memprof callback.

6 days ago

obs: make production profiles actionable by default High-rate multidomain captures previously distorted wall and GC budgets, hid application counters, and required an opt-in native sampler. Make the default capture self-diagnosing while preserving explicit opt-outs for overhead measurements.

1 week ago

obs: keep background waits off the headline Use the reconstructed critical path to classify blocking and CPU hotspots. Keep aggregate multidomain waiting in the capacity budget, but do not recommend work on idle listener fibers or count suspended span time as CPU.

5 days ago

obs: profile native CPU with xctrace Replace sample(1)'s blocked-thread snapshots with scheduled Time Profiler samples on macOS. Capture all processes once, retain process identities, and report the target alongside external CPU consumers. Save the native sidecar in every default bundle, load it from obs report, and handshake benchmark triggers only after recording is active.

5 days ago

obs: split the observe library from the obs CLI One opam package shipped both, so a program that wanted Cmdliner logging terms declared a dependency on sqlite, hdr, uuidm, cstruct, eio_main and monitor. Those belong to the profiler. obs now carries the CLI, its collector and its trace adapters; observe carries the library. observe also drops nox-tty, since Tty.Display now installs the log-suspending reporter wrapper itself. Dune ties a public_name to its package, so every observe.* sublibrary stays in observe, and the CLI's four libraries take public names under obs: obs.model, obs.fxt, obs.memtrace, obs.offline. They had to become public because dune forbids a public library depending on a private one, and being public brought their modules under merlint's untested-module rule. So they are tested now: 18 cases over the Fuchsia trace codec and 24 over the collector, its adapters and the CTF reader, each with a hostile case the public API must reject. observe.opam still declares nox-protobuf, nox-varint, bytesrw and vsock through observe.producer, and nox-json and nox-catapult through observe.capture. Neither is the logging library's business; both are addressable by moving those sublibraries into packages of their own.

2 weeks ago

obs: split the observe library from the obs CLI One opam package shipped both, so a program that wanted Cmdliner logging terms declared a dependency on sqlite, hdr, uuidm, cstruct, eio_main and monitor. Those belong to the profiler. obs now carries the CLI, its collector and its trace adapters; observe carries the library. observe also drops nox-tty, since Tty.Display now installs the log-suspending reporter wrapper itself. Dune ties a public_name to its package, so every observe.* sublibrary stays in observe, and the CLI's four libraries take public names under obs: obs.model, obs.fxt, obs.memtrace, obs.offline. They had to become public because dune forbids a public library depending on a private one, and being public brought their modules under merlint's untested-module rule. So they are tested now: 18 cases over the Fuchsia trace codec and 24 over the collector, its adapters and the CTF reader, each with a hostile case the public API must reject. observe.opam still declares nox-protobuf, nox-varint, bytesrw and vsock through observe.producer, and nox-json and nox-catapult through observe.capture. Neither is the logging library's business; both are addressable by moving those sublibraries into packages of their own.

2 weeks ago

obs: split the observe library from the obs CLI One opam package shipped both, so a program that wanted Cmdliner logging terms declared a dependency on sqlite, hdr, uuidm, cstruct, eio_main and monitor. Those belong to the profiler. obs now carries the CLI, its collector and its trace adapters; observe carries the library. observe also drops nox-tty, since Tty.Display now installs the log-suspending reporter wrapper itself. Dune ties a public_name to its package, so every observe.* sublibrary stays in observe, and the CLI's four libraries take public names under obs: obs.model, obs.fxt, obs.memtrace, obs.offline. They had to become public because dune forbids a public library depending on a private one, and being public brought their modules under merlint's untested-module rule. So they are tested now: 18 cases over the Fuchsia trace codec and 24 over the collector, its adapters and the CTF reader, each with a hostile case the public API must reject. observe.opam still declares nox-protobuf, nox-varint, bytesrw and vsock through observe.producer, and nox-json and nox-catapult through observe.capture. Neither is the logging library's business; both are addressable by moving those sublibraries into packages of their own.

2 weeks ago

*: relicense author-original packages MIT -> ISC Standardise on ISC (matching irmin and mirage, and the repo majority): flip (license MIT) to (license ISC) in each dune-project, regenerate the opam files, swap LICENSE.md to the ISC text, and update the SPDX-License-Identifier headers. memtrace and ocaml-xdg keep their license: they are vendored Jane Street code, not ours to relicense.

1 month ago

test_observe.ml

observe: consolidate trace tooling, add obs launcher Profiling used to need --probe/--memtrace flags on every CLI, so all consumers linked the probe and memtrace capture stack just to expose flags they rarely used, and those flags clashed with alcotest's own -q/-v. Move profiling out of the binaries into an external obs launcher that spawns CMD under runtime_events and MEMTRACE, reads the ring out of process, and writes catapult or fxt traces with a hotspots report. Vendor the forked memtrace as observe.memtrace so there is no external fork to maintain and the capture format can evolve. The probe runtime_events capture moves to observe.capture, leaving the default observe library with only logging and env-driven memtrace.

1 month ago

test_observe.mli

*: relicense author-original packages MIT -> ISC Standardise on ISC (matching irmin and mirage, and the repo majority): flip (license MIT) to (license ISC) in each dune-project, regenerate the opam files, swap LICENSE.md to the ISC text, and update the SPDX-License-Identifier headers. memtrace and ocaml-xdg keep their license: they are vendored Jane Street code, not ours to relicense.

1 month ago