Commits
A branch ref makes a fresh opam install resolve whatever main points
at that day; every other pin in the tree names a commit. This is
main's current head, the revision the switch already has installed.
macOS 13.3 deprecated the classic CBLAS entry points, so every build of the
sgemm baseline stubs printed a cblas_sgemm deprecation warning. Define
ACCELERATE_NEW_LAPACK before including the framework header; without
ACCELERATE_LAPACK_ILP64 the new interface keeps 32-bit dimensions, so the
existing int locals are unchanged.
One wire pin, one hash: every package pinning
parsimoni-labs/ocaml-wire moves from 77116df9 to f13082a3 together, so
no two packages resolve different generators for the same codecs. The
new revision emits context-guarded, toolchain-portable 3d rules (see
the ocaml-virtio regeneration).
The test asserting that backend () agrees with available () compared the
two-constructor enum structurally. The module already publishes pp_backend
for rendering a backend; give it the matching equality so callers
branching on which path gemm took have one to use.
The new ocaml-wire main generates archive rules that localize each package's raw
<Name>Validate* symbols (macOS ld -r -exported_symbol; Linux ld -r plus objcopy
--keep-global-symbol, allowlisting the wrapper), so a C consumer linking the
installed archive reaches only the checked <Name>Check* wrapper. The raw
validators subtract StartPosition before proving it is within InputLength
(unsigned underflow) and are an external-linkage footgun; the wrapper always
passes zero. Bumps the pin in every .opam and .opam.template and regenerates each
package's c/dune.inc; the generated C is unchanged, the fix is entirely in the
archive rule. nm now shows Validate* local and Check* global, and the agree
differential tests still link.
ocaml-wire d77c63a stops installing the raw <Base>.h validator header, whose
EverParse entrypoints bound a read as N <= InputLength - StartPosition with no
prior StartPosition <= InputLength check, so a direct C caller passing a start
past the buffer underflows the unsigned span and reads out of bounds. The
standalone install now exposes only the checked <Base>Wrapper.h (always entered
at offset 0) and the compiled archive. Bump the pin across every package and
regenerate each c/dune.inc; the OCaml-vs-C agreement tests are unchanged.
wire's released 1.0.0 predates three changes the tree now depends on:
the { at; field; kind } decode-error redesign (#219, #221), uint32 and
uint63 decoding through Optint so a high-bit value survives a
31-bit-int host (#226), and Expr.land64 for masking a full-width
uint64 inside a constraint (#227). Retarget every package pin to the
main commit carrying all three, and record the pin for nox-linear,
which depended on wire without one.
The flat linear-model format (uint32 classes, uint32 features, the
float32 weights row-major, then an optional bias per class) had four
definitions: Metal.Linear's hand-rolled of_bytes/to_bytes, the detect
example's hand-rolled encoder, the prose in tessera's probe_data.mli,
and build_probe.py. Only the first was tested, and a guest could not
link it -- nox-metal carries Objective-C stubs, so a unikernel shipping
a model had to re-implement the bytes it was shipping.
nox-linear owns the format now: pure OCaml over wire and bytesrw, no
stubs, so both halves of an offloaded inference link the same codec.
Metal.Linear keeps the GPU, holding a Linear.t and its resident copy.
Its byte-exact tests move with the format, and the decoder now rejects
bytes past the bias section, which it used to ignore.
An agent that answers one embedding per call pays the GPU's ~180us
command-buffer floor on every one, and a matrix-vector product has no
arithmetic to hide it behind. logits_batch scores N vectors as one
matrix product and one dispatch, returning a logit vector per input; the
choice of GPU or CPU is now on the batch's total work, so a batch can
clear the threshold a lone vector never does.
score unifies the single and batch paths on that rule, and logits is the
n=1 case of it.
Dispatching to the GPU costs about 180us of command buffer, submission
and wait, whatever the arithmetic behind it -- and a matrix-vector
product has none to hide it behind. A land-cover probe is 5 classes over
128-d embeddings: 2560 weights, 8us of CPU. Sending it to the
accelerator took 179us, so the offload the agent exists to provide made
the inference 23 times slower.
Measured, and the crossover is between 5K and 65K weights, so score
anything under 32768 of them here. The bench that says so is in
bench/bench.ml: at 1000x1024 the GPU wins by 58x, which is the model
size an offload is actually for.
Every inference copied the whole weight matrix across the bus to
multiply it by one vector: the gemm stub allocated a fresh MTLBuffer for
both operands per call. On the 2.5KB probe the examples carry that is
lost in the round trip, and on a real model it makes the accelerator
slower than the CPU -- which is exactly the case an offload exists to
serve, since a vAccel agent registers a model once and then answers
inference calls against it.
A classifier now uploads its weights when it is built and keeps them
there; each call sends only its feature vector. The buffer is owned by
the OCaml heap, so the GC releases it with the model that holds it.
weight_uploads counts what actually crossed the bus, so the test pins
the property rather than timing it: ten inferences against one model
upload it once, not ten times.
A trained probe is W . x + b, not W . x: the bias is the intercept, and
without it a class cannot win on prior alone, so a model exported from
any real training run scored wrong here.
The flat model format grows an optional trailing bias section, so a blob
that ends after the weights still loads and scores as the unbiased
classifier it is. A trailing section of any other length is malformed
rather than quietly ignored.
These packages are published only to the opam overlay, not the public
opam-repository. Without the repo-add line a reader who runs opam install
gets nothing and has to hunt down the overlay URL by hand.
METAL_DISABLE=1 forces the CPU path, for testing the fallback and for
isolating non-GPU cost when profiling. The benchmark compares GEMM at
neural-network layer sizes across the naive CPU loop, Apple Accelerate
(cblas_sgemm on the AMX unit) as the SOTA CPU baseline, and Metal MPS
on the GPU -- the GPU overtakes Accelerate above N=1024, losing below
it to per-dispatch overhead.
A thin binding to MetalPerformanceShaders' MPSMatrixMultiplication --
the GEMM at the core of neural-network inference -- over float32
row-major matrices as Bigarray. On a Mac with a Metal device the
product runs on the GPU; on any other platform, or a Mac without one,
the same call computes on the CPU, so code built on it runs unchanged.
The one C stub is Objective-C under ARC on macOS and a raising fallback
elsewhere. Metal.Linear adds the linear classifier the GEMM's first
real use is. The opam name is prefixed because metal.0.1.1 already
exists in opam-repository.
macOS 13.3 deprecated the classic CBLAS entry points, so every build of the
sgemm baseline stubs printed a cblas_sgemm deprecation warning. Define
ACCELERATE_NEW_LAPACK before including the framework header; without
ACCELERATE_LAPACK_ILP64 the new interface keeps 32-bit dimensions, so the
existing int locals are unchanged.
The new ocaml-wire main generates archive rules that localize each package's raw
<Name>Validate* symbols (macOS ld -r -exported_symbol; Linux ld -r plus objcopy
--keep-global-symbol, allowlisting the wrapper), so a C consumer linking the
installed archive reaches only the checked <Name>Check* wrapper. The raw
validators subtract StartPosition before proving it is within InputLength
(unsigned underflow) and are an external-linkage footgun; the wrapper always
passes zero. Bumps the pin in every .opam and .opam.template and regenerates each
package's c/dune.inc; the generated C is unchanged, the fix is entirely in the
archive rule. nm now shows Validate* local and Check* global, and the agree
differential tests still link.
ocaml-wire d77c63a stops installing the raw <Base>.h validator header, whose
EverParse entrypoints bound a read as N <= InputLength - StartPosition with no
prior StartPosition <= InputLength check, so a direct C caller passing a start
past the buffer underflows the unsigned span and reads out of bounds. The
standalone install now exposes only the checked <Base>Wrapper.h (always entered
at offset 0) and the compiled archive. Bump the pin across every package and
regenerate each c/dune.inc; the OCaml-vs-C agreement tests are unchanged.
wire's released 1.0.0 predates three changes the tree now depends on:
the { at; field; kind } decode-error redesign (#219, #221), uint32 and
uint63 decoding through Optint so a high-bit value survives a
31-bit-int host (#226), and Expr.land64 for masking a full-width
uint64 inside a constraint (#227). Retarget every package pin to the
main commit carrying all three, and record the pin for nox-linear,
which depended on wire without one.
The flat linear-model format (uint32 classes, uint32 features, the
float32 weights row-major, then an optional bias per class) had four
definitions: Metal.Linear's hand-rolled of_bytes/to_bytes, the detect
example's hand-rolled encoder, the prose in tessera's probe_data.mli,
and build_probe.py. Only the first was tested, and a guest could not
link it -- nox-metal carries Objective-C stubs, so a unikernel shipping
a model had to re-implement the bytes it was shipping.
nox-linear owns the format now: pure OCaml over wire and bytesrw, no
stubs, so both halves of an offloaded inference link the same codec.
Metal.Linear keeps the GPU, holding a Linear.t and its resident copy.
Its byte-exact tests move with the format, and the decoder now rejects
bytes past the bias section, which it used to ignore.
An agent that answers one embedding per call pays the GPU's ~180us
command-buffer floor on every one, and a matrix-vector product has no
arithmetic to hide it behind. logits_batch scores N vectors as one
matrix product and one dispatch, returning a logit vector per input; the
choice of GPU or CPU is now on the batch's total work, so a batch can
clear the threshold a lone vector never does.
score unifies the single and batch paths on that rule, and logits is the
n=1 case of it.
Dispatching to the GPU costs about 180us of command buffer, submission
and wait, whatever the arithmetic behind it -- and a matrix-vector
product has none to hide it behind. A land-cover probe is 5 classes over
128-d embeddings: 2560 weights, 8us of CPU. Sending it to the
accelerator took 179us, so the offload the agent exists to provide made
the inference 23 times slower.
Measured, and the crossover is between 5K and 65K weights, so score
anything under 32768 of them here. The bench that says so is in
bench/bench.ml: at 1000x1024 the GPU wins by 58x, which is the model
size an offload is actually for.
Every inference copied the whole weight matrix across the bus to
multiply it by one vector: the gemm stub allocated a fresh MTLBuffer for
both operands per call. On the 2.5KB probe the examples carry that is
lost in the round trip, and on a real model it makes the accelerator
slower than the CPU -- which is exactly the case an offload exists to
serve, since a vAccel agent registers a model once and then answers
inference calls against it.
A classifier now uploads its weights when it is built and keeps them
there; each call sends only its feature vector. The buffer is owned by
the OCaml heap, so the GC releases it with the model that holds it.
weight_uploads counts what actually crossed the bus, so the test pins
the property rather than timing it: ten inferences against one model
upload it once, not ten times.
A trained probe is W . x + b, not W . x: the bias is the intercept, and
without it a class cannot win on prior alone, so a model exported from
any real training run scored wrong here.
The flat model format grows an optional trailing bias section, so a blob
that ends after the weights still loads and scores as the unbiased
classifier it is. A trailing section of any other length is malformed
rather than quietly ignored.
METAL_DISABLE=1 forces the CPU path, for testing the fallback and for
isolating non-GPU cost when profiling. The benchmark compares GEMM at
neural-network layer sizes across the naive CPU loop, Apple Accelerate
(cblas_sgemm on the AMX unit) as the SOTA CPU baseline, and Metal MPS
on the GPU -- the GPU overtakes Accelerate above N=1024, losing below
it to per-dispatch overhead.
A thin binding to MetalPerformanceShaders' MPSMatrixMultiplication --
the GEMM at the core of neural-network inference -- over float32
row-major matrices as Bigarray. On a Mac with a Metal device the
product runs on the GPU; on any other platform, or a Mac without one,
the same call computes on the CPU, so code built on it runs unchanged.
The one C stub is Objective-C under ARC on macOS and a raising fallback
elsewhere. Metal.Linear adds the linear classifier the GEMM's first
real use is. The opam name is prefixed because metal.0.1.1 already
exists in opam-repository.