gazagnaire.org
vaccel#
The vAccel RPC protocol in pure OCaml.
vAccel forwards acceleration operations -- ML inference, image
classification, or a generic typed-argument genop -- from an application
without accelerator access (a VM or unikernel) to a host agent that owns the
hardware. This library is the wire contract of that conversation:
hand-written nox-protobuf codecs for every
vaccel-rpc-proto message and the
typed method table of the sync AgentService, built on the
ttrpc protocol. It does no I/O of its own.
Installation#
Install with opam:
$ opam install vaccel
If opam cannot find the package, it may not yet be released in the public
opam-repository. Add the overlay repository, then install it:
$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install vaccel
Usage#
vaccel-client calls a running agent. A session is the capability every
operation runs under, and the bracket destroys it however the body leaves --
returning, failing or raising:
let ( let* ) = Result.bind
(* Register a model, then score a batch of feature vectors against it: one
genop call carries the whole batch, and one score vector comes back per
input. *)
let score ~sw ~clock ~net ~token ~model ~embeddings agent =
let client =
Vaccel_client.connect ~sw ~clock ~net
~metadata:(Vaccel_client.bearer token)
agent
in
Vaccel_client.Session.with_ client @@ fun session ->
let* _model =
Vaccel_client.Resource.register session
~blobs:[ Vaccel.Resource.blob ~name:"landcover" model ]
()
in
let* written =
Vaccel_client.genop session (Vaccel.Genop.float32_arrays embeddings)
in
Ok (Vaccel.Genop.to_float32_arrays written)
Underneath, each RPC is a Ttrpc.Method.t pairing the service-qualified name
with its request and response codecs, so an agent, a broker or a test can drive
the same methods directly:
# Ttrpc.Method.service Vaccel.Agent.genop
- : string = "vaccel.sync.agent.AgentService"
# Ttrpc.Method.name Vaccel.Agent.session_create
- : string = "CreateSession"
# Protobuf.to_string
(Ttrpc.Method.request Vaccel.Agent.session_create)
{ Vaccel.Session.Create.flags = 1l }
- : string = "\b\001"
The upstream Rust vaccel-rpc-agent listens on vsock://2:2048 by default;
the pure Ttrpc.Client machine drives the same methods without I/O, which is
how test/test_vaccel.ml runs full calls in memory.
Spec#
The message schemas follow the .proto files of
nubificus/vaccel-rust's
vaccel-rpc-proto crate (session, resource, genop, image, tf, tflite,
torch, profiling, and sync/agent.proto for the service). The test suite
cites them per module and pins byte-exact vectors.
Related work#
- nubificus/vaccel -- the C runtime and plugin framework this protocol fronts.
- nubificus/vaccel-rust -- the Rust RPC agent and client, the interop peers for this library.
- ttrpc -- the transport layer underneath.
License#
ISC. See LICENSE.md.