gdiazlo.tngl.sh / arrow
An experimental library which implements a subset of the apache arrow format in ocaml.
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

arrow / README.md
1.7 kB

arrow#

A standalone OCaml library for the Apache Arrow columnar in-memory format and its IPC streaming serialization.

The library targets a v1 type subset of Arrow (Boolean, Int32, Int64, UInt8, UInt16, …) with byte-faithful encode/decode of the Arrow IPC streaming format. It is a pure-OCaml bounded context: a published .mli API over the OCaml stdlib and Bigarray, with no web/IO stack pulled in.

Targeted Arrow format version#

Arrow format version: MetadataVersion.V5 (Arrow ≥ 1.0).

Every IPC metadata message this library produces declares V5, and the codec decodes any known MetadataVersion while rejecting unknown ordinals. The constant lives in exactly one place — the Arrow.Version module (Arrow.Version.target) — and the IPC metadata codec (Arrow.Metadata.target_version) refers to it rather than carrying its own copy. The test suite asserts that the version stamped into emitted metadata equals Arrow.Version.target.

Status#

First release (0.1.0). The public API surface is the single Arrow umbrella module (see lib/arrow.mli), which re-exports the consumer-facing modules: Datatype, Field, Schema, Array, Record_batch, Builder, Dictionary_builder, Ipc (Writer/Reader/Framing), and C_data. The low-level byte machinery (Buffer, Offsets, Validity, the in-tree FlatBuffers helper, and the IPC Metadata codec) is deliberately not re-exported and may change without a breaking release.

Round-trip fidelity is checked against a versioned gold corpus and against independent references (pyarrow and DuckDB) in the test suite.

Building#

dune build
dune runtest

License#

ISC. See LICENSE.