Commits
Bumps the websocket dep to v0.1.5, which guards the handshake over_read
computation against a split terminating CRLF (integer-overflow crashloop
under worker churn). Also re-exports websocket from the root module so
dependents consume zat's single pinned version instead of taking a direct
dependency.
Based off v0.3.4 (not main) to ship the crashloop fix without main's
in-flight websocket-fork migration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
standard.site declares the backing record via a single
<link rel="site.standard.publication">. the extra rel="alternate"
pointing at an at:// uri is not part of the spec, is read by nothing
(cardyb resolves the publication rel directly), and misuses alternate
(which is for fetchable alternate representations, not at:// uris).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bluesky's link-unfurl proxy (cardyb) reads <link rel="site.standard.publication"
href="at://..."> to render a rich publication embed. zat.dev is a registered
site.standard.publication but wasn't emitting the tag, so pasted links got no
preview. add the link-rel (mirroring leaflet's output) plus a basic OG/twitter
card block for clients without standard.site support.
note: SPA serves one index.html for all routes, so this covers the publication
homepage; per-document previews would need SSR/per-route tags.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the change-detection guard piped `git diff-tree` into `grep -Eq`, but grep
isn't among the workflow's nixpkgs deps (curl/git/gnutar/xz). the pipeline
failed with `grep: command not found`; `if ! <failed pipeline>` then
evaluated true, so every non-tag push skipped the ATProto docs publish
(devlog 011 deployed to the Wisp site but never published as a record).
rewrite the guard in pure shell (case over changed paths) so it needs no
extra dependency, and add the workflow's own path to the publish allowlist
so this push re-runs the publish that was skipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zds — an AT Protocol PDS in Zig built on zat — and the one change building
it pushed back into the library (lenient JOSE/JWT verification, v0.3.3).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A JWT is a JOSE token (RFC 7515), where the atproto low-S requirement
does not apply and WebCrypto-based signers (OAuth clients, DPoP) routinely
emit high-S signatures. zat's strict Jwt.verify was the outlier — it
rejected valid service-auth and OAuth tokens that both reference SDKs
accept (@atproto/crypto's allowMalleableSig, atmos's HashAndVerifyLenient).
- add jwt.verifyJose(alg, message, signature, public_key): lenient
ES256/ES256K verifier accepting high-S, for OAuth client assertions / DPoP.
- Jwt.verify now verifies leniently (high-S accepted).
Strict low-S enforcement is unchanged for content-addressed signatures:
verifyCommitCar, verifyCommitDiff, and verifyDidKeySignature still reject
high-S via verifyP256/verifySecp256k1. Additive + strictly-more-permissive
JWT verification — non-breaking patch. 431/431 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
groups the 10 sibling repos that depend on zat by role: firehose
consumers, CBOR/CAR/MST users, and identity+XRPC users. each row
links to the tangled repo and live demo where one exists.
Picks up zzstoatzz/websocket.zig#1: streaming Handshake.parse onto
std.http.HeadParser plus RFC 7230 strictness. Fixes the recurring
class of TCP-fragmentation parser bugs by replacing the one-shot
buffer matcher with a stdlib SIMD streaming state machine.
Two new error variants from the upstream parser:
error.WhitespaceBeforeColon (RFC 7230 §3.2)
error.AmbiguousBodyLength (RFC 7230 §3.3)
Both manifest as 400 responses; downstreams that switch on
Handshake.parse errors should add arms for these if they want
specific handling, otherwise they fall through to the generic 400 path.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sequel to 008 (the io migration). covers the april 5-10 period:
evented coverage degradation, the acute april 8-9 outage, three
wrong hypotheses, the rollback, and the decision to switch back
to Io.Threaded. also covers the socket-after-close race that
ReleaseSafe surfaced on the first Threaded deploy.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
add car.streamBlocks + BlockIterator for zero-alloc CAR iteration over
large repos. strictly additive — read/readWithOptions/Car/findBlock are
unchanged. shares parser helpers (readUvarint, cidLength, verifyBlockHash)
and the CarError set with the existing path, so future CAR fixes benefit
both.
motivated by ken (semantic search over a PDS) which trips the 200k block
guard on repos like pfrazee.com (~72 MB, 248k blocks). the existing
read() path eagerly builds a StringHashMapUnmanaged indexing every CID —
~12 MB of auxiliary metadata on top of whatever the caller is doing.
streamBlocks skips the hashmap and Block[] entirely; iterator is O(1)
space beyond the input buffer, and offset() lets callers build a CID →
byte-offset index that's strictly smaller.
tests:
- round-trip equivalence against read (same block ordering + bytes)
- corruption test confirms BadBlockHash still fires through the iterator
- ZAT_STRESS=1 gated test against cached /tmp/pfrazee.car (72 MB, 248k
blocks) — asserts stream + read see identical block count, total bytes,
and cid xor. not in CI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
propagate the underlying std.http.Client.fetch error from
HttpTransport.fetch and DidResolver.resolve instead of collapsing every
transport-layer failure (DNS, connect, TLS) to error.RequestFailed /
error.DidResolutionFailed.
motivation: zlay 2026-04-08 incident — host_authority resolver pool hit
~99% rejection rate and the only error visible to the validator was
error.DidResolutionFailed because both layers had been swallowed. we had
to ship a workaround (keep_alive=false on the pool) without ever
identifying the actual transport error. this fix unblocks the next
investigation.
regression test: src/internal/identity/did_resolver.zig asserts the
returned error name is not the catch-all sentinel for a connect-refused
case (did:web:127.0.0.1 → 127.0.0.1:443 unlistened).
soft-breaking: the inferred error set on resolve/fetch widens. callers
using `try` or `catch |err| { ... }` are unaffected. callers using an
exhaustive switch on the prior narrow set would need to handle the
additional variants — none in zat itself, and zlay's validator uses the
catch-binding pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Handshake.parse used endsWith("\r\n\r\n") to detect a complete request,
which is only true for header-only GETs. Any POST with a body never
matched, so parse returned null forever and the handshake worker hung
until the connection's idle timeout.
Symptom in zlay: the reconnect cronjob POSTs requestCrawl through the
same socket as the firehose. Every request stalled, no PDSes were
re-announced, and the job hit its activeDeadlineSeconds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merges Jim Calabro's comprehensive correctness improvements:
- non-minimal CBOR encoding rejection (RFC 8949)
- UTF-8 validation on text strings
- map key sort order + duplicate key rejection
- safe integer arithmetic throughout (std.math.add/cast)
- errdefer on all decode allocations
- low-level zero-copy CBOR read/write API
- MST tests ported from atmos, delete rebalancing fix
- CAR v1 validation gaps, readUvarint overflow fix
- ECDSA signature verification overflow fix
- firehose smoke test against live production data
- CBOR codec benchmarks
decode+verify throughput: 290k → 202k fps (still 13x faster than Go).
the regression is from correctness checks that were missing before.
also: bump zig to 0.16.0-dev.3070, update CI, fix tangled.org URLs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- remove sidebar/hamburger, put nav inline in header
- add dark/light theme toggle with localStorage
- improve typography, details, blockquotes, tables
- update roadmap for v0.3.0-alpha
- fix repo link to tangled.org
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
the production SIGSEGV we blamed on fiber context-switch was actually
a one-line off-by-one in the websocket handshake reader. ReleaseSafe
caught it immediately after we switched build modes. updated the devlog
with the full story, the disassembly findings, and the lesson about
assuming two crashes are the same bug.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
update README examples for 0.16 Io parameter, add subscribe(handler)
pattern for streaming clients, add CHANGELOG 0.3.0 entry, devlog 008
covering the full migration saga.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Connects to the real Bluesky firehose, decodes 10k CBOR frames, parses
CAR blocks, verifies CIDs, and decodes records. Exercises the full
refactored pipeline on production data. Run with `just firehose-smoke`.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer allocator.free(entries) in decodeMstNode so the entries
slice is freed if a subsequent readMstEntry call fails. Verified with
checkAllAllocationFailures using a hand-built MST node CBOR fixture.
Other MST functions (put, delete, merge, etc.) mutate persistent tree
state and are designed for arena allocators — they can't be verified
with checkAllAllocationFailures due to cumulative allocation patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer cleanup for roots, blocks, and block_index in
readWithOptions so partial allocations are freed on error. Use a
temporary ArenaAllocator for the header CBOR decode so the header's
Value tree is always freed (it's only needed to extract version + roots).
Note: checkAllAllocationFailures cannot verify CAR read/write because
readWithOptions uses an internal ArenaAllocator whose page-level
allocations are non-deterministic from the tool's perspective. The
errdefer + arena pattern provides equivalent safety.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer allocator.free() for array items and map entries slices in
decodeAt so partial allocations are cleaned up on error. Add 3 tests
using std.testing.checkAllAllocationFailures to exhaustively verify that
every allocation failure in decode (flat map, nested record, array) is
handled without leaking — tests use ArenaAllocator over the failing
allocator to match the intended usage pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes from thorough code review:
- skipValue: arg.val * 2 overflow on crafted map headers (use std.math.mul)
- peekTypeAt: @intCast panic on huge map count (use std.math.cast)
- readUvarint: 10th byte silently truncated (reject byte > 1 at shift 63)
- readUvarint: use u7 shift to avoid saturation arithmetic
- Reject empty CIDs (just 0x00 prefix, no version/codec bytes) in both
the high-level decoder and the low-level readCidLink
- CAR reader: reject non-CID values in roots array (was silently skipping)
- MST loadFromBlocks: remove unnecessary 512-byte buffer copy for root key
Add 4 tests: varint 10th byte overflow/acceptance, empty CID rejection,
too-short CID rejection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After deleting keys, the tree trim loop could reduce root_layer below
what remaining keys require. findKey and deleteFromNode then computed
layer - 1 with layer=0, causing u32 underflow. Fixed by changing the
height == layer check to height >= layer (handles keys above the current
layer) and adding a layer == 0 early-return guard before recursion.
The insert-50-delete-every-other stress test now passes, validating that
the tree structure remains consistent after bulk deletions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 10 new MST tests: get from empty tree, delete nonexistent key,
remove all keys produces empty tree CID, update existing key, order
independence (50 keys in 3 orderings), 100-key stress test, keyHeight
edge cases. One test (insert-then-remove-every-other) is skipped because
it exposes an integer overflow bug in MST tree rebalancing after
deletions — the tree.get() call panics after removing keys. This needs
investigation in the delete/rebalance path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reject high-S signatures before calling Signature.fromBytes, which does
internal scalar arithmetic that overflows on out-of-range S values. The
check was previously done after fromBytes using the parsed sig.s field,
but the construction itself panicked on malformed signatures (e.g.,
high-S or DER-encoded test vectors from the interop fixtures). Now
rejectHighS operates on the raw signature bytes directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run 778 canonical CBOR specification vectors from RFC 8949 Appendix A:
invalid vectors must be rejected, valid canonical vectors must round-trip,
valid non-canonical vectors must be rejected by DAG-CBOR.
The vectors exposed an integer overflow bug: malformed CBOR with huge
length claims (e.g., 0xFFFFFFFFFFFFFFFF byte string) caused arithmetic
overflow in pos + len instead of returning UnexpectedEof. Fixed by using
std.math.add and std.math.cast for all length arithmetic in decodeAt,
readText, readBytes, and skipValue.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Measures writeText, writeUint, writeCidLink, writeRecord (manual 434-byte
record), readText, readUint, readCidLink, skipValue, peekType. Uses
runtime-opaque inputs and keeps output buffers alive to prevent dead
store elimination. Key result: manual record write via low-level API is
5 ns vs 124 ns for the Writer-based encoder (25x).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fuse header+payload into a single writeAll for text strings < 24 bytes
(all AT Protocol map keys), halving writer dispatch count per key. Use a
stack buffer for sorting maps with ≤16 entries (all AT Protocol records),
eliminating allocator calls on the sort path. Encode+CID path improves
~9%, full record encode ~7%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CAR reader now validates version==1, requires non-empty roots, rejects
zero-length blocks, and enforces a 1 MiB per-block size limit (matching
atmos). Fix readUvarint where the overflow check was dead code — shift
was u6 (max 63) so the >= 64 check never triggered; replaced with a
bounded for(0..10) loop. Add 17 CAR tests covering header validation,
truncation, corruption, round-trip determinism, and size limits. Add 5
CAR benchmarks (read/write/round-trip with and without hash verification).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three targeted optimizations based on benchmark profiling:
- Skip map key sort allocation when keys are already in DAG-CBOR order.
Decoded data always has sorted keys, so the decode-re-encode
verification path is now allocation-free for maps (-12%).
- Batch writeArgument into a single writeAll call per argument instead
of 2-3 separate writer dispatches (-8% encode).
- Build CID bytes in a 72-byte stack buffer then dupe, replacing the
dynamically-growing Writer.Allocating for the fixed-size output.
Also adds diagnostic benchmarks (SHA-256 isolation, UTF-8 cost, 10x
scaling, stack CID) to support data-driven optimization decisions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
15 benchmarks covering encode/decode for full records, primitives (text,
uint, CID link, varint), CID computation, and composite operations.
Uses FixedBufferAllocator to measure codec work rather than mmap
syscalls. Run with `just bench` (builds with ReleaseFast).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bumps the websocket dep to v0.1.5, which guards the handshake over_read
computation against a split terminating CRLF (integer-overflow crashloop
under worker churn). Also re-exports websocket from the root module so
dependents consume zat's single pinned version instead of taking a direct
dependency.
Based off v0.3.4 (not main) to ship the crashloop fix without main's
in-flight websocket-fork migration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
standard.site declares the backing record via a single
<link rel="site.standard.publication">. the extra rel="alternate"
pointing at an at:// uri is not part of the spec, is read by nothing
(cardyb resolves the publication rel directly), and misuses alternate
(which is for fetchable alternate representations, not at:// uris).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bluesky's link-unfurl proxy (cardyb) reads <link rel="site.standard.publication"
href="at://..."> to render a rich publication embed. zat.dev is a registered
site.standard.publication but wasn't emitting the tag, so pasted links got no
preview. add the link-rel (mirroring leaflet's output) plus a basic OG/twitter
card block for clients without standard.site support.
note: SPA serves one index.html for all routes, so this covers the publication
homepage; per-document previews would need SSR/per-route tags.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the change-detection guard piped `git diff-tree` into `grep -Eq`, but grep
isn't among the workflow's nixpkgs deps (curl/git/gnutar/xz). the pipeline
failed with `grep: command not found`; `if ! <failed pipeline>` then
evaluated true, so every non-tag push skipped the ATProto docs publish
(devlog 011 deployed to the Wisp site but never published as a record).
rewrite the guard in pure shell (case over changed paths) so it needs no
extra dependency, and add the workflow's own path to the publish allowlist
so this push re-runs the publish that was skipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A JWT is a JOSE token (RFC 7515), where the atproto low-S requirement
does not apply and WebCrypto-based signers (OAuth clients, DPoP) routinely
emit high-S signatures. zat's strict Jwt.verify was the outlier — it
rejected valid service-auth and OAuth tokens that both reference SDKs
accept (@atproto/crypto's allowMalleableSig, atmos's HashAndVerifyLenient).
- add jwt.verifyJose(alg, message, signature, public_key): lenient
ES256/ES256K verifier accepting high-S, for OAuth client assertions / DPoP.
- Jwt.verify now verifies leniently (high-S accepted).
Strict low-S enforcement is unchanged for content-addressed signatures:
verifyCommitCar, verifyCommitDiff, and verifyDidKeySignature still reject
high-S via verifyP256/verifySecp256k1. Additive + strictly-more-permissive
JWT verification — non-breaking patch. 431/431 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up zzstoatzz/websocket.zig#1: streaming Handshake.parse onto
std.http.HeadParser plus RFC 7230 strictness. Fixes the recurring
class of TCP-fragmentation parser bugs by replacing the one-shot
buffer matcher with a stdlib SIMD streaming state machine.
Two new error variants from the upstream parser:
error.WhitespaceBeforeColon (RFC 7230 §3.2)
error.AmbiguousBodyLength (RFC 7230 §3.3)
Both manifest as 400 responses; downstreams that switch on
Handshake.parse errors should add arms for these if they want
specific handling, otherwise they fall through to the generic 400 path.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sequel to 008 (the io migration). covers the april 5-10 period:
evented coverage degradation, the acute april 8-9 outage, three
wrong hypotheses, the rollback, and the decision to switch back
to Io.Threaded. also covers the socket-after-close race that
ReleaseSafe surfaced on the first Threaded deploy.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
add car.streamBlocks + BlockIterator for zero-alloc CAR iteration over
large repos. strictly additive — read/readWithOptions/Car/findBlock are
unchanged. shares parser helpers (readUvarint, cidLength, verifyBlockHash)
and the CarError set with the existing path, so future CAR fixes benefit
both.
motivated by ken (semantic search over a PDS) which trips the 200k block
guard on repos like pfrazee.com (~72 MB, 248k blocks). the existing
read() path eagerly builds a StringHashMapUnmanaged indexing every CID —
~12 MB of auxiliary metadata on top of whatever the caller is doing.
streamBlocks skips the hashmap and Block[] entirely; iterator is O(1)
space beyond the input buffer, and offset() lets callers build a CID →
byte-offset index that's strictly smaller.
tests:
- round-trip equivalence against read (same block ordering + bytes)
- corruption test confirms BadBlockHash still fires through the iterator
- ZAT_STRESS=1 gated test against cached /tmp/pfrazee.car (72 MB, 248k
blocks) — asserts stream + read see identical block count, total bytes,
and cid xor. not in CI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
propagate the underlying std.http.Client.fetch error from
HttpTransport.fetch and DidResolver.resolve instead of collapsing every
transport-layer failure (DNS, connect, TLS) to error.RequestFailed /
error.DidResolutionFailed.
motivation: zlay 2026-04-08 incident — host_authority resolver pool hit
~99% rejection rate and the only error visible to the validator was
error.DidResolutionFailed because both layers had been swallowed. we had
to ship a workaround (keep_alive=false on the pool) without ever
identifying the actual transport error. this fix unblocks the next
investigation.
regression test: src/internal/identity/did_resolver.zig asserts the
returned error name is not the catch-all sentinel for a connect-refused
case (did:web:127.0.0.1 → 127.0.0.1:443 unlistened).
soft-breaking: the inferred error set on resolve/fetch widens. callers
using `try` or `catch |err| { ... }` are unaffected. callers using an
exhaustive switch on the prior narrow set would need to handle the
additional variants — none in zat itself, and zlay's validator uses the
catch-binding pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Handshake.parse used endsWith("\r\n\r\n") to detect a complete request,
which is only true for header-only GETs. Any POST with a body never
matched, so parse returned null forever and the handshake worker hung
until the connection's idle timeout.
Symptom in zlay: the reconnect cronjob POSTs requestCrawl through the
same socket as the firehose. Every request stalled, no PDSes were
re-announced, and the job hit its activeDeadlineSeconds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merges Jim Calabro's comprehensive correctness improvements:
- non-minimal CBOR encoding rejection (RFC 8949)
- UTF-8 validation on text strings
- map key sort order + duplicate key rejection
- safe integer arithmetic throughout (std.math.add/cast)
- errdefer on all decode allocations
- low-level zero-copy CBOR read/write API
- MST tests ported from atmos, delete rebalancing fix
- CAR v1 validation gaps, readUvarint overflow fix
- ECDSA signature verification overflow fix
- firehose smoke test against live production data
- CBOR codec benchmarks
decode+verify throughput: 290k → 202k fps (still 13x faster than Go).
the regression is from correctness checks that were missing before.
also: bump zig to 0.16.0-dev.3070, update CI, fix tangled.org URLs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
the production SIGSEGV we blamed on fiber context-switch was actually
a one-line off-by-one in the websocket handshake reader. ReleaseSafe
caught it immediately after we switched build modes. updated the devlog
with the full story, the disassembly findings, and the lesson about
assuming two crashes are the same bug.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer allocator.free(entries) in decodeMstNode so the entries
slice is freed if a subsequent readMstEntry call fails. Verified with
checkAllAllocationFailures using a hand-built MST node CBOR fixture.
Other MST functions (put, delete, merge, etc.) mutate persistent tree
state and are designed for arena allocators — they can't be verified
with checkAllAllocationFailures due to cumulative allocation patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer cleanup for roots, blocks, and block_index in
readWithOptions so partial allocations are freed on error. Use a
temporary ArenaAllocator for the header CBOR decode so the header's
Value tree is always freed (it's only needed to extract version + roots).
Note: checkAllAllocationFailures cannot verify CAR read/write because
readWithOptions uses an internal ArenaAllocator whose page-level
allocations are non-deterministic from the tool's perspective. The
errdefer + arena pattern provides equivalent safety.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add errdefer allocator.free() for array items and map entries slices in
decodeAt so partial allocations are cleaned up on error. Add 3 tests
using std.testing.checkAllAllocationFailures to exhaustively verify that
every allocation failure in decode (flat map, nested record, array) is
handled without leaking — tests use ArenaAllocator over the failing
allocator to match the intended usage pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes from thorough code review:
- skipValue: arg.val * 2 overflow on crafted map headers (use std.math.mul)
- peekTypeAt: @intCast panic on huge map count (use std.math.cast)
- readUvarint: 10th byte silently truncated (reject byte > 1 at shift 63)
- readUvarint: use u7 shift to avoid saturation arithmetic
- Reject empty CIDs (just 0x00 prefix, no version/codec bytes) in both
the high-level decoder and the low-level readCidLink
- CAR reader: reject non-CID values in roots array (was silently skipping)
- MST loadFromBlocks: remove unnecessary 512-byte buffer copy for root key
Add 4 tests: varint 10th byte overflow/acceptance, empty CID rejection,
too-short CID rejection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After deleting keys, the tree trim loop could reduce root_layer below
what remaining keys require. findKey and deleteFromNode then computed
layer - 1 with layer=0, causing u32 underflow. Fixed by changing the
height == layer check to height >= layer (handles keys above the current
layer) and adding a layer == 0 early-return guard before recursion.
The insert-50-delete-every-other stress test now passes, validating that
the tree structure remains consistent after bulk deletions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 10 new MST tests: get from empty tree, delete nonexistent key,
remove all keys produces empty tree CID, update existing key, order
independence (50 keys in 3 orderings), 100-key stress test, keyHeight
edge cases. One test (insert-then-remove-every-other) is skipped because
it exposes an integer overflow bug in MST tree rebalancing after
deletions — the tree.get() call panics after removing keys. This needs
investigation in the delete/rebalance path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reject high-S signatures before calling Signature.fromBytes, which does
internal scalar arithmetic that overflows on out-of-range S values. The
check was previously done after fromBytes using the parsed sig.s field,
but the construction itself panicked on malformed signatures (e.g.,
high-S or DER-encoded test vectors from the interop fixtures). Now
rejectHighS operates on the raw signature bytes directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run 778 canonical CBOR specification vectors from RFC 8949 Appendix A:
invalid vectors must be rejected, valid canonical vectors must round-trip,
valid non-canonical vectors must be rejected by DAG-CBOR.
The vectors exposed an integer overflow bug: malformed CBOR with huge
length claims (e.g., 0xFFFFFFFFFFFFFFFF byte string) caused arithmetic
overflow in pos + len instead of returning UnexpectedEof. Fixed by using
std.math.add and std.math.cast for all length arithmetic in decodeAt,
readText, readBytes, and skipValue.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Measures writeText, writeUint, writeCidLink, writeRecord (manual 434-byte
record), readText, readUint, readCidLink, skipValue, peekType. Uses
runtime-opaque inputs and keeps output buffers alive to prevent dead
store elimination. Key result: manual record write via low-level API is
5 ns vs 124 ns for the Writer-based encoder (25x).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fuse header+payload into a single writeAll for text strings < 24 bytes
(all AT Protocol map keys), halving writer dispatch count per key. Use a
stack buffer for sorting maps with ≤16 entries (all AT Protocol records),
eliminating allocator calls on the sort path. Encode+CID path improves
~9%, full record encode ~7%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CAR reader now validates version==1, requires non-empty roots, rejects
zero-length blocks, and enforces a 1 MiB per-block size limit (matching
atmos). Fix readUvarint where the overflow check was dead code — shift
was u6 (max 63) so the >= 64 check never triggered; replaced with a
bounded for(0..10) loop. Add 17 CAR tests covering header validation,
truncation, corruption, round-trip determinism, and size limits. Add 5
CAR benchmarks (read/write/round-trip with and without hash verification).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three targeted optimizations based on benchmark profiling:
- Skip map key sort allocation when keys are already in DAG-CBOR order.
Decoded data always has sorted keys, so the decode-re-encode
verification path is now allocation-free for maps (-12%).
- Batch writeArgument into a single writeAll call per argument instead
of 2-3 separate writer dispatches (-8% encode).
- Build CID bytes in a 72-byte stack buffer then dupe, replacing the
dynamically-growing Writer.Allocating for the fixed-size output.
Also adds diagnostic benchmarks (SHA-256 isolation, UTF-8 cost, 10x
scaling, stack CID) to support data-driven optimization decisions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
15 benchmarks covering encode/decode for full records, primitives (text,
uint, CID link, varint), CID computation, and composite operations.
Uses FixedBufferAllocator to measure codec work rather than mmap
syscalls. Run with `just bench` (builds with ReleaseFast).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>