zzstoatzz.io
Commits
std.posix.setsockopt maps BADF/NOTSOCK/INVAL/FAULT to `unreachable`
("always a race condition"). That holds for a listening socket. It does not
hold for a connection socket: the peer can reset, or another thread can close
the fd, between connect and the timeout call. `unreachable` cannot be caught,
so the race aborts the process instead of returning an error the caller could
reconnect on.
Found downstream in Stream (zat.dev/stream), whose firehose consumer aborted
roughly one full-suite run in seven with `reached unreachable code` in
setsockopt, reached from Stream.setsockopt via readTimeout inside
HandShakeReply.read during zat's connectAndRead. An upstream relay dropping
the connection around handshake killed the process -- previously misread as a
transient transport error.
Route the client's connection-socket setsockopt through setConnSockOpt, which
issues the raw syscall and swallows exactly the four arms the stdlib declares
impossible while keeping every other errno meaningful. The timeouts are
advisory: if the socket really is gone, the following read or write reports it
properly.
This is the client-side counterpart of 00bed03, which did the same for the
four server connection-socket call sites. Stream.setsockopt keeps its
`!void` signature so no consumer's `try` breaks.
Test closes the fd under a live Stream and asserts the timeout calls return
instead of aborting; before this change it panics on `.BADF => unreachable`.
64 of 64 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
runIo reuses Blocking(H)'s read loop, which passed fba=undefined on the
assumption that comptime blockingMode() compiles the fba branch out.
that holds for true blocking mode, but runIo runs with blockingMode()
false on linux/macos, so an allocator-taking clientMessage handler
walked into the FallbackAllocator and dereferenced undefined memory.
make the per-thread fba optional: the worker path passes it as before,
blocking-style read loops pass null and the message allocator falls
back to the arena. regression test drives runIo with TestHandler's
allocator variant (panics without the fix).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The readme was upstream's verbatim (still describing Zig 0.15.1 and with no
fork notice). Replace the top with a short readme that attributes the library
to Karl Seguin / karlseguin/websocket.zig, points at the retained MIT LICENSE,
states this fork targets 0.16, and lists what diverged. The complete original
upstream readme is preserved unmodified inside a collapsed <details> block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The package's build.zig.zon .paths whitelist excludes zig-pkg, so the
vendored copy was never part of the package hash or the consumed module
— it was dead weight committed to the repo (present since v0.1.7). Remove
it from tracking and add zig-pkg/ to .gitignore to match zat/zlay/zds.
Does not affect the v0.1.8 release hash.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tags v0.1.3..v0.1.6 all shipped .version = "0.1.2", so two tags of this
package in one build graph collide (same fingerprint+version, different
hashes — zig can't unify them). from here the zon version matches the tag.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
the compat Address held a bare 16-byte sockaddr, so it could only ever
represent IPv4 (and initUnix was quietly writing a 110-byte sockaddr.un
through it). back it with sockaddr.storage, accept "::" in parseIp
(AF_INET6 any — needed for IPv6-only networks like fly 6PN, dual-stack
on linux where V6ONLY defaults off), size accept/getpeername out-params
to storage, and report in6 in getOsSockLen/format.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When TCP delivers the final \r of the blank-line CRLF as the last byte of
a read and the \n arrives later, pos == line_start + 1 and the end-of-headers
branch computed over_read = pos - (line_start + 2), underflowing usize and
panicking with integer overflow under ReleaseSafe. Break for more data when
the \n hasn't arrived yet, mirroring the existing sibling-case guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
std.posix.setsockopt maps EBADF/ENOTSOCK to `unreachable` ("always a
race condition" per the stdlib). When a peer resets or another thread
closes a connection socket mid-flight, the RCVTIMEO/SNDTIMEO/TCP_NODELAY
calls panic the worker thread (SIGSEGV) instead of failing — and the
existing `catch {}` / `catch return` can't swallow an `unreachable`.
Route all four connection-socket setsockopt calls (handshake RCVTIMEO,
readLoop RCVTIMEO, accept-time TCP_NODELAY, error-write SNDTIMEO) through
a best-effort helper that issues the raw posix.system.setsockopt syscall
and ignores the result. Listen-socket setup calls stay fatal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The blocking-mode cleanupConn (line 663) calls `hc.conn.closeSocket()`
before destroying the HandlerConn. The nonblocking-mode counterpart was
missing that call, leaking one file descriptor per accepted connection.
Symptom in downstream consumers: under sustained WS or HTTP-fallback
traffic, `lsof` shows accumulating sockets in CLOSE_WAIT then CLOSED
state held by the server process until ulimit exhaustion wedges the
worker.
Caught by prefect-server's `scripts/test-background-tasks` smoke,
which subscribes to a /task_runs/subscriptions/scheduled WS,
disconnects, and re-runs. Each subscribe-disconnect cycle was leaking
one FD; consecutive runs piled up into a wedge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously, `Handshake.parse` returned `error.InvalidConnection` /
`error.InvalidUpgrade` *inline* the moment it saw a header that
disqualified the request as a WebSocket upgrade (e.g. an httpx-style
`Connection: keep-alive`). The body-completeness gate at the end of the
function was inside `if (required_headers != 15)`, so it never fired on
these paths — the worker dispatched `httpFallback` with whatever bytes
happened to be in the buffer.
Effect: a POST/PATCH with `Connection: keep-alive` (httpx default) whose
headers and body were delivered in separate TCP segments would surface
to `httpFallback` with `body.len == 0` even when `Content-Length` was
non-zero. Intermittent under load (~10-20% of requests under fast
back-to-back smoke tests). Manifested in prefect-server's zig port as
"failed to read body" 400s on `/api/flow_runs/{id}/labels`,
`/api/work_pools/{pool}/workers/heartbeat`, and similar endpoints.
Fix: defer the `InvalidUpgrade` / `InvalidConnection` errors and keep
parsing so `Content-Length` lands in the headers KeyValue. Run the
body-completeness check on every fallback path (MissingHeaders +
InvalidConnection + InvalidUpgrade); only return the deferred error
after the body is fully buffered. If `Content-Length` is absent or
unparseable, return the deferred error immediately (preserving prior
semantics for those cases).
Adds three regression tests covering: (a) keep-alive + Content-Length
where headers arrived alone (must return null, request more data),
(b) keep-alive + Content-Length where the full body is buffered (must
surface InvalidConnection so worker dispatches), and (c) non-websocket
Upgrade token with partial body (must return null).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
streaming Handshake.parse onto std.http.HeadParser + RFC 7230 strictness.
see PR #1 for the full rationale and 8 new test groups.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
handshake: streaming parse onto std.http.HeadParser + zen/RFC strictness
Two changes pass on top of the streaming rewrite (53e0b26), each
independently reviewable.
Zen: invariant-protected paths now use `unreachable`
----------------------------------------------------
The streaming rewrite introduced four paths that are unreachable if
HeadParser's contract holds:
1. body_start < 4 (HeadParser .finished implies ≥4)
2. first CRLF not in buf (CRLFCRLF tail guarantees a CRLF)
3. no CRLF inside header line (rest is bounded by HeadParser)
4. empty line mid-iteration (HeadParser stops at first empty line)
Initially I returned named errors on those paths, defensively. Per
"runtime crashes are better than bugs": if any of those fires we have
a parser invariant violation, and a clean panic at the violation site
is more debuggable than a silent misclassification of valid input as
malformed. Switched to `unreachable`.
The doc comment now declares this contract explicitly: every named
error corresponds to genuinely malformed external input; every
unreachable corresponds to an internal-invariant assertion.
RFC 7230 strictness: two new error variants
-------------------------------------------
error.WhitespaceBeforeColon — RFC 7230 §3.2: "No whitespace is
allowed between the header field-name and colon. Servers MUST
reject ... with a 400." This is a request-smuggling defense.
Previously we silently trimmed (tolerant); now we reject.
error.AmbiguousBodyLength — RFC 7230 §3.3: "If a message is
received with both a Transfer-Encoding and a Content-Length ...
such a message might indicate an attempt to perform request
smuggling." Previously TE was ignored entirely; now the
combination is rejected explicitly.
Also rejects obsolete line folding (RFC 7230 §3.2.4 deprecated):
a continuation line beginning with whitespace surfaces as
InvalidHeader rather than getting silently merged into the prior
value. Documented as a deliberate non-feature in the doc comment.
Tests
-----
Three new test groups, every case covers the rejection path:
- whitespace-before-colon (3 cases: SP, HTAB, mixed-required-headers)
- Transfer-Encoding + Content-Length (2 cases: POST + would-be WS)
- obsolete line folding (1 case)
server.zig: respondToHandshakeError grew two switch arms for the
new error variants so 400 responses carry a specific Error: header.
50/50 tests pass. zig fmt clean.
motivation: this fork has had five distinct parser/handshake bugs in
~5 weeks, all in the same shape — TCP fragmentation breaks the
one-shot buffer matcher in a way that wasn't covered by tests:
- 2bf2099 (mar 1): TCP split fix
- 0f11cfc (apr 2): client write lock missing
- 4222f98 (apr 2): httpFallback dispatch path
- 9ac64da (apr 5): handshake panic on TCP split mid-CRLF
- 3c6794a (apr 7): POST-with-body causes parse → null forever
each was fixed reactively after a downstream production incident.
the underlying issue is structural: parse() was a one-shot matcher
that re-scanned the whole buffer with indexOf on every call, and
the existing test suite only covered all-at-once happy paths. any
edge case in TCP fragmentation could miss the test net.
what changed
------------
State now carries a streaming end-of-headers detector
(std.http.HeadParser, stdlib's SIMD-optimized state machine) plus a
head_fed counter. parse() feeds only NEW bytes per call so the SIMD
scan is amortized across reads. boundary detection is no longer
sensitive to where TCP packet splits land.
the request/header parser proper is now line-based on the bounded
headers section, with explicit named errors at every malformed-input
exit. there are no `unreachable` branches reachable from any input.
raw_header byte slice is preserved verbatim for callers that re-emit
the original headers (existing test asserts this; passes unchanged).
test coverage
-------------
all 42 prior tests pass unchanged. five new test groups added,
~150 lines:
1. byte-split equivalence for valid upgrade — feed every chunk size
from 1 to N bytes, assert same final outcome
2. CRLF/CRLFCRLF split torture — every two-chunk split position
across a complete request, assert same result
3. malformed inputs return errors not panics — 8 adversarial cases
covering each named error path
4. POST body byte-split equivalence — every split position must
hold null until body fully arrives, then MissingHeaders
5. idempotence under repeated calls — parse twice with same len
produces same answer
47/47 tests pass. zig fmt clean.
what's not changed
------------------
external API: `Handshake`, `parse`, `State` field semantics that
callers depend on (`buf`, `len`, `req_headers`) all preserved.
this is a drop-in replacement for the parse internals; no
downstream code change required.
Handshake.parse used endsWith("\r\n\r\n") to decide when the buffer held
a complete request. That check only works for header-only requests
(GET). Any POST with a body ends on body bytes, so parse returned null
indefinitely — the worker kept reading, the client had already sent
everything, and the connection leaked until its idle timeout.
Concrete symptom: zlay's reconnect cronjob POSTs requestCrawl through
the same socket the WebSocket firehose is served on. Each request
stalled for the full recv timeout, so the job never re-announced any
PDSes and eventually hit its activeDeadlineSeconds.
Fix:
- locate header terminator with indexOf("\r\n\r\n") (not endsWith)
- parse only the header slice, leaving the body in state.buf for
parseHttpRequest to extract after MissingHeaders is raised
- reset req_headers.len at the top of parse so re-entry with more
data does not accumulate duplicate headers
- before raising MissingHeaders, consult Content-Length: if the body
hasn't fully arrived yet, return null so the worker reads more
before dispatching the fallback (avoids truncated body handoff)
Regression coverage: three new cases in "handshake: parse POST with
body signals MissingHeaders (for httpFallback)":
- full POST buffer → MissingHeaders (triggers httpFallback)
- partial body → null (read more)
- POST with no Content-Length → MissingHeaders (immediate)
when a TCP read delivers a \r at the buffer boundary without its \n,
line_start advances past pos. the next indexOfScalar call then panics
on buf[line_start..pos] because start > end.
under ReleaseFast this was a silent SIGSEGV (~every 30-90 min under
sustained load with ~2,800 connections). ReleaseSafe surfaces it as a
bounds check panic at client.zig:766.
fix: break out of the inner parse loop when line_start exceeds pos,
falling through to the outer read loop to get more data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously initWithStream hardcoded tls_client to null, silently
ignoring config.tls. Now it initializes TLS (using config.host for
SNI) when requested, matching the behavior of init().
This enables callers that bring their own pre-connected stream
(e.g. after resolving DNS through a separate Io backend) to still
get TLS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
type-erase ctx parameter via local wrapper struct so
Group.concurrent gets concrete ArgsTuple. fix .receive -> .recv
to match 0.16 Io.net.ShutdownHow enum.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
instead of discarding futures from io.concurrent per connection,
collect them in an Io.Group and cancel on shutdown. prevents
resource leaks from unowned futures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
thread io: Io through Server.init → WorkerState → ConnManager → Conn,
replacing hardcoded std.Options.debug_io. add Server.runIo() which uses
Io.net.Server.accept + io.concurrent per connection — under Evented
this becomes fiber-based (no OS thread per connection), under Threaded
it behaves like the old blocking path.
existing listen()/listenInNewThread() preserved for backwards compat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
read() now returns null on WouldBlock (SO_RCVTIMEO fired), letting
callers decide what to do. readLoopWithHeartbeat() uses this to send
pings on idle intervals and close after max_failures consecutive
timeouts — eliminating the need for a separate ping task/thread.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
exposes the _closed atomic flag so callers (e.g. ping loops) can
check whether the connection is dead before attempting writes.
prevents use-after-free when a concurrent task outlives the
connection teardown.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
when Handshake.parse fails with MissingHeaders, InvalidConnection,
or InvalidUpgrade — indicating a valid HTTP request that isn't a
websocket upgrade — check if the handler defines httpFallback and
dispatch to it instead of returning 400.
this restores the ability to serve health probes, XRPC endpoints,
and admin HTTP on the same port as the websocket server. the
fallback is gated by comptime hasFn: handlers without httpFallback
get no new codepath compiled.
the HTTP parser (parseHttpRequest) re-parses method/url/headers/body
from the raw buffer rather than reusing partial state from
Handshake.parse, which may have exited early with incomplete headers
and partially-lowercased names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
client.writeFrame() is called concurrently from:
- pingLoop task (subscriber keepalive)
- readLoop auto-pong (server ping response)
- close path (connection teardown)
without serialization, concurrent writes interleave frame headers/payloads,
corrupting the stream and causing GPF in memcpy → Writer.zig → writeAll.
adds _write_lock: Io.Mutex to Client struct, acquired around the two
writeAll calls in writeFrame(). header construction and payload masking
happen outside the lock to minimize hold time.
the server-side Conn already has this pattern (lock: Io.Mutex around
writeFrame/writeFramed/writeAllIOVec). this brings the client to parity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0x0004 is O_NONBLOCK on macOS but not Linux (0o4000). Sockets/pipes
weren't set non-blocking on Linux, causing epoll server to hang.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
linux.epoll_ctl and epoll_wait return usize (raw syscall).
check errno for errors instead of directly returning the value.
also fix epoll_wait missing maxevents argument.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- TLS .ca.bundle now requires gpa, io, lock, bundle pointer
- .realtime_now takes Io.Timestamp, not seconds
- add getPort() to Address shim
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Bundle{} missing required fields on Linux — use .{ .map = .empty, .bytes = .empty }
- epoll_create1 returns usize, cast to i32 fd via std.math.cast
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Linux sun_path is 108 bytes, macOS is 104. Use @typeInfo to get
the correct size at comptime.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All networking now goes through std.Io vtable (netRead/netWrite/netClose)
instead of raw libc recv/send/close. Mutex/Condition use Io variants.
PriorityDequeue, DebugAllocator, and Io.Clock APIs updated for 0.16.
All 31 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- use libc directly for socket operations (debug_io doesn't support real network)
- add Client.initWithStream() for testing with pre-existing sockets
- fix TestStream/testStream in server.zig to use libc
- fix SocketPair in t.zig to use libc socket/connect/send/recv
- darwin workaround: use fcntl for CLOEXEC instead of SOCK.CLOEXEC
concessions made:
- bypasses new Io abstraction for socket ops (libc direct calls)
- darwin-specific sockaddr workarounds
- test-only initWithStream API addition
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Still TODO:
- Io.net.Stream API completely changed - needs Reader/Writer wrappers
- Client.init now requires io: Io parameter
- Server networking needs similar Io threading
- Test utilities need io parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
linkLibC() method removed from Build.Step.Compile in 0.16.
use .link_libc = true in module options instead.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
fix: replace ambiguous '{}' with '{f}' in log.info for std.net.Address
server: Wait for threads to exit when stopping
This can happen if the client is disconneted just before the server is shutdown.
https://github.com/karlseguin/websocket.zig/issues/80
Fix client regression: Calling client.close() on a TLS connection while readLoop is running results in a crash
std.posix.setsockopt maps BADF/NOTSOCK/INVAL/FAULT to `unreachable`
("always a race condition"). That holds for a listening socket. It does not
hold for a connection socket: the peer can reset, or another thread can close
the fd, between connect and the timeout call. `unreachable` cannot be caught,
so the race aborts the process instead of returning an error the caller could
reconnect on.
Found downstream in Stream (zat.dev/stream), whose firehose consumer aborted
roughly one full-suite run in seven with `reached unreachable code` in
setsockopt, reached from Stream.setsockopt via readTimeout inside
HandShakeReply.read during zat's connectAndRead. An upstream relay dropping
the connection around handshake killed the process -- previously misread as a
transient transport error.
Route the client's connection-socket setsockopt through setConnSockOpt, which
issues the raw syscall and swallows exactly the four arms the stdlib declares
impossible while keeping every other errno meaningful. The timeouts are
advisory: if the socket really is gone, the following read or write reports it
properly.
This is the client-side counterpart of 00bed03, which did the same for the
four server connection-socket call sites. Stream.setsockopt keeps its
`!void` signature so no consumer's `try` breaks.
Test closes the fd under a live Stream and asserts the timeout calls return
instead of aborting; before this change it panics on `.BADF => unreachable`.
64 of 64 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
runIo reuses Blocking(H)'s read loop, which passed fba=undefined on the
assumption that comptime blockingMode() compiles the fba branch out.
that holds for true blocking mode, but runIo runs with blockingMode()
false on linux/macos, so an allocator-taking clientMessage handler
walked into the FallbackAllocator and dereferenced undefined memory.
make the per-thread fba optional: the worker path passes it as before,
blocking-style read loops pass null and the message allocator falls
back to the arena. regression test drives runIo with TestHandler's
allocator variant (panics without the fix).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The readme was upstream's verbatim (still describing Zig 0.15.1 and with no
fork notice). Replace the top with a short readme that attributes the library
to Karl Seguin / karlseguin/websocket.zig, points at the retained MIT LICENSE,
states this fork targets 0.16, and lists what diverged. The complete original
upstream readme is preserved unmodified inside a collapsed <details> block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The package's build.zig.zon .paths whitelist excludes zig-pkg, so the
vendored copy was never part of the package hash or the consumed module
— it was dead weight committed to the repo (present since v0.1.7). Remove
it from tracking and add zig-pkg/ to .gitignore to match zat/zlay/zds.
Does not affect the v0.1.8 release hash.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
the compat Address held a bare 16-byte sockaddr, so it could only ever
represent IPv4 (and initUnix was quietly writing a 110-byte sockaddr.un
through it). back it with sockaddr.storage, accept "::" in parseIp
(AF_INET6 any — needed for IPv6-only networks like fly 6PN, dual-stack
on linux where V6ONLY defaults off), size accept/getpeername out-params
to storage, and report in6 in getOsSockLen/format.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When TCP delivers the final \r of the blank-line CRLF as the last byte of
a read and the \n arrives later, pos == line_start + 1 and the end-of-headers
branch computed over_read = pos - (line_start + 2), underflowing usize and
panicking with integer overflow under ReleaseSafe. Break for more data when
the \n hasn't arrived yet, mirroring the existing sibling-case guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
std.posix.setsockopt maps EBADF/ENOTSOCK to `unreachable` ("always a
race condition" per the stdlib). When a peer resets or another thread
closes a connection socket mid-flight, the RCVTIMEO/SNDTIMEO/TCP_NODELAY
calls panic the worker thread (SIGSEGV) instead of failing — and the
existing `catch {}` / `catch return` can't swallow an `unreachable`.
Route all four connection-socket setsockopt calls (handshake RCVTIMEO,
readLoop RCVTIMEO, accept-time TCP_NODELAY, error-write SNDTIMEO) through
a best-effort helper that issues the raw posix.system.setsockopt syscall
and ignores the result. Listen-socket setup calls stay fatal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The blocking-mode cleanupConn (line 663) calls `hc.conn.closeSocket()`
before destroying the HandlerConn. The nonblocking-mode counterpart was
missing that call, leaking one file descriptor per accepted connection.
Symptom in downstream consumers: under sustained WS or HTTP-fallback
traffic, `lsof` shows accumulating sockets in CLOSE_WAIT then CLOSED
state held by the server process until ulimit exhaustion wedges the
worker.
Caught by prefect-server's `scripts/test-background-tasks` smoke,
which subscribes to a /task_runs/subscriptions/scheduled WS,
disconnects, and re-runs. Each subscribe-disconnect cycle was leaking
one FD; consecutive runs piled up into a wedge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously, `Handshake.parse` returned `error.InvalidConnection` /
`error.InvalidUpgrade` *inline* the moment it saw a header that
disqualified the request as a WebSocket upgrade (e.g. an httpx-style
`Connection: keep-alive`). The body-completeness gate at the end of the
function was inside `if (required_headers != 15)`, so it never fired on
these paths — the worker dispatched `httpFallback` with whatever bytes
happened to be in the buffer.
Effect: a POST/PATCH with `Connection: keep-alive` (httpx default) whose
headers and body were delivered in separate TCP segments would surface
to `httpFallback` with `body.len == 0` even when `Content-Length` was
non-zero. Intermittent under load (~10-20% of requests under fast
back-to-back smoke tests). Manifested in prefect-server's zig port as
"failed to read body" 400s on `/api/flow_runs/{id}/labels`,
`/api/work_pools/{pool}/workers/heartbeat`, and similar endpoints.
Fix: defer the `InvalidUpgrade` / `InvalidConnection` errors and keep
parsing so `Content-Length` lands in the headers KeyValue. Run the
body-completeness check on every fallback path (MissingHeaders +
InvalidConnection + InvalidUpgrade); only return the deferred error
after the body is fully buffered. If `Content-Length` is absent or
unparseable, return the deferred error immediately (preserving prior
semantics for those cases).
Adds three regression tests covering: (a) keep-alive + Content-Length
where headers arrived alone (must return null, request more data),
(b) keep-alive + Content-Length where the full body is buffered (must
surface InvalidConnection so worker dispatches), and (c) non-websocket
Upgrade token with partial body (must return null).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
streaming Handshake.parse onto std.http.HeadParser + RFC 7230 strictness.
see PR #1 for the full rationale and 8 new test groups.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Two changes pass on top of the streaming rewrite (53e0b26), each
independently reviewable.
Zen: invariant-protected paths now use `unreachable`
----------------------------------------------------
The streaming rewrite introduced four paths that are unreachable if
HeadParser's contract holds:
1. body_start < 4 (HeadParser .finished implies ≥4)
2. first CRLF not in buf (CRLFCRLF tail guarantees a CRLF)
3. no CRLF inside header line (rest is bounded by HeadParser)
4. empty line mid-iteration (HeadParser stops at first empty line)
Initially I returned named errors on those paths, defensively. Per
"runtime crashes are better than bugs": if any of those fires we have
a parser invariant violation, and a clean panic at the violation site
is more debuggable than a silent misclassification of valid input as
malformed. Switched to `unreachable`.
The doc comment now declares this contract explicitly: every named
error corresponds to genuinely malformed external input; every
unreachable corresponds to an internal-invariant assertion.
RFC 7230 strictness: two new error variants
-------------------------------------------
error.WhitespaceBeforeColon — RFC 7230 §3.2: "No whitespace is
allowed between the header field-name and colon. Servers MUST
reject ... with a 400." This is a request-smuggling defense.
Previously we silently trimmed (tolerant); now we reject.
error.AmbiguousBodyLength — RFC 7230 §3.3: "If a message is
received with both a Transfer-Encoding and a Content-Length ...
such a message might indicate an attempt to perform request
smuggling." Previously TE was ignored entirely; now the
combination is rejected explicitly.
Also rejects obsolete line folding (RFC 7230 §3.2.4 deprecated):
a continuation line beginning with whitespace surfaces as
InvalidHeader rather than getting silently merged into the prior
value. Documented as a deliberate non-feature in the doc comment.
Tests
-----
Three new test groups, every case covers the rejection path:
- whitespace-before-colon (3 cases: SP, HTAB, mixed-required-headers)
- Transfer-Encoding + Content-Length (2 cases: POST + would-be WS)
- obsolete line folding (1 case)
server.zig: respondToHandshakeError grew two switch arms for the
new error variants so 400 responses carry a specific Error: header.
50/50 tests pass. zig fmt clean.
motivation: this fork has had five distinct parser/handshake bugs in
~5 weeks, all in the same shape — TCP fragmentation breaks the
one-shot buffer matcher in a way that wasn't covered by tests:
- 2bf2099 (mar 1): TCP split fix
- 0f11cfc (apr 2): client write lock missing
- 4222f98 (apr 2): httpFallback dispatch path
- 9ac64da (apr 5): handshake panic on TCP split mid-CRLF
- 3c6794a (apr 7): POST-with-body causes parse → null forever
each was fixed reactively after a downstream production incident.
the underlying issue is structural: parse() was a one-shot matcher
that re-scanned the whole buffer with indexOf on every call, and
the existing test suite only covered all-at-once happy paths. any
edge case in TCP fragmentation could miss the test net.
what changed
------------
State now carries a streaming end-of-headers detector
(std.http.HeadParser, stdlib's SIMD-optimized state machine) plus a
head_fed counter. parse() feeds only NEW bytes per call so the SIMD
scan is amortized across reads. boundary detection is no longer
sensitive to where TCP packet splits land.
the request/header parser proper is now line-based on the bounded
headers section, with explicit named errors at every malformed-input
exit. there are no `unreachable` branches reachable from any input.
raw_header byte slice is preserved verbatim for callers that re-emit
the original headers (existing test asserts this; passes unchanged).
test coverage
-------------
all 42 prior tests pass unchanged. five new test groups added,
~150 lines:
1. byte-split equivalence for valid upgrade — feed every chunk size
from 1 to N bytes, assert same final outcome
2. CRLF/CRLFCRLF split torture — every two-chunk split position
across a complete request, assert same result
3. malformed inputs return errors not panics — 8 adversarial cases
covering each named error path
4. POST body byte-split equivalence — every split position must
hold null until body fully arrives, then MissingHeaders
5. idempotence under repeated calls — parse twice with same len
produces same answer
47/47 tests pass. zig fmt clean.
what's not changed
------------------
external API: `Handshake`, `parse`, `State` field semantics that
callers depend on (`buf`, `len`, `req_headers`) all preserved.
this is a drop-in replacement for the parse internals; no
downstream code change required.
Handshake.parse used endsWith("\r\n\r\n") to decide when the buffer held
a complete request. That check only works for header-only requests
(GET). Any POST with a body ends on body bytes, so parse returned null
indefinitely — the worker kept reading, the client had already sent
everything, and the connection leaked until its idle timeout.
Concrete symptom: zlay's reconnect cronjob POSTs requestCrawl through
the same socket the WebSocket firehose is served on. Each request
stalled for the full recv timeout, so the job never re-announced any
PDSes and eventually hit its activeDeadlineSeconds.
Fix:
- locate header terminator with indexOf("\r\n\r\n") (not endsWith)
- parse only the header slice, leaving the body in state.buf for
parseHttpRequest to extract after MissingHeaders is raised
- reset req_headers.len at the top of parse so re-entry with more
data does not accumulate duplicate headers
- before raising MissingHeaders, consult Content-Length: if the body
hasn't fully arrived yet, return null so the worker reads more
before dispatching the fallback (avoids truncated body handoff)
Regression coverage: three new cases in "handshake: parse POST with
body signals MissingHeaders (for httpFallback)":
- full POST buffer → MissingHeaders (triggers httpFallback)
- partial body → null (read more)
- POST with no Content-Length → MissingHeaders (immediate)
when a TCP read delivers a \r at the buffer boundary without its \n,
line_start advances past pos. the next indexOfScalar call then panics
on buf[line_start..pos] because start > end.
under ReleaseFast this was a silent SIGSEGV (~every 30-90 min under
sustained load with ~2,800 connections). ReleaseSafe surfaces it as a
bounds check panic at client.zig:766.
fix: break out of the inner parse loop when line_start exceeds pos,
falling through to the outer read loop to get more data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously initWithStream hardcoded tls_client to null, silently
ignoring config.tls. Now it initializes TLS (using config.host for
SNI) when requested, matching the behavior of init().
This enables callers that bring their own pre-connected stream
(e.g. after resolving DNS through a separate Io backend) to still
get TLS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
thread io: Io through Server.init → WorkerState → ConnManager → Conn,
replacing hardcoded std.Options.debug_io. add Server.runIo() which uses
Io.net.Server.accept + io.concurrent per connection — under Evented
this becomes fiber-based (no OS thread per connection), under Threaded
it behaves like the old blocking path.
existing listen()/listenInNewThread() preserved for backwards compat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
read() now returns null on WouldBlock (SO_RCVTIMEO fired), letting
callers decide what to do. readLoopWithHeartbeat() uses this to send
pings on idle intervals and close after max_failures consecutive
timeouts — eliminating the need for a separate ping task/thread.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
when Handshake.parse fails with MissingHeaders, InvalidConnection,
or InvalidUpgrade — indicating a valid HTTP request that isn't a
websocket upgrade — check if the handler defines httpFallback and
dispatch to it instead of returning 400.
this restores the ability to serve health probes, XRPC endpoints,
and admin HTTP on the same port as the websocket server. the
fallback is gated by comptime hasFn: handlers without httpFallback
get no new codepath compiled.
the HTTP parser (parseHttpRequest) re-parses method/url/headers/body
from the raw buffer rather than reusing partial state from
Handshake.parse, which may have exited early with incomplete headers
and partially-lowercased names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
client.writeFrame() is called concurrently from:
- pingLoop task (subscriber keepalive)
- readLoop auto-pong (server ping response)
- close path (connection teardown)
without serialization, concurrent writes interleave frame headers/payloads,
corrupting the stream and causing GPF in memcpy → Writer.zig → writeAll.
adds _write_lock: Io.Mutex to Client struct, acquired around the two
writeAll calls in writeFrame(). header construction and payload masking
happen outside the lock to minimize hold time.
the server-side Conn already has this pattern (lock: Io.Mutex around
writeFrame/writeFramed/writeAllIOVec). this brings the client to parity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- use libc directly for socket operations (debug_io doesn't support real network)
- add Client.initWithStream() for testing with pre-existing sockets
- fix TestStream/testStream in server.zig to use libc
- fix SocketPair in t.zig to use libc socket/connect/send/recv
- darwin workaround: use fcntl for CLOEXEC instead of SOCK.CLOEXEC
concessions made:
- bypasses new Io abstraction for socket ops (libc direct calls)
- darwin-specific sockaddr workarounds
- test-only initWithStream API addition
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Still TODO:
- Io.net.Stream API completely changed - needs Reader/Writer wrappers
- Client.init now requires io: Io parameter
- Server networking needs similar Io threading
- Test utilities need io parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>