xeiaso.net
Proof of concept mechanical port of ircnet/ircd to Rust as part of a bit about C being insecure for network services
docs(ircd-golden): document L2 golden harness architecture and test writing guide
Add comprehensive documentation covering:
- How Client and Ircd types couple through the fixture port
- The dual-boot differential test model (reference-C vs Rust)
- One-shot vs persistent client drivers (drive vs Client)
- Boot family (boot, boot_standalone, boot_args, boot_conf)
- LD_PRELOAD shim for config redirection
- Canonicalization of volatile tokens
- Test structure and patterns
- Conventions checklist
- Build system wiring (build.rs)
- Concurrency model (flock serialization)
- Project-wide invariants (NO_DNS_LOOKUP, real clock, shared port)
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
author
Xe Iaso
date
(Jun 3, 2026, 9:34 AM -0400)
commit
66232e62
66232e62df31ebf1bbe2f2b8daeeb39c7b796d5d
parent
08d260ee
08d260eea5cdbf50883bb5b2c5fa85d450134353
+254
+254
docs/ircd-golden.md
+254
docs/ircd-golden.md
···
1
+
# ircd-golden: the L2 golden / characterization harness
2
+
3
+
`ircd-golden` is the differential test harness that gates the strangler-fig
4
+
port. Each test boots **two** real ircd processes — the pristine reference-C
5
+
binary and the Rust binary — drives the *same* scripted IRC session against
6
+
each over a real TCP socket, canonicalizes both transcripts, and asserts they
7
+
are **byte-identical**. If the Rust port's wire output diverges from C for any
8
+
covered scenario, the test fails.
9
+
10
+
This is "L2" in the migration's two-layer scheme: L1 is the in-process
11
+
`cref_`-differential unit harness (`ircd-testkit`); L2 is this end-to-end,
12
+
boot-the-whole-server, talk-IRC-over-a-socket harness. L2 is the gate every P5
13
+
command-handler port must pass.
14
+
15
+
## The big picture
16
+
17
+
```
18
+
#[test]
19
+
│
20
+
├─ boot(CREF_IRCD) ──► reference-C ircd ──┐ binds 127.0.0.1:16667
21
+
│ drive / Client ──► scripted session │ (one at a time, flock-gated)
22
+
│ canonicalize ◄── raw transcript ────┘
23
+
│
24
+
├─ boot(RUST_IRCD) ──► Rust ircd ──┐ binds 127.0.0.1:16667
25
+
│ drive / Client ──► same session │ (previous one already dropped)
26
+
│ canonicalize ◄── raw transcript ────┘
27
+
│
28
+
└─ assert_eq!(reference, rust)
29
+
```
30
+
31
+
Both binaries bind the **same** fixture port (`PORT = 16667`), so only one may
32
+
be alive at any instant. Boots are serialized by a cross-process advisory file
33
+
lock; the C transcript is captured and the C process killed before the Rust
34
+
process boots.
35
+
36
+
## How a client connects to an ircd
37
+
38
+
There is no handle linking a client to a server. The coupling is implicit,
39
+
through the fixed localhost port plus the boot lock:
40
+
41
+
- **`Ircd`** (`src/lib.rs`) is the server guard. `boot*()` returns it; it owns
42
+
the spawned `Child` and the lock `File`. `Drop for Ircd` kills the child and
43
+
releases the lock.
44
+
- **`Client`** holds only a `TcpStream`. `Client::connect()` dials
45
+
`127.0.0.1:16667` with no reference to any `Ircd`. Whatever server is bound to
46
+
that port right now is the one it talks to.
47
+
- The contract: a `Client` (or `drive`) is only valid while an `Ircd` guard is
48
+
alive in the same test. When the guard drops, the server dies and the sockets
49
+
go dead. Keep the guard bound (`let _ircd = boot(...)`) for the whole test.
50
+
51
+
## Two ways to drive a session
52
+
53
+
### One-shot drivers (single client)
54
+
55
+
`drive` and `drive_registered` open one socket, script it, and return the raw
56
+
transcript. Use them when one client is enough.
57
+
58
+
- **`drive(send_lines, sentinel)`** — connect, send each line (CRLF appended,
59
+
50ms barrier between them), then read until `sentinel` appears or a 5s idle
60
+
timeout. The session starts *unregistered*, so this is what registration tests
61
+
use.
62
+
- **`drive_registered(nick, post_lines, sentinel)`** — connect, send
63
+
`NICK`/`USER` for `nick`, drain the welcome banner through the `422` line
64
+
(MOTD-missing, the banner terminator in the fixture config), then send
65
+
`post_lines` and read until `sentinel`. Returns **only** the
66
+
post-registration transcript — the banner is dropped (it's covered by
67
+
`golden_registration`).
68
+
69
+
### Persistent `Client` (multi-connection)
70
+
71
+
`Client` keeps the socket open so several clients can run against one booted
72
+
server at once. This is required for **channel relay** tests:
73
+
`sendto_channel_butone` delivers a sender's PRIVMSG/NOTICE to the *other*
74
+
members and never echoes to the sender, so you must read the relayed line on a
75
+
second connection.
76
+
77
+
```rust
78
+
let _ircd = boot_standalone(binary);
79
+
let mut alice = Client::register("alice");
80
+
let mut bob = Client::register("bob");
81
+
alice.send("JOIN #chan");
82
+
bob.send("JOIN #chan");
83
+
alice.drain(); // discard JOIN echo + NAMES
84
+
bob.drain();
85
+
alice.send("PRIVMSG #chan :hello");
86
+
let got = canonicalize(&bob.read_until(" PRIVMSG #chan "));
87
+
```
88
+
89
+
`Client` methods:
90
+
91
+
- **`connect()`** — dial the port (assumes the ircd is already booted).
92
+
- **`register(nick)`** — connect + `NICK`/`USER`, draining the banner through
93
+
`422`.
94
+
- **`send(line)`** — send one line (CRLF appended) with a 50ms barrier so the
95
+
server processes one action at a time.
96
+
- **`read_until(sentinel)`** — read until `sentinel` or a 5s idle timeout;
97
+
return the transcript.
98
+
- **`drain()`** — discard pending input under a short (300ms) idle timeout, so
99
+
the next `read_until` sees only the message under test. Essential before
100
+
reading an echo, because the `SPLIT_CONNECT_NOTICE` can race into the
101
+
post-422 stream and falsely satisfy a `NOTICE` sentinel.
102
+
103
+
## Booting: the boot family
104
+
105
+
All boot functions acquire the cross-process lock, spawn the binary under the
106
+
`LD_PRELOAD` config-redirect shim pointed at a fixture config, poll the port
107
+
until it accepts (up to 5s), and return an `Ircd` guard.
108
+
109
+
- **`boot(binary)`** — `-t -s` (foreground, no iauth), `minimal.conf`. The
110
+
default.
111
+
- **`boot_standalone(binary)`** — adds `-p standalone` (`iconf.split = -1`, so
112
+
`IsSplit()` is false). A lone server is otherwise *permanently* split, which
113
+
forbids creating global `#` channels. Standalone lets `#`, `+`, and `&` all be
114
+
created — use it for channel tests so every prefix is covered uniformly.
115
+
- **`boot_args(binary, args)`** — arbitrary argv against `minimal.conf`.
116
+
- **`boot_conf(binary, args, conf)`** — arbitrary argv against
117
+
`fixtures/<conf>`. Use a dedicated config (e.g. `oper.conf` adds an O-line)
118
+
to isolate a test's fixture needs from every other test.
119
+
120
+
### The config-redirect shim
121
+
122
+
The ircd has `CMDLINE_CONFIG` undefined, so it always opens the compiled-in
123
+
path `/usr/local/etc/ircd.conf`. `csrc/l2shim.c` is an `LD_PRELOAD` shim that
124
+
intercepts `open`/`open64` and rewrites *that one path* to
125
+
`$IRCD_TEST_CONFIG`. This lets tests supply a per-run config with no root and
126
+
no recompile. The shim deliberately does **not** touch the clock: pinning
127
+
`time()` to a constant freezes the event loop (the global `timeofday` never
128
+
advances and the listen socket is never serviced), so time determinism is
129
+
handled by `canonicalize` instead.
130
+
131
+
## Canonicalization: masking volatile tokens
132
+
133
+
Two servers booted milliseconds apart will emit a few wall-clock- or
134
+
heap-dependent tokens that differ. `canonicalize(transcript)` masks exactly
135
+
those positions so the runs compare equal. Keep it in sync with `ircd/s_err.c`.
136
+
Currently it:
137
+
138
+
- drops the `SPLIT_CONNECT_NOTICE` line (lands non-deterministically due to TCP
139
+
chunking; belongs to no command's response),
140
+
- masks `242 RPL_STATSUPTIME` (`:Server Up …`),
141
+
- masks numeric tokens in `317 RPL_WHOISIDLE` (idle/signon seconds),
142
+
- masks the `391 RPL_TIME` trailing localtime string,
143
+
- masks `sbrk(0)-etext:` (heap break, differs between the two binaries),
144
+
- masks `:time connected …` in STATS t (wall-clock volatile).
145
+
146
+
If a new scenario surfaces a volatile token, add a mask here — don't loosen the
147
+
`assert_eq!`.
148
+
149
+
## Anatomy of a golden test
150
+
151
+
The standard shape: a helper that runs one scripted session against one binary
152
+
and returns its canonicalized transcript, then a `#[test]` that runs it against
153
+
both binaries, sanity-checks the reference output, and asserts equality.
154
+
155
+
```rust
156
+
use ircd_golden::{boot, canonicalize, drive};
157
+
158
+
fn registration_transcript(binary: &str) -> String {
159
+
let _ircd = boot(binary); // killed on drop
160
+
let raw = drive(
161
+
&["NICK alice", "USER alice 0 * :Alice Tester"],
162
+
" 422 ", // sentinel ends the banner
163
+
);
164
+
canonicalize(&raw)
165
+
}
166
+
167
+
#[test]
168
+
fn registration_banner_matches_reference() {
169
+
// Sequential: both bind the same port, so never concurrently.
170
+
let reference = registration_transcript(env!("CREF_IRCD"));
171
+
let rust = registration_transcript(env!("RUST_IRCD"));
172
+
173
+
// Sanity-check the reference: prove the scenario actually exercised the
174
+
// code path before trusting the equality assertion.
175
+
assert!(reference.contains(" 001 alice "), "no banner:\n{reference}");
176
+
assert!(
177
+
reference.contains("alice!~alice@127.0.0.1"),
178
+
"host not numeric (DNS not disabled?):\n{reference}"
179
+
);
180
+
181
+
assert_eq!(reference, rust, "reference-C vs Rust wire output diverged");
182
+
}
183
+
```
184
+
185
+
### Why the reference sanity-check matters
186
+
187
+
`assert_eq!(reference, rust)` passes trivially if *both* transcripts are empty —
188
+
e.g. a sentinel that never arrives times out on both sides and yields two empty
189
+
strings. The `assert!(reference.contains(...))` lines guard against that: they
190
+
prove the reference run actually reached the behavior under test, so equality
191
+
means "both produced the right thing," not "both produced nothing."
192
+
193
+
### Picking a sentinel
194
+
195
+
`read_until` / `drive` stop at the first occurrence of the sentinel substring.
196
+
Choose the last line you expect for the command — a final numeric (`" 401 "`,
197
+
`" 422 "`) or the echoed command token (`" PRIVMSG alice "`). Surround numerics
198
+
with spaces so they don't match inside another token. If a notice can race into
199
+
the stream and contain your sentinel substring (the split-mode notice contains
200
+
`NOTICE`), `drain()` first.
201
+
202
+
## Conventions checklist for a new test
203
+
204
+
1. Add `tests/golden_<area>.rs` (each file is its own test binary).
205
+
2. Write a `*_transcript(binary: &str) -> String` helper: `boot*`, drive, then
206
+
`canonicalize`.
207
+
3. In the `#[test]`, call the helper with `env!("CREF_IRCD")` and
208
+
`env!("RUST_IRCD")`.
209
+
4. Sanity-check the reference transcript contains the expected marker.
210
+
5. `assert_eq!(reference, rust, "<what diverged>")`.
211
+
6. Use `boot_standalone` for channel tests; use `boot_conf` with a dedicated
212
+
fixture for tests needing extra config (opers, kill grants, …).
213
+
7. For multi-client scenarios use `Client`; `drain()` before reading an echo.
214
+
8. If a new volatile token appears, mask it in `canonicalize` — never loosen
215
+
the equality assertion.
216
+
217
+
## How the binaries get built (build.rs)
218
+
219
+
`build.rs` produces the two artifacts and exports their paths as compile-time
220
+
env vars consumed via `env!()`:
221
+
222
+
- **`CREF_IRCD`** — the reference-C ircd. `ircd.c` is recompiled with a native
223
+
`main` (the cbuild `ircd.o` is built with `-Dmain=c_ircd_main`, so it has
224
+
none) via `make --eval`, then linked against the pristine all-C base objects
225
+
(`REF_OBJS`, the plain `.o` oracle objects — *not* the `_link.o`
226
+
partial-port variants). Built with `-DNO_DNS_LOOKUP` so DNS is disabled and
227
+
hosts come out numeric.
228
+
- **`RUST_IRCD`** — derived from `OUT_DIR` as `target/<profile>/ircd`.
229
+
- **`L2_SHIM`** — the compiled `l2shim.so`.
230
+
- **`L2_FIXTURES`** — the `fixtures/` directory.
231
+
232
+
A `build-dependency` on `ircd-sys` forces its `build.rs` to run first, so
233
+
`cbuild/*.o` exists before this crate links the reference binary. The build
234
+
asserts loudly if `cbuild/` isn't materialized.
235
+
236
+
## Concurrency model
237
+
238
+
`cargo test` runs each test *file* as its own process and runs several files in
239
+
parallel. An in-process `Mutex` can't serialize boots across processes, so
240
+
`acquire_port_lock()` takes an exclusive `flock` on `fixtures/.port.lock`. The
241
+
lock is held for the lifetime of the `Ircd` guard (it owns the `File`) and
242
+
released when the guard drops and the fd closes. Separate fds contend even
243
+
within one process, so this also serializes parallel `#[test]` threads inside a
244
+
single binary. Net effect: only one ircd ever owns port 16667 at a time, across
245
+
all test processes and threads.
246
+
247
+
## Project-wide invariants this harness relies on
248
+
249
+
- **DNS disabled** (`NO_DNS_LOOKUP`) so hostnames are numeric and deterministic
250
+
(`alice!~alice@127.0.0.1`).
251
+
- **Real clock** — never pin `time()`; the canonicalizer masks the handful of
252
+
time-bearing tokens instead.
253
+
- **Same fixture port** for both binaries — the source of the one-at-a-time
254
+
boot invariant.