ircd-golden: the L2 golden / characterization harness#

ircd-golden is the differential test harness that gates the strangler-fig port. Each test boots two real ircd processes — the pristine reference-C binary and the Rust binary — drives the same scripted IRC session against each over a real TCP socket, canonicalizes both transcripts, and asserts they are byte-identical. If the Rust port's wire output diverges from C for any covered scenario, the test fails.

This is "L2" in the migration's two-layer scheme: L1 is the in-process cref_-differential unit harness (ircd-testkit); L2 is this end-to-end, boot-the-whole-server, talk-IRC-over-a-socket harness. L2 is the gate every P5 command-handler port must pass.

The big picture#

  #[test]
    │
    ├─ boot(CREF_IRCD)  ──► reference-C ircd  ──┐  binds 127.0.0.1:16667
    │     drive / Client ──► scripted session   │  (one at a time, flock-gated)
    │     canonicalize    ◄── raw transcript ────┘
    │
    ├─ boot(RUST_IRCD)  ──► Rust ircd          ──┐  binds 127.0.0.1:16667
    │     drive / Client ──► same session       │  (previous one already dropped)
    │     canonicalize    ◄── raw transcript ────┘
    │
    └─ assert_eq!(reference, rust)

Both binaries bind the same fixture port (PORT = 16667), so only one may be alive at any instant. Boots are serialized by a cross-process advisory file lock; the C transcript is captured and the C process killed before the Rust process boots.

How a client connects to an ircd#

There is no handle linking a client to a server. The coupling is implicit, through the fixed localhost port plus the boot lock:

Ircd (src/lib.rs) is the server guard. boot*() returns it; it owns the spawned Child and the lock File. Drop for Ircd kills the child and releases the lock.
Client holds only a TcpStream. Client::connect() dials 127.0.0.1:16667 with no reference to any Ircd. Whatever server is bound to that port right now is the one it talks to.
The contract: a Client (or drive) is only valid while an Ircd guard is alive in the same test. When the guard drops, the server dies and the sockets go dead. Keep the guard bound (let _ircd = boot(...)) for the whole test.

Two ways to drive a session#

One-shot drivers (single client)#

drive and drive_registered open one socket, script it, and return the raw transcript. Use them when one client is enough.

drive(send_lines, sentinel) — connect, send each line (CRLF appended, 50ms barrier between them), then read until sentinel appears or a 5s idle timeout. The session starts unregistered, so this is what registration tests use.
drive_registered(nick, post_lines, sentinel) — connect, send NICK/USER for nick, drain the welcome banner through the 422 line (MOTD-missing, the banner terminator in the fixture config), then send post_lines and read until sentinel. Returns only the post-registration transcript — the banner is dropped (it's covered by golden_registration).

Persistent `Client` (multi-connection)#

Client keeps the socket open so several clients can run against one booted server at once. This is required for channel relay tests: sendto_channel_butone delivers a sender's PRIVMSG/NOTICE to the other members and never echoes to the sender, so you must read the relayed line on a second connection.

let _ircd = boot_standalone(binary);
let mut alice = Client::register("alice");
let mut bob   = Client::register("bob");
alice.send("JOIN #chan");
bob.send("JOIN #chan");
alice.drain();           // discard JOIN echo + NAMES
bob.drain();
alice.send("PRIVMSG #chan :hello");
let got = canonicalize(&bob.read_until(" PRIVMSG #chan "));

Client methods:

connect() — dial the port (assumes the ircd is already booted).
register(nick) — connect + NICK/USER, draining the banner through 422.
send(line) — send one line (CRLF appended) with a 50ms barrier so the server processes one action at a time.
read_until(sentinel) — read until sentinel or a 5s idle timeout; return the transcript.
drain() — discard pending input under a short (300ms) idle timeout, so the next read_until sees only the message under test. Essential before reading an echo, because the SPLIT_CONNECT_NOTICE can race into the post-422 stream and falsely satisfy a NOTICE sentinel.

Booting: the boot family#

All boot functions acquire the cross-process lock, spawn the binary under the LD_PRELOAD config-redirect shim pointed at a fixture config, poll the port until it accepts (up to 5s), and return an Ircd guard.

boot(binary) — -t -s (foreground, no iauth), minimal.conf. The default.
boot_standalone(binary) — adds -p standalone (iconf.split = -1, so IsSplit() is false). A lone server is otherwise permanently split, which forbids creating global # channels. Standalone lets #, +, and & all be created — use it for channel tests so every prefix is covered uniformly.
boot_args(binary, args) — arbitrary argv against minimal.conf.
boot_conf(binary, args, conf) — arbitrary argv against fixtures/<conf>. Use a dedicated config (e.g. oper.conf adds an O-line) to isolate a test's fixture needs from every other test.

The config-redirect shim#

The ircd has CMDLINE_CONFIG undefined, so it always opens the compiled-in path /usr/local/etc/ircd.conf. csrc/l2shim.c is an LD_PRELOAD shim that intercepts open/open64 and rewrites that one path to $IRCD_TEST_CONFIG. This lets tests supply a per-run config with no root and no recompile. The shim deliberately does not touch the clock: pinning time() to a constant freezes the event loop (the global timeofday never advances and the listen socket is never serviced), so time determinism is handled by canonicalize instead.

Canonicalization: masking volatile tokens#

Two servers booted milliseconds apart will emit a few wall-clock- or heap-dependent tokens that differ. canonicalize(transcript) masks exactly those positions so the runs compare equal. Keep it in sync with ircd/s_err.c. Currently it:

drops the SPLIT_CONNECT_NOTICE line (lands non-deterministically due to TCP chunking; belongs to no command's response),
masks 242 RPL_STATSUPTIME (:Server Up …),
masks numeric tokens in 317 RPL_WHOISIDLE (idle/signon seconds),
masks the 391 RPL_TIME trailing localtime string,
masks sbrk(0)-etext: (heap break, differs between the two binaries),
masks :time connected … in STATS t (wall-clock volatile).

If a new scenario surfaces a volatile token, add a mask here — don't loosen the assert_eq!.

Anatomy of a golden test#

The standard shape: a helper that runs one scripted session against one binary and returns its canonicalized transcript, then a #[test] that runs it against both binaries, sanity-checks the reference output, and asserts equality.

use ircd_golden::{boot, canonicalize, drive};

fn registration_transcript(binary: &str) -> String {
    let _ircd = boot(binary);                       // killed on drop
    let raw = drive(
        &["NICK alice", "USER alice 0 * :Alice Tester"],
        " 422 ",                                    // sentinel ends the banner
    );
    canonicalize(&raw)
}

#[test]
fn registration_banner_matches_reference() {
    // Sequential: both bind the same port, so never concurrently.
    let reference = registration_transcript(env!("CREF_IRCD"));
    let rust = registration_transcript(env!("RUST_IRCD"));

    // Sanity-check the reference: prove the scenario actually exercised the
    // code path before trusting the equality assertion.
    assert!(reference.contains(" 001 alice "), "no banner:\n{reference}");
    assert!(
        reference.contains("alice!~alice@127.0.0.1"),
        "host not numeric (DNS not disabled?):\n{reference}"
    );

    assert_eq!(reference, rust, "reference-C vs Rust wire output diverged");
}

Why the reference sanity-check matters#

assert_eq!(reference, rust) passes trivially if both transcripts are empty — e.g. a sentinel that never arrives times out on both sides and yields two empty strings. The assert!(reference.contains(...)) lines guard against that: they prove the reference run actually reached the behavior under test, so equality means "both produced the right thing," not "both produced nothing."

Picking a sentinel#

read_until / drive stop at the first occurrence of the sentinel substring. Choose the last line you expect for the command — a final numeric (" 401 ", " 422 ") or the echoed command token (" PRIVMSG alice "). Surround numerics with spaces so they don't match inside another token. If a notice can race into the stream and contain your sentinel substring (the split-mode notice contains NOTICE), drain() first.

Conventions checklist for a new test#

Add tests/golden_<area>.rs (each file is its own test binary).
Write a *_transcript(binary: &str) -> String helper: boot*, drive, then canonicalize.
In the #[test], call the helper with env!("CREF_IRCD") and env!("RUST_IRCD").
Sanity-check the reference transcript contains the expected marker.
assert_eq!(reference, rust, "<what diverged>").
Use boot_standalone for channel tests; use boot_conf with a dedicated fixture for tests needing extra config (opers, kill grants, …).
For multi-client scenarios use Client; drain() before reading an echo.
If a new volatile token appears, mask it in canonicalize — never loosen the equality assertion.

How the binaries get built (build.rs)#

build.rs produces the two artifacts and exports their paths as compile-time env vars consumed via env!():

CREF_IRCD — the reference-C ircd. ircd.c is recompiled with a native main (the cbuild ircd.o is built with -Dmain=c_ircd_main, so it has none) via make --eval, then linked against the pristine all-C base objects (REF_OBJS, the plain .o oracle objects — not the _link.o partial-port variants). Built with -DNO_DNS_LOOKUP so DNS is disabled and hosts come out numeric.
RUST_IRCD — derived from OUT_DIR as target/<profile>/ircd.
L2_SHIM — the compiled l2shim.so.
L2_FIXTURES — the fixtures/ directory.

A build-dependency on ircd-sys forces its build.rs to run first, so cbuild/*.o exists before this crate links the reference binary. The build asserts loudly if cbuild/ isn't materialized.

Concurrency model#

cargo test runs each test file as its own process and runs several files in parallel. An in-process Mutex can't serialize boots across processes, so acquire_port_lock() takes an exclusive flock on fixtures/.port.lock. The lock is held for the lifetime of the Ircd guard (it owns the File) and released when the guard drops and the fd closes. Separate fds contend even within one process, so this also serializes parallel #[test] threads inside a single binary. Net effect: only one ircd ever owns port 16667 at a time, across all test processes and threads.

Project-wide invariants this harness relies on#

DNS disabled (NO_DNS_LOOKUP) so hostnames are numeric and deterministic (alice!~alice@127.0.0.1).
Real clock — never pin time(); the canonicalizer masks the handful of time-bearing tokens instead.
Same fixture port for both binaries — the source of the one-at-a-time boot invariant.

Configure Feed