the three upstream fixes weir surfaced have landed in their proper
homes, so weir now depends on blessed sources instead of the temporary
reference branches:

- zat -> v0.3.7 (release; carries the MST encoding-cache fix + the
collectBlocksInto/corruption fix)
- http.zig -> master (websocket v0.1.7 alignment, merged)
- redis -> main tip 507f69f (retire-read-buffers fix, merged)

removes NOTE-upstream-fixes-needed.md per its own exit condition.
verified: clean build, 6/6 tests, demo flow runs end to end, boot
hydration replays the repo (51 records) without BadBlockHash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

read all of guidry's proven + disproven proofs (datafusion reads and
caching, filter compilation, cross-entity queries, cas semantics,
schema evolution, partition pruning, compaction, and the three
disproven architectures) and lined them up against weir.

headline: we independently re-ran his disproven all-you-need-is-
iceberg experiment — commit-per-write storming a snapshot store — and
landed on the same lever (flush cadence / commit_every) with the same
mitigation (unflushed writes stay queryable). and his other disproven
path, the hand-rolled parquet+redis catalog, is the trap weir never
entered because the atproto repo IS the snapshot model he migrated to:
iceberg for records, plus signatures.

his unabsorbed lessons become weir's ordered roadmap: pushdown, skip
hydration when the appview is warm, TID-range pruning for listEvents,
snapshot-shaped compaction, query-engine pooling when multi-tenant.
weir's side of the ledger: verifiability, protocol-grade portability,
the public seam, one binary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

codifies the direction conversation: the ORM decomposes (lexicons take
schema+migrations, the appview takes query building, the commit takes
transactions, nothing takes object-row mapping because the mismatch is
gone), the seam is the signed commit and the firehose is that seam made
public, and the four-way rule — policy writes are procedures, plain
writes are applyWrites, reads are declared appview queries, everything
else subscribes.

first declared queries: io.prefect.v0.listRuns / getRun (the appview
reads every REST filter/paginate translates onto), listEvents (served
from the canonical log itself), and proposeState — the policy-gated
procedure that set_state becomes, with ACCEPT/REJECT/ABORT/WAIT and a
StaleTransition CAS error.

v0 posture stated in the doc: experiment freely, add optional fields
without ceremony, mint v1 instead of breaking, let old records rest in
the log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

POST /events/filter and /events/count-by/{countable} are real now —
and they read straight from the repo's io.prefect.v0.event collection:
TID-keyed records decode back into prefect's ReceivedEvent shape,
filtered by occurred range / event prefix / resource id, newest-first,
with EventPage and EventCount envelopes. the histogram buckets by hour
and carries each bucket's min/max occurred as start/end.

this is the page where the architecture stops hiding: the feed the ui
renders IS the signed event log, no derived table in between. (at
scale it wants an appview index — collection scan per request is honest
about being v0.) replaces the empty-envelope fallback that the graph
fix introduced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

pixi's graph parser hard-requires start_time on every node ('Start time
is required'). two layers of fix: hydration replay now falls back to
the event's occurred when validated_state has no timestamp (websocket-
ingested task-run events never carry one — the live ingest path always
stamped occurred, so this only bit after a restart), and graphV2 never
emits an empty-string time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

the ui's task-runs view starves on garnish endpoints, not data — fixed:
- /flow_runs/{id}/graph-v2 builds real nodes from the run's task runs,
with parent/child edges derived from task_inputs references and
root_node_ids for parentless tasks (cribbed from prefect-server)
- /ui/flow_runs/count-task-runs answers from the flowrun membership
sets (the tab badge read 0 from the {} stub)
- GET /ui/task_runs/{id} returns the run + flow_run_name
- task_runs/filter supports id.any_ (the runs page enrichment query)
- POST /events/filter returns prefect's {events,total,next_page}
envelope — the bare [] fallback crashed the graph overlay
(e.events is undefined, api.ts:96)

also restores filterAny/intersect/hydrate/cborFieldJson, casualties of
an over-greedy patch boundary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

prefect 3.x creates task runs locally and reports them only as events
over ws /api/events/in — so weir now speaks that socket: upgrade with
the 'prefect' subprotocol echoed when offered, the auth handshake
({type:auth} -> {type:auth_success}; every token succeeds, auth is an
acknowledged gap), and ingest of task-run events: upsert into the
appview + verbatim io.prefect.v0.event records + entity record on
first sighting. flow-run events are skipped (already canonical via
REST). the missing handshake reply was why flows hung at teardown —
the client blocks on recv after sending auth.

found + fixed a corruption regression in the zat encoding cache along
the way: nodeCid cleared dirty without refreshing the cached encoding,
so every CAR written after the second commit failed reload with
BadBlockHash. fixed upstream (zat#mst-block-cache 3960e70) with a
regression test there and a multi-commit reload test here.

infra hygiene per review: compose.yml owns the appview redis and the
justfile is reorganized around it (up/down/serve/demo/demo-flow/mirror
/bench-*). scripts/demo-flow is a task-rich etl: 8 mapped transforms,
a retry survivor (run_count=3), a tolerated failure.

verified: etl_demo() = 204, 11 task runs in the appview (10 completed,
1 failed), 38 events in the canonical log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

the ui-v2 dist ships with /PREFECT_UI_SERVE_BASE_REPLACE_PLACEHOLDER
baked into asset urls; the python server rewrites it at startup
(empty string when serving at /). fetch-ui now does the same with sed,
matching server.py's replace_placeholder_string_in_files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

weir now serves the prefect react ui (ui-v2 bundle, copied from the
python package via just fetch-ui): static SPA serving with index.html
fallback, /ui-settings, real /paginate for flows + flow/task runs
(the list pages use paginate, not filter), a graph-v2 stub for the run
detail page, and generic empty-state fallbacks for the ui's long tail
of filter/count/history/paginate queries.

scripts/space-mirror is the first background service in the
architecture's own idiom: it consumes weir's canonical log (listRecords
polling — the firehose consumer's placeholder) and pushes new records
into a permissioned io.prefect.v0.workspace space on zds as waow.tech.
restart-safe via diffing against the space's existing rkeys.

verified end to end: basic_flow runs against weir; the ui's paginate
shows the run COMPLETED; the mirror lands all 6 records in
ats://did:plc:siv7.../io.prefect.v0.workspace/weir-demo where the
Pending->Running->Completed events read back with auth and strangers
get 401. just demo wires the whole thing up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

a weir shapes flow, measures it, and never owns the water. the name
also stops claiming pds (this is an appview + write gateway hosting a
native repo) and stops squatting the prefect prefix on the repo's
public face — io.prefect.v0.* stays as the lexicon namespace, which is
a separate, deliberately-versioned decision.

binary, package name, and env vars follow (WEIR_PORT, WEIR_DATA_DIR,
WEIR_CACHE, WEIR_REDIS_HOST/PORT, WEIR_COMMIT_EVERY). new readme leads
with mermaid diagrams contrasting prefect's self-hosted shape (LB ->
api xN -> shared postgres+redis) against weir's canonical/derived
split, and the ownership asymmetry table that is the actual thesis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

production shape lands: the Cache union gains a real-redis-over-TCP
backend (PREFECT_PDS_CACHE=redis, container via orbstack), selected at
boot; burner stays the zero-infra default. commit cadence becomes a
knob (PREFECT_PDS_COMMIT_EVERY) — the same lever as the cloud
prototype's delta-flush cadence.

scripts/bench-scale drives 16 concurrent client lifecycles in
cumulative tiers and found three real bugs before producing numbers:
quadratic CAR retention here (fixed: per-commit scratch arena +
child-allocator-owned latest_car), quadratic MST re-serialization in
zat (fixed upstream on zat#mst-block-cache), and realloc-dangled
response slices in the redis client under large SMEMBERS (fixed
upstream on redis#retired-read-buffers).

results (docs/scale-results.md): 8000 runs / ~30k records, no crashes.
point reads flat at ~1ms across 32x scale; per-write commits decay
O(records) as predicted; commit_every=20 flattens throughput at
840 w/s with p95 *improving* at depth. remaining walls are slopes with
named fixes, not cliffs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

the appview grows up: membership sets per (kind, state / flow / parent)
in burner-redis — the same shape as the cloud prototype's redis run-
state indices — back POST /flow_runs/filter, /count, /task_runs/filter
and /flows/filter. boot now rebuilds the whole appview by replaying the
repo (entities, then the event log in TID order), so reads survive
restarts: the delete-the-cache-replay-the-truth property is exercised
on every boot.

docs/access-patterns.md grounds the design against guidry's proofs in
prefectlabs/object-store-orchestration (latency bands, the delta-size
lever, the disproven many-small-objects pattern) and lays out the
three-tier aggregation story (maintained counters / bounded scans /
CAR-export analytics).

scripts/bench-reads runs the same seeded workload against any prefect
REST target. results (docs/bench-results.md): membership reads 4-11x
faster than python on sqlite or postgres, bulk lists tied at n=300,
202/s writes while signing a commit and rewriting the CAR every
mutation. all four client read paths (point + filter) now pass against
a restarted server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

go-crazy namespace decision: everything lives under io.prefect.v0.* and
problems down the road drive v1. seven lexicon files land in lexicons/
(defs with the shared #state shape, event, flow, flowRun, taskRun,
logBatch, workspace).

the server now writes state transitions as io.prefect.v0.event records
mirroring prefect's own Event schema — name prefect.{flow-run,task-run}.
{StateName}, resource labels, intended {from,to} + validated_state in
the payload, TID rkeys. the state collections are gone; current state
is appview-only, per docs/lexicons.md.

verified live: python client runs basic_flow to Completed and the repo
holds Pending->Running->Completed events with causal from-chains; zds
poc reruns clean with the v0 workspace space type and oplog.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

the central move: there is no flowRunState record. prefect has one State
shape and already broadcasts transitions as Events — so the lexicon is
one generic io.prefect.event record (mirroring prefect's Event schema:
occurred/event/resource/related/payload/follows, TID rkeys) plus
state as a shared def, entity records without live state, and the
io.prefect.workspace space type. live current-state is appview
territory, not a record — same split as the cloud initiative's
orchestration (redis) vs observability (iceberg) planes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

the workspace-as-space model, demonstrated against a real zds instance:
an io.prefect.workspace space holds private io.prefect.flowRunState
records (ats:// uris), getRepoOplog serves the private event stream,
and unauthenticated reads get 401. the public layer works as on any pds.

zds also caught a spec violation bsky.social silently accepted:
io.prefect.flowRun.state is an invalid nsid (authority segments must be
lowercase; camelCase only in the final name segment). renamed to
io.prefect.flowRunState / io.prefect.taskRunState everywhere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

writes io.prefect.flowRun.state to an ordinary bsky.social account and
watches it arrive on the public jetstream — proving the remote-pds track
needs zero infrastructure: identity auth (app password), arbitrary
lexicon collections accepted, and bluesky's relay as the event stream.

verified live: record at .../io.prefect.flowRun.state/3mo36ld65m62c
round-tripped pds -> jetstream -> consumer in seconds.

also: just watch (zig build --watch -fincremental).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>