zzstoatzz.io
This repository has no description
docs: compared learnings with the cloud prototype's proof corpus
read all of guidry's proven + disproven proofs (datafusion reads and
caching, filter compilation, cross-entity queries, cas semantics,
schema evolution, partition pruning, compaction, and the three
disproven architectures) and lined them up against weir.
headline: we independently re-ran his disproven all-you-need-is-
iceberg experiment — commit-per-write storming a snapshot store — and
landed on the same lever (flush cadence / commit_every) with the same
mitigation (unflushed writes stay queryable). and his other disproven
path, the hand-rolled parquet+redis catalog, is the trap weir never
entered because the atproto repo IS the snapshot model he migrated to:
iceberg for records, plus signatures.
his unabsorbed lessons become weir's ordered roadmap: pushdown, skip
hydration when the appview is warm, TID-range pruning for listEvents,
snapshot-shaped compaction, query-engine pooling when multi-tenant.
weir's side of the ledger: verifiability, protocol-grade portability,
the public seam, one binary.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
author
zzstoatzz
co-author
Claude Fable 5
date
(Jun 12, 2026, 4:57 PM -0500)
commit
eab4f0cc
eab4f0cc7e7d75a40cfe959467c8992220188534
parent
7750deda
7750deda7fbfb5f03d84b63800de0f03f5b8f62b
+123
+123
docs/cloud-prototype-comparison.md
+123
docs/cloud-prototype-comparison.md
···
1
+
# weir vs the cloud prototype: compared learnings
2
+
3
+
`prefectlabs/object-store-orchestration` (guidry) is the proof-driven
4
+
prototype behind the cloud data-architecture initiative: 22 proven proofs,
5
+
3 disproven ones, a GKE whale-workspace simulator, and a rust
6
+
datafusion/iceberg read layer. weir is the atproto-canonical version of the
7
+
same instinct. this doc lines up what each has learned, because the overlap
8
+
is not coincidence — it's the same architecture meeting different trust
9
+
boundaries.
10
+
11
+
## convergences we found independently
12
+
13
+
**commit-per-write doesn't scale on snapshot stores — buffer and batch.**
14
+
his disproven `all-you-need-is-iceberg`: committing every write storms the
15
+
catalog (200+ commits/s → CAS losers dominate, ~720k manifests/hour, query
16
+
planning walks a million manifest entries, tiny files defeat the cache).
17
+
weir hit the identical wall from the other side: signing + rewriting the
18
+
CAR per write decays from 855 → 118 writes/s as the repo grows. both
19
+
architectures landed on the same lever — his delta buffer with flush
20
+
cadence, our `commit_every` — and both keep unflushed writes queryable
21
+
(his read-time delta merge; our appview dual-write). same physics,
22
+
discovered twice.
23
+
24
+
**hand-rolling the catalog is a trap — adopt a snapshot model.** his
25
+
disproven `all-you-need-is-parquet-and-redis`: the bespoke file-list HASH
26
+
catalog suffered CAS storms under power-law load, cache-hostile in-place
27
+
overwrites, and no schema evolution; iceberg's snapshot pointer fixed all
28
+
three. weir never hit this because **the atproto repo IS a snapshot model
29
+
— iceberg for records, plus signatures**: metadata.json pointer CAS ≈ the
30
+
commit swap; manifest list ≈ the MST; immutable content-addressed data
31
+
files ≈ blocks. we inherited on day one the thing he had to migrate to.
32
+
33
+
**many small objects is death.** his disproven `state-partitioned-indexes`
34
+
(52–95× degradation at 500 delta files) and the iceberg tiny-files problem
35
+
↔ our one-CAR/batched-commits decision. both sides now treat "one object
36
+
per record" as a known anti-pattern.
37
+
38
+
**live state is not archival state.** his two-tier active-run design
39
+
(redis holds live attributes; the durable record holds identity + config;
40
+
handoff at terminal) is exactly weir's entity-records-without-state +
41
+
appview split — we cited his plane doc when we made that call, but the
42
+
proofs show how load-bearing it is: it's what keeps the hot path off the
43
+
snapshot store entirely.
44
+
45
+
**schema evolution without migrations.** iceberg's column-ID evolution
46
+
(rename/drop safely, old snapshots stay consistent) and lexicon evolution
47
+
(additive-only, breaking = mint v1) are different mechanisms for the same
48
+
vow: never run a blocking migration again. iceberg evolves in place by ID;
49
+
lexicons freeze and version. ours is stricter and simpler; his preserves
50
+
renamability.
51
+
52
+
## his lessons weir has not yet absorbed
53
+
54
+
1. **pushdown.** his filter compilation turns prefect filters into
55
+
parameterized datafusion SQL with LIMIT/OFFSET pushed down — ~40× over
56
+
the N+1 approach, only the requested page materializes. weir's filter
57
+
engine loads every candidate row, sorts, then slices (visible in the
58
+
scale results: completed_50 grows 16 → 477ms). the fix is known
59
+
(time-ordered zset, slice before loading) and his numbers say it's
60
+
worth doing before any real workspace.
61
+
2. **cold-start economics.** his read path evolved three times:
62
+
download-everything → native byte-range reads → cluster-shared redis
63
+
byte-range cache with write-time priming (cold 23.5s → warm-everywhere
64
+
tens-of-ms). weir's equivalent sin: hydration replays the entire repo
65
+
on every boot even when the redis appview is already warm — we should
66
+
skip replay when the appview is populated, and prime it at write time
67
+
(which we already do; we just throw it away on restart).
68
+
3. **stats-based pruning beats filename contracts.** his hourly-file
69
+
pruning worked (linear ~7ms/file) but iceberg's per-file min/max stats
70
+
superseded it. weir's analog is better than the filename version and
71
+
unbuilt: TID rkeys make the event collection a time-ordered keyspace,
72
+
so an MST range walk IS partition pruning. `listEvents` should learn
73
+
`since`/`until` key-range walks instead of full collection scans.
74
+
4. **compaction wants snapshot atomicity.** his pre-iceberg compaction
75
+
needed deferred deletes, two-pass rewrites, version gating; iceberg
76
+
collapsed it to "write new files, swap pointer, expire old snapshots."
77
+
weir's block-level CAR append + periodic repack inherits the same
78
+
trivial shape from content addressing — old blocks stay valid, the
79
+
swap is one rename. design accordingly when the CAR rewrite finally
80
+
gets replaced.
81
+
5. **query-engine pooling for multi-tenancy.** datafusion over duckdb
82
+
because SessionContexts are ~1MB and stateless (LRU pool of 128) while
83
+
duckdb forces instance-per-workspace or shared-instance-no-isolation.
84
+
irrelevant at weir's single-workspace scale; decisive the day weir
85
+
hosts many. the trait stack he praises (replaceable I/O layers) is the
86
+
same property we exploit in zat.
87
+
88
+
## what weir has that the prototype cannot express
89
+
90
+
- **verifiability.** signed commits, offline verification against the
91
+
workspace DID. his planes have no provenance story at all; ours is the
92
+
default.
93
+
- **portability as protocol, not format.** his BYOS argument is "iceberg
94
+
in your bucket — point any engine at it." ours is that plus identity:
95
+
`getRepo` hands you the workspace, the DID makes it resolvable, and the
96
+
format is specified by a protocol with multiple implementations.
97
+
- **the seam is public.** his event stream feeds internal consumers;
98
+
search rewiring onto it is an initiative project. weir's firehose/oplog
99
+
is the architecture's front door — the event feed UI reads the
100
+
canonical log directly, and any third party could too.
101
+
- **one binary.** his prototype spans python + rust + redis + GCS + a
102
+
catalog; weir is a ~1MB zig binary plus optional redis. different
103
+
scales, but the operational floor difference is real.
104
+
105
+
## shared open questions
106
+
107
+
- concurrent throughput at production fan-in (his latency numbers are
108
+
single-query; ours stop at 16 clients).
109
+
- whale-scale terminal queries (~23.5M rows) — neither side has measured.
110
+
- the postgres head-to-head he lists as open — weir's bench-reads covers
111
+
it only at n=300.
112
+
113
+
## verdict
114
+
115
+
the prototype's proofs read like a de-risking document for weir's roadmap:
116
+
the order of work he discovered the hard way (batch commits → pushdown →
117
+
shared cache → stats pruning → snapshot compaction) is the order weir
118
+
should climb, and two of those rungs we already stand on because atproto
119
+
shipped them. what weir adds is the layer his stack has no vocabulary for:
120
+
the store is signed, portable, and public by construction. his
121
+
architecture asks "can object storage replace postgres?" — proven. weir
122
+
asks "if the storage layer is a protocol, who else gets to read it?" —
123
+
and that question is the product.