weir/scripts/ at main · zzstoatzz.io/weir · Tangled

zzstoatzz.io / weir

0

This repository has no description

0

weir / scripts

7 files

at main

zzstoatzz events websocket: task runs arrive; compose + justfile cleanup 6w ago

events websocket: task runs arrive; compose + justfile cleanup prefect 3.x creates task runs locally and reports them only as events over ws /api/events/in — so weir now speaks that socket: upgrade with the 'prefect' subprotocol echoed when offered, the auth handshake ({type:auth} -> {type:auth_success}; every token succeeds, auth is an acknowledged gap), and ingest of task-run events: upsert into the appview + verbatim io.prefect.v0.event records + entity record on first sighting. flow-run events are skipped (already canonical via REST). the missing handshake reply was why flows hung at teardown — the client blocks on recv after sending auth. found + fixed a corruption regression in the zat encoding cache along the way: nodeCid cleared dirty without refreshing the cached encoding, so every CAR written after the second commit failed reload with BadBlockHash. fixed upstream (zat#mst-block-cache 3960e70) with a regression test there and a multi-commit reload test here. infra hygiene per review: compose.yml owns the appview redis and the justfile is reorganized around it (up/down/serve/demo/demo-flow/mirror /bench-*). scripts/demo-flow is a task-rich etl: 8 mapped transforms, a retry survivor (run_count=3), a tolerated failure. verified: etl_demo() = 204, 11 task runs in the appview (10 completed, 1 failed), 38 events in the canonical log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

weir: rename from prefect-pds + readme with architecture contrast a weir shapes flow, measures it, and never owns the water. the name also stops claiming pds (this is an appview + write gateway hosting a native repo) and stops squatting the prefect prefix on the repo's public face — io.prefect.v0.* stays as the lexicon namespace, which is a separate, deliberately-versioned decision. binary, package name, and env vars follow (WEIR_PORT, WEIR_DATA_DIR, WEIR_CACHE, WEIR_REDIS_HOST/PORT, WEIR_COMMIT_EVERY). new readme leads with mermaid diagrams contrasting prefect's self-hosted shape (LB -> api xN -> shared postgres+redis) against weir's canonical/derived split, and the ownership asymmetry table that is the actual thesis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

events websocket: task runs arrive; compose + justfile cleanup prefect 3.x creates task runs locally and reports them only as events over ws /api/events/in — so weir now speaks that socket: upgrade with the 'prefect' subprotocol echoed when offered, the auth handshake ({type:auth} -> {type:auth_success}; every token succeeds, auth is an acknowledged gap), and ingest of task-run events: upsert into the appview + verbatim io.prefect.v0.event records + entity record on first sighting. flow-run events are skipped (already canonical via REST). the missing handshake reply was why flows hung at teardown — the client blocks on recv after sending auth. found + fixed a corruption regression in the zat encoding cache along the way: nodeCid cleared dirty without refreshing the cached encoding, so every CAR written after the second commit failed reload with BadBlockHash. fixed upstream (zat#mst-block-cache 3960e70) with a regression test there and a multi-commit reload test here. infra hygiene per review: compose.yml owns the appview redis and the justfile is reorganized around it (up/down/serve/demo/demo-flow/mirror /bench-*). scripts/demo-flow is a task-rich etl: 8 mapped transforms, a retry survivor (run_count=3), a tolerated failure. verified: etl_demo() = 204, 11 task runs in the appview (10 completed, 1 failed), 38 events in the canonical log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

io.prefect.v0.*: versioned lexicons, transitions become events go-crazy namespace decision: everything lives under io.prefect.v0.* and problems down the road drive v1. seven lexicon files land in lexicons/ (defs with the shared #state shape, event, flow, flowRun, taskRun, logBatch, workspace). the server now writes state transitions as io.prefect.v0.event records mirroring prefect's own Event schema — name prefect.{flow-run,task-run}. {StateName}, resource labels, intended {from,to} + validated_state in the payload, TID rkeys. the state collections are gone; current state is appview-only, per docs/lexicons.md. verified live: python client runs basic_flow to Completed and the repo holds Pending->Running->Completed events with causal from-chains; zds poc reruns clean with the v0 workspace space type and oplog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

the convergence demo: prefect ui served by weir + waow.tech workspace weir now serves the prefect react ui (ui-v2 bundle, copied from the python package via just fetch-ui): static SPA serving with index.html fallback, /ui-settings, real /paginate for flows + flow/task runs (the list pages use paginate, not filter), a graph-v2 stub for the run detail page, and generic empty-state fallbacks for the ui's long tail of filter/count/history/paginate queries. scripts/space-mirror is the first background service in the architecture's own idiom: it consumes weir's canonical log (listRecords polling — the firehose consumer's placeholder) and pushes new records into a permissioned io.prefect.v0.workspace space on zds as waow.tech. restart-safe via diffing against the space's existing rkeys. verified end to end: basic_flow runs against weir; the ui's paginate shows the run COMPLETED; the mirror lands all 6 records in ats://did:plc:siv7.../io.prefect.v0.workspace/weir-demo where the Pending->Running->Completed events read back with auth and strangers get 401. just demo wires the whole thing up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

weir: rename from prefect-pds + readme with architecture contrast a weir shapes flow, measures it, and never owns the water. the name also stops claiming pds (this is an appview + write gateway hosting a native repo) and stops squatting the prefect prefix on the repo's public face — io.prefect.v0.* stays as the lexicon namespace, which is a separate, deliberately-versioned decision. binary, package name, and env vars follow (WEIR_PORT, WEIR_DATA_DIR, WEIR_CACHE, WEIR_REDIS_HOST/PORT, WEIR_COMMIT_EVERY). new readme leads with mermaid diagrams contrasting prefect's self-hosted shape (LB -> api xN -> shared postgres+redis) against weir's canonical/derived split, and the ownership asymmetry table that is the actual thesis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago

io.prefect.v0.*: versioned lexicons, transitions become events go-crazy namespace decision: everything lives under io.prefect.v0.* and problems down the road drive v1. seven lexicon files land in lexicons/ (defs with the shared #state shape, event, flow, flowRun, taskRun, logBatch, workspace). the server now writes state transitions as io.prefect.v0.event records mirroring prefect's own Event schema — name prefect.{flow-run,task-run}. {StateName}, resource labels, intended {from,to} + validated_state in the payload, TID rkeys. the state collections are gone; current state is appview-only, per docs/lexicons.md. verified live: python client runs basic_flow to Completed and the repo holds Pending->Running->Completed events with causal from-chains; zds poc reruns clean with the v0 workspace space type and oplog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

1 month ago