Clone this repository
For self-hosted knots, clone URLs may differ based on your setup.
Every handler package carried its own near-identical writeError/
handleServiceError pair. Because each mapped a different subset of errors,
whatever a package forgot fell through to a 500 — most damagingly a dead
OAuth session on post, comment, and vote, which left clients with no signal
to re-authenticate and users stuck on "an internal error occurred" until
they signed out by hand.
The fix spans three layers, because the handler switch was only the visible
half: services were discarding the PDS cause before it ever reached the
boundary, and the expired-session path never produced a PDS error at all.
Changes:
- Add pds.ErrSessionExpired and IsReauthRequired. An unresumable OAuth
session fails at client construction, before any PDS request, so it had no
status to map; factory.go now tags it. Tagged narrowly, on
ErrSessionNotFound only — a store outage or cancelled request must not
read as expired, or a brief database blip would 401 every user in flight.
- Stop services swallowing the cause: 17 sites in votes, comments, and
communities now return fmt.Errorf("%w: %w", DomainSentinel, err) so the
domain sentinel and the PDS cause both stay matchable.
- Add internal/api/xrpc.Mapper, a rule table replacing all 13 switches.
Precedence is re-auth, then domain rules, then shared rules (typed
coreerrors, remaining PDS, request lifecycle), then a fallback. Re-auth
jumps the queue so a domain sentinel mapping to 403 cannot mask a 401.
- Strip pds sentinels from post delete: posts live in the community's repo,
so a rejected community credential must not tell the caller to sign in.
- Replace all 50 err == sentinel comparisons with errors.Is.
- Classify identity resolver errors in ResolveHandleToDID, so an unknown
handle is 404 and a resolver outage is 500 rather than both being 404.
- Delete the duplicate writers: handlers.WriteError, writeJSONError,
writeUpdateProfileError, and three inline switches in update_profile.go.
- Fix along the way: comments 409 on concurrent edit (was 500), community
429/413 (was 500), update_profile 400 (was 500), and routes/user.go
reporting a resolver outage as 404 ProfileNotFound.
- Add 30 tests: mapper precedence, per-package code contracts, session
classification, and handle resolution.
Error codes are a client contract and are preserved, with one deliberate
exception: a wrapped communities.ErrHandleTaken now answers NameTaken
instead of AlreadyExists. The old `err ==` never matched because the service
wraps it, so the create path always returned AlreadyExists; errors.Is now
matches and yields the code the switch plainly intended. Clients keying on
AlreadyExists for a handle collision need updating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the p1 resilience gap in the backlog
(2026-07-22-no-db-pool-limits-no-http-timeouts).
The HTTP server was constructed with only Addr and Handler, so all four
net/http timeouts were zero — which means "no deadline". A client that
opened a connection and dribbled request headers held a goroutine and a
file descriptor indefinitely; enough of them exhaust the process without a
single complete request ever arriving. Separately, sql.Open was called with
no pool configuration at all, and database/sql defaults MaxOpenConns to
unlimited, so a traffic spike could open connections until PostgreSQL's
max_connections was exhausted — locking out psql and the cmd/ maintenance
tools along with the AppView.
Both are now bounded and env-tunable. Query time is bounded server-side via
statement_timeout injected into the DSN rather than per-query context
deadlines: lib/pq does send a CancelRequest on context cancellation, but
that path needs a second connection, races the query finishing, and does
nothing if the client dies outright. Schema migrations deliberately run on
a separate connection with statement_timeout stripped, since a CREATE INDEX
killed halfway is worse than a slow one.
main() was 1090 lines with 30 inline os.Getenv calls and no config struct.
Configuration now lives in internal/config, which applies defaults, rejects
malformed values, and enforces the requirements that differ between dev and
production — reporting every problem at once so a misconfigured deployment
is fixed in one pass instead of one restart per mistake. main() is now ~240
lines and returns an error rather than calling log.Fatal, which is what
makes the deferred cleanup reachable: os.Exit skips defers, so a fatal call
partway through startup abandoned the database pool and discarded the
buffered OpenTelemetry spans describing the failure.
Changes:
- Set ReadHeaderTimeout, ReadTimeout, WriteTimeout, IdleTimeout (cmd/server/httpserver.go)
- Configure the pool: MaxOpenConns, MaxIdleConns, ConnMaxLifetime, ConnMaxIdleTime (cmd/server/database.go)
- Inject statement_timeout into the app DSN; strip it for migrations (internal/config/dsn.go)
- Add internal/config with Load/Validate and per-subsystem config structs
- Split cmd/server into main, wiring, routes, consumers, database, httpserver, jobs, health, pds
- Embed goose migrations (internal/db/migrations/embed.go), removing the
working-directory dependency and the Dockerfile's migrations COPY
- Drain background work concurrently with the listener rather than after it,
so a slow in-flight request cannot consume the whole shutdown budget and
leave Jetstream cursors unflushed; drain on the listener-failure path too,
and report shutdown failures through the exit code
- Bound each background job cycle and recover per cycle rather than per
goroutine, so neither a panic nor a hang can silently kill the job
permanently; run a cycle at startup instead of waiting out the first tick
- Resolve the OAuth session store once at boot and fail loudly if absent,
rather than turning the cleanup job into a silent hourly no-op
- Add a context timeout to the instance PDS login, which used
http.DefaultClient and could hang the boot forever
- Document the new HTTP_* and DB_* variables in the env examples
Production now fails closed on OAUTH_SEAL_SECRET (base64, 32 bytes),
CURSOR_SECRET (rejects the documented CHANGE_ME placeholder), JETSTREAM_FEEDS,
APPVIEW_PUBLIC_URL and PDS_URL (must not be loopback), a non-DID INSTANCE_DID,
SKIP_DID_WEB_VERIFICATION, and a zero value for any HTTP timeout. The current
docker-compose.prod.yml and .env.prod.example satisfy all of these; a live
.env.prod with a short CURSOR_SECRET or a malformed OAUTH_SEAL_SECRET will
refuse to boot.
Verified: make test-all green across all three stages, make fmt-check clean,
go vet clean, race detector clean on the changed packages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
internal/core/errors was dead code — zero importers — while six domain
packages each reimplemented the same ValidationError with a slightly
different message format. Because those were distinct Go types with
identical shapes, a handler could not ask "is this a validation failure?"
in one place: it had to call posts.IsValidationError, then
communities.IsValidationError, then aggregators.IsValidationError, and any
domain it forgot fell through to a 500.
The six packages now alias the shared type rather than redefining it:
type ValidationError = coreerrors.ValidationError
A Go type alias is the *same* type, not a similar one, so a single
errors.As at the API boundary matches validation failures from every
aliasing domain while each package keeps its own constructors and
predicates. Existing call sites compile unchanged.
Two latent bugs surfaced and are fixed:
- timeline.IsValidationError and discover.IsValidationError used a bare
type assertion rather than errors.As, so a validation error wrapped
anywhere in the stack with %w stopped being recognised and returned 500
instead of 400.
- posts.IsNotFound compared with == rather than errors.Is, which breaks
the moment any layer adds context.
Changes:
- Rewrite internal/core/errors as the canonical ValidationError,
NotFoundError, and ConflictError, with Is methods bridging the latter
two to shared sentinels
- Alias the shared types in posts, communities, aggregators,
communityFeeds, discover, and timeline
- Switch timeline/discover predicates to errors.As so they unwrap
- Switch posts.IsNotFound to errors.Is
- Replace a hand-rolled contains/anySubstring reimplementation of
strings.Contains in posts with the stdlib
- Add errors_test.go asserting the cross-domain property the whole change
exists for: reverting any alias to a local struct still compiles and
still passes each domain's own tests, so only this test catches it
Note for clients: ValidationError.Error() is now uniformly "field:
message". Previously posts returned "validation error (field): msg",
aggregators "validation error: field - msg", and timeline/discover the
bare message with no field prefix. These strings reach clients as the
XRPC error *message*; the error *code* is unchanged, so the lexicon
contract holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make fmt-check, and therefore make lint, was already failing on main across
22 files before this branch of work. This applies `make fmt` and clears it.
Whitespace only: alignment of struct tags and trailing comments, one
over-long single-line function body split in list_test.go, and trailing
blank lines stripped at EOF. Verified by confirming every file here is
byte-identical to gofmt applied to its parent version, so the change is
semantics-preserving by construction. go build and go vet are clean and
make test-all still passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An empty PLCURL silently meant the production plc.directory, so a config
that merely omitted the field resolved identities against production. The
same hazard sat in identity.DefaultConfig(), which hardcoded production and
was used unmodified by ~30 integration test sites - so test runs issued real
lookups upstream for DIDs that only exist on the local PLC.
Nothing here could ever WRITE to a directory: DID registration is a PDS
operation and the dev PDS is pinned to the local PLC container, so no
production records were at risk. The silent default was still a footgun, and
an empty field should never be the way you select production.
Changes:
- NewOAuthClient errors when PLCURL is empty instead of falling through to
indigo's default directory; the directory override is now unconditional
and the startup warning became a structured log line
- identity.DefaultConfig honors PLC_DIRECTORY_URL, which the Makefile
already exports from .env.dev, pointing every test resolver at the local
PLC without touching ~30 call sites
- OAuth test helpers name their directory explicitly; unit tests use an
unroutable address so an unintended resolution fails loudly rather than
quietly reaching production
Also refreshes four tests that pinned superseded behavior:
- TestValidateActorProfile: actor.profile has had no required fields since
81ecb15, so "missing required field" is no longer a failure mode; now
asserts the constraints the schema does enforce
- TestGetCommunityFeed_BlobURLTransformation: expects embed.external#view
per the postView union, since a thumb rewritten to a URL no longer fits
the record schema
- TestOAuth_SessionFixationAttackPrevention: 62f9066 replaced raw-400
dead-ends with a first-party 302, so "not 302" asserted the mechanism
rather than the property; now asserts the redirect is first-party and
carries no session material
- TestUserCreationAndRetrieval/Resolve Handle to DID: resolves the real
bretton.dev, so it gets its own production-pinned READ-ONLY resolver
make test-all passes: 30 packages across unit, integration and E2E.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The RSS bridges wrote raw non-percent-encoded feed URLs into
embed.external.sources[].uri and richtext facet#link.uri. A literal "é" in a
slug resolves fine in a browser but fails the atproto `uri` format (printable
ASCII only after the scheme), so ~12% of live posts are invalid for any
third-party tool that resolves our lexicons and validates the firehose.
Fixed at both ends: the bridges encode at the source, and the AppView
normalizes on the write path. The AppView signs community post records into
the PDS itself, so it is the only layer that can guarantee a conforming record
regardless of client. Recoverable input is repaired rather than rejected, so
pasting a link containing an accented character keeps working; only genuinely
unusable input errors.
Encoding splits the URI with plain string operations instead of net/url.
url.Parse rejects several trivially recoverable inputs (a stray "%", an
interior tab, non-ASCII userinfo, an at:// URI whose DID reads as a port), and
round-tripping through url.URL.String() decodes reserved characters — turning
%2F into a path separator and silently repointing the link at a different
resource.
Changes:
- add internal/validation/uri.go: NormalizeURI/ValidURI with a typed sentinel
per failure cause
- add uri_sanitizer.py to both bridges, wired into create_external_embed and
RichTextBuilder.add_link
- pin cross-language agreement with a shared 46-case corpus at
internal/validation/testdata/uri_vectors.json, read by both the Go and Python
suites, so a one-sided change turns the other language's tests red
- punycode hosts via UTS-46 with DNS length verification (Go) and the IDNA2008
idna package (Python); the raw Punycode profile and the stdlib IDNA2003 codec
each produced a different domain than the user linked to
- refuse javascript:/data:/vbscript:/file:/mailto: rather than repairing them
into valid records; the check runs before the already-conforming fast path,
since those URIs are pure ASCII and would otherwise pass untouched
- enforce the lexicon's maxLength of 50 on embed.external.sources, and reject a
present-but-non-array sources value instead of silently skipping it
- keep the error chain intact: ValidationError gains Err/Unwrap and
NewValidationErrorFrom, comment wrapping uses %w:%w, so errors.Is reaches the
validation sentinels from the API boundary
- stop logging rejected URIs, which can carry signed-URL tokens; log
dropped/total source counts instead, escalating to ERROR when none survive
- declare idna as a direct dependency of both bridges
Facet link URIs are handled on the write path only. SanitizeFacets (firehose
ingest) is deliberately untouched and pinned by a test: existing federated
records are not being backfilled, and dropping their link facets on reindex
would be visible content loss.
Note: a post carrying a mailto: or javascript: link facet is now rejected
outright rather than degraded to unlinked text.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Never hardcode the Cloudflare token in the tracked script — source it from
scripts/.env.lexicon (gitignored) instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
goat lex lint flagged 77 string fields with no maxLength across params,
outputs, and knownValues tokens. Cap them all (cursors 500, knownValues 64,
plus per-field caps for email/password/JWTs/api-keys/search queries).
Remaining large-string warnings are intentional (base64 image fields,
long-form content at the 10:1 grapheme ratio).
Add docs/LEXICON_PUBLISHING.md (publish/hold-back sets, decisions log,
workflow) and scripts/publish-lexicon-dns.sh (idempotent Cloudflare API
upsert of the 9 _lexicon TXT delegations; moderation deferred via flag).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prepare the social.coves.* lexicons for publication (com.atproto.lexicon.schema
records + _lexicon DNS). Once published, atproto evolution rules freeze the
schemas to additive-only changes, so this pass fixes everything breaking-shaped
now: style-guide violations from the original Opus-era schemas, and every place
a schema contradicted what the Go AppView actually reads and writes. All ~11k
live records on pds.coves.me + pds.bretton.dev verified unaffected via the new
validate-live sweep (only pre-existing bridge-URI failures remain). Reviewed by
multi-model /second-opinion; its confirmed findings are folded in here.
Schema correctness:
- Record account refs pinned to DIDs (post.community, ban.community — handles
are mutable/reassignable); live posts already store DIDs
- Removed literal "$type" properties from rules.json union members; removed
invalid additionalProperties (tagCounts -> unknown); dropped record-baked
governance defaults/minimums (policy lives app-side)
- knownValues normalized to kebab-case (moderator permissions, getPosts
filters); Go normalizes legacy snake_case filters for old clients
- 1:1 byte/grapheme caps fixed to ~10:1 across records AND procedures (embeds,
post tags, community name/description, comment content, editNote; post.update
content limits were inverted)
- Over-required fields relaxed pre-freeze (profile createdBy/hostedBy,
actor.profile createdAt, rule fields, memberView.reputation);
actor.block.createdAt now required; banView.indexedAt required
- name capped at 63 bytes everywhere to match the RFC 1035 check Go enforces
Schemas now tell the truth about the wire:
- actor.profile record/views renamed bio->description(+Facets), viewer
blockUri->blocking — matching writer, firehose consumer, and getProfile;
updateProfile input keeps `bio` (the actual wire name clients send)
- community.update input is communityDid; create/update/updateProfile declare
avatarBlob/bannerBlob as base64 strings + mime types (what handlers decode),
and update only advertises fields the handler supports
- New social.coves.community.post.defs holds the shared view types (moved out
of the post.get query file); comment/feed refs re-pointed; dangling
embed.record#view refs eliminated
- New embed external#view + post#view defs; blob_transform now rewrites $type
to the #view form when it URL-ifies thumbs or resolves quoted posts, so
served payloads validate against the type they declare
- vote.create output uri/cid optional (absent on toggle-off) instead of
required-but-empty-string; handler emits omitempty; pds CreateRecord now
errors on a 200 with empty uri/cid so {} can only mean toggle-off
- moderation ban subset cleaned (banView instead of refs to the record type,
unbanUser success boolean dropped, tribunalCase as strongRef); governance
trio (ruleProposal/tribunalVote/vote) deliberately untouched and held back
from publishing until tribunal design lands
Tooling and enforcement:
- New cmd/validate-live: sweeps every social.coves.* record on a live PDS via
public XRPC and validates against local schemas; exits 2 on incomplete
sweeps, guards non-advancing cursors, sorted deterministic output
- cmd/validate-lexicon cross-ref list updated for the new defs
- update_profile handler validates displayName/bio by graphemes+bytes (uniseg)
per the published contract; fixtures and tests updated
Client coordination required before next appview deploy (tracked in memory):
transformed embeds now declare #view $types; vote toggle-off omits uri/cid.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds base-uri 'self', object-src 'none', and frame-ancestors 'none' to
the coves.social CSP (frame-ancestors matches the existing site-wide
X-Frame-Options DENY).
Also documents the carve-out needed when the SvelteKit frontend joins
this stack: this static header REPLACES the app's per-request nonce'd
CSP, and its script-src 'unsafe-inline' would silently downgrade the
frontend's script-src hardening — flagged in the July 2026 frontend
security review.
Deploy note: single-file bind mount pins the inode — use
`docker compose up -d --force-recreate caddy`, not `caddy reload`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>