atproto upvotes and link aggregation
atproto lexicon upvote
1

Configure Feed

Select the types of activity you want to include in your feed.

TypeScript 73.2%
Svelte 22.3%
Lua 2.9%
HTML 1.5%
JavaScript 0.2%
Other 0.1%
58 1 0

Clone this repository

https://git.vm.fail/bnb.im/upvote.at https://git.vm.fail/did:plc:kbbsajlbkvuayxnt2q6vlxoa
ssh://git@knot1.tangled.sh:2222/bnb.im/upvote.at ssh://git@knot1.tangled.sh:2222/did:plc:kbbsajlbkvuayxnt2q6vlxoa

For self-hosted knots, clone URLs may differ based on your setup.


README.md

upvote.at#

Atmospheric Upvotes

Approach#

Data lives in the atmosphere#

Ideally, this app is entirely a client that displays upvotes in the Atmosphere.

Link posting is irrelevant and doesn't need it's own lexicon because the upvote records serve as a "post" of the link. This gives us a number of things for free:

  • Upvotes are universal. you don't need to upvote on upvote.at for upvote.at to surface users' engagement. If an upvote exists publicly in the Atmosphere, we get it.
  • Your domain is your aura. Rather than the user getting credit for whatever's being posted, the origin of the link being upvoted gets the credit. This shifts the focus to content rather than to karma farming by racing to be the first to post something.
  • History is persistent. If a link gets 50,000 upvotes 20 years ago and then it gets shared again, all the history of engagement with that link will persist and we can fine-tune how that's surfaced through the front-end with different views of the data.

The site currently implements this by:

  • Writing an at.upvote.signal record to a user's PDS
  • Every URL has one submission
    • This is a fundamentally different approach to other link aggregators
    • It is our responsibility to figure out how to surface new upvotes and decay old upvotes in a way that is respectful to new interest while still persisting the historical engagement
  • A link's upvotes are one single number, the "submission" is simply the earliest upvote.

Some Useful Technical Details#

Here's a ton of technical details that have been written out as I've developed the app. There's some duplication, mostly just trying to get the information down. I'll make a pass at cleaning it up later.

  • This app relies on HappyView as its AppView.
    • We use the HAPPYVIEW_KEY env var to connect, and HAPPYVIEW_URL to pick the instance (defaults to https://happyview.upvote.at).
  • The /settings page's "authorize the bot" link points at upbot via the UPBOT_URL env var (src/lib/server/upbot.ts), unset by default so the link only shows where the bot is actually deployed. upbot has its own PUBLIC_URL this needs to match; see that repo's README.
  • Identity is entirely client-side (src/lib/auth.svelte.ts, via @happyview/oauth-client-browser)
    • The browser provisions a PKCE-bound DPoP keypair from HappyView, runs the standard atproto OAuth flow with your own PDS using that key, and registers the session back; tokens and keys live in browser storage.
  • The feed is built by aggregating signals in src/lib/server/signals.ts
  • There are three feeds, ranking happens in the browser (src/lib/feeds.ts) and the routes share one page via the optional [[feed=feed]] param (matcher in src/params/feed.ts). The feeds are:
    • hot (/, points decayed by submission age),
    • new (/new, newest submission first),
    • top (/top, most points ever, with ties decided by recency)
  • Subjects describe themselves, and display metadata is resolved at index load (src/lib/server/subjects.ts) then cached in memory: derived, never stored, so nothing is lost on restart.
    • at-uri subjects resolve in batched reads through HappyView's published at.upvote.getRecords query (index-first on the instance, fetching through to the repo's resolved authority on a miss; this server never discovers or contacts a PDS)
  • Embed rendering
    • a web-URL subject is fetched with SSRF guards and titled from its OpenGraph tags (og:title, then twitter:title, then <title>), with og:description/og:image rendered when the page offers them
    • Rich embeds render natively from records:
    • A app.bsky.feed.post record subject shows the post itself (text, images, video, external link).
    • A standard.site document subject renders as an article, not a quoted box.
      • No inner frame: a publication kicker (icon, name, reading time, a per-card cover toggle, and a small hostname ↗ that is the card's only link out), the title in the site's own type, and the author byline.
      • Clicking the card opens the in-place reader; linking out is deliberately secondary.
      • Content is normalized server-side (src/lib/server/docContent.ts) from the platform unions the network actually publishes (blog.pckt.content, pub.leaflet.content) plus the spec's plaintext textContent fallback, into blocks the reader renders (headings, styled text with facets, quotes, lists, code, images, rules); the browser fetches blocks lazily via /api/doc/[uri] so feed payloads stay lean.
  • Profiles (record + handle) arrive batched via the published at.upvote.getProfiles query.
  • HappyView handles writes of the at.upvote.signal record into your repo.
  • Upvote toggles call the published at.upvote.toggleSignal procedure on HappyView with a DPoP proof.
  • The server's only jobs are:
    • rendering the public view.
    • keeping a small cache of HappyView's index, read through the published at.upvote.getSignals query:
      • a TTL-gated full pull of everyone's signals when rendering (so the feed reflects the network, not just this process's visitors),
      • a per-account re-pull after writes (POST /api/index/refresh)
    • resolving display metadata (subjects, profiles, verifications) from the Atmosphere on demand, cached in memory
    • holding the materialized verifier index (every trusted verifier's app.bsky.graph.verification records, via at.upvote.getVerifierRecords) so the record path of any account's badges is an in-memory lookup; warmed at boot, refreshed behind renders
    • serving the small JSON routes the browser reads the display metadata through (/api/signals/[did], /api/verification/[did], the batched /api/verification the following/followers modal's badges use, and /api/subjects for submit-box normalization)
  • Submit-box normalization converges equivalent inputs onto one subject:
    • at:// URIs pass through (handle authorities resolved to DIDs)
    • Discovery Tags
      • A web page that is identifiably an app.bsky.feed.post record (currently only Blacksky) is normalized and rendered prettily
      • a web page carrying the standard.site discovery tag (<link rel="site.standard.document" href="at://…">) is normalized and rendered prettily
      • Discovery tags are controlled by the submitted page, so both are verified before they're trusted:
        • the standard.site claim's record (with its publication) must rebuild the submitted URL.
        • the post claim's own /profile/<actor>/post/<rkey> path must resolve to the claimed record's author and rkey
        • otherwise any page could wear another author's byline.
    • Any other http(s) URL is its own subject, fragment stripped.
    • The Bluesky-post detection also runs at display time (a web-URL subject's page is fetched for its OpenGraph tags anyway), so an at.upvote.signal record whose subject is a raw post URL (written by another client, or by hand) still renders as a native post embed, since the same-post convergence otherwise only happens for links submitted through this app.
  • In dev, browse at http://127.0.0.1:5173; the server canonicalizes localhost onto the IP literal because browser storage is origin-scoped and the OAuth loopback redirect always returns to 127.0.0.1.

Longer Notes on Technical Details, Challenges, and Design Decisions#

Ideally, I'd love to normalize on one detection method for app.bsky.feed.post records, but we don't seem to be there yet. Check in on https://discourse.atprotocol.community/t/mapping-web-pages-to-canonical-at-records/904 to see if there's a better way to do this in the future.

The public Bluesky AppView remains for what only it can answer: its computed verification view on getProfiles, plus the follow/follower count aggregates and (for the badge endpoint's lean path) a current handle without a display-data walk. That is a deliberate, permanent decision, not a pending migration. Bluesky's trusted-verifier set isn't published as records, so "verified via Bluesky" can only be recomputed from a curated roster that drifts: a verifier Bluesky removes would keep counting here until the roster refreshed, showing badges Bluesky has revoked. The appview read is always current with Bluesky's real trusted set and validity rules (which also invalidate on displayName changes, not just handle changes), batched 25 accounts per call. Blacksky and Eurosky verifications resolve entirely through HappyView's record path: the server materializes every trusted verifier's whole record output (at.upvote.getVerifierRecords, refreshed on a clock, warmed at boot) into one in-memory subject-keyed index, falling back to the per-subject at.upvote.getVerifications walk only while an instance predates the by-verifier query. Bluesky's org is the only verifier platform we implement through a computed view.

Blobs are deliberately coming from the public Bluesky CDN: display-only content the app never reasons about, loaded browser-side as <img src>, resolvable for any repo's blobs by did+cid (BLOB_URL_TEMPLATE swaps the image host in one move; the local dev environment uses it to serve blobs from the dev PDS).

At present the HappyView API client is configured to run on localhost and to get it in prod we'll need to set up an API client for production.

Lexicons#

Lexicons can be found in Lexicons. Lexicons are critical to the system and define how it works. Currently, there are twelve:

  • signal.json: an upvote is stored as a signal. Upvotes apply to any URI: atmosphere at-uris, or plain http(s) links from the open web.
  • toggleSignal.json: an upvote can be toggled on or off. Served by HappyView as a procedure; the site's upvote button calls it.
  • getSignals.json: query for signals. Lists indexed signals newest first, optionally filtered by account. Served by HappyView (via a small Lua script) over the at.upvote.signal collection.
  • resolveDid.json: query resolving a DID to its current, bidirectionally verified handle. Served by HappyView (via a small Lua script) from the identity layer, so clients need no identity infrastructure of their own.
  • getVerifications.json: query for the app.bsky.graph.verification records indexed from the network that name an account as their subject. Deliberately policy-free; it reports what exists, and callers apply their own trust in verifiers and their own validity rules (the app checks the pinned handle against the account's current one).
  • getVerifierRecords.json: the by-verifier twin of getVerifications: every indexed verification record a VERIFIER has issued. Lets a consumer with a small trusted-verifier set materialize the entire record path in a handful of requests (which is exactly what this server does) instead of asking once per subject. Same policy-free stance.
  • getListMembers.json: query for the app.bsky.graph.listitem records indexed from the network that belong to a list; this is how the app resolves an org's published trusted-verifier list.
  • getProfiles.json: batched display data: each DID's app.bsky.actor.profile record (index-first, fetching through to the account's repo on a miss) plus its current, bidirectionally verified handle, up to 25 accounts per round trip.
  • getRecords.json: batched record reads by at-uri (index-first, fetch-through on miss), up to 25 per round trip; this is how subject records (Bluesky posts, standard.site documents and their publications) resolve.
  • toggleFollow.json: a follow (app.bsky.graph.follow) can be toggled on or off, the same way an upvote can. Served by HappyView as a procedure; the profile page's follow button calls it.
  • settings.json: a viewer's device preferences (theme, feed default, fonts, thin mode, and so on), stored as a singleton record (rkey always self) in their own repo so they follow the account across browsers and devices.
  • putSettings.json: upserts the caller's at.upvote.settings record. Served by HappyView as a procedure; the settings sync module calls it on every change.

Copies of the Lua scripts our HappyView instance runs are versioned in happyview/. This includes:

These are mirrors of the live scripts running in HappyView and are documented here for reference and reuse by others.

Some HappyView design notes:

  • Deliberately no record.create/update/delete triggers for app.bsky.feed.post, site.standard.document, or site.standard.publication. A firehose trigger for a collection indexes every record of that type from every repo on the network; it has no concept of "only the ones upvote.at cares about." An earlier version of this instance registered triggers for the standard.site collections as a caching optimization and ended up backfilling the entire network's documents and publications, forcing a full HappyView reset. getRecords.json's index-first-then-fetch-through path (see xrpc.query:at.upvote.getRecords.lua) already resolves any at-uri correctly with no trigger at all; a cache miss just costs a live PDS round trip instead of a local index hit. app.bsky.actor.profile follows the same fetch-through-only pattern (see xrpc.query:at.upvote.getProfiles.lua) and has never had a trigger. Do not register collection-wide triggers for any large, third-party, general-purpose collection; at.upvote.signal is different: it's this app's own narrow, purpose-built collection, and indexing all of it network-wide is the actual point. app.bsky.graph.verification and app.bsky.graph.listitem (behind getVerifications/getVerifierRecords/getListMembers) are also tracked collections, but that's a different, necessary case: those queries have no fetch-through fallback (there's no way to know which repo to ask for "who verified this account" without an index), and both collections are small and curated rather than open to the whole network.

  • "No trigger" only means "not backfilled"; a miss-path fetch-through can still grow the index unbounded if it's implemented on top of HappyView's own indexed dispatcher rather than a direct PDS read. getProfiles.lua originally fetched-through misses via xrpc.query("com.atproto.repo.getRecord", ...) - that call reaches HappyView's own record dispatcher, not the repo directly, and everything it fetches through is a plausible candidate for HappyView to persist as a side effect (the same generic-indexing behavior xrpc.procedure:at.upvote.toggleFollow.lua separately flags as unverified for app.bsky.graph.follow). Since this path runs once per DID for every voter and author rendered anywhere on the site, over the site's lifetime that's effectively every account upvote.at has ever displayed to anyone, a strong candidate for "why does this instance have so many profiles indexed." getProfiles.lua now reads misses straight from the repo's own PDS instead (the same fix getRecords.lua already applied for the identical reason), which needs no indexing at all. If storage is still growing after this fix ships to the instance, the generic-indexing suspicion in toggleFollow.lua is the next thing to verify.

TODOs#

Supporting Projects#

  • Another medium-shot project I'd like to create is a number of ways to upvote without going to the website:
    • Chrome extension that you can click and it will upvote the link you're on.
    • A bookmarklet that you can click and it will upvote the link you're on.
    • An iOS shortcut that you can share to and it will upvote the link you shared.
  • Implement labeler processing for the feed:
    • Labelers that apply to the default feed
    • Leverage labelers within the built-in feeds (hot / new / top)
    • Let users create custom feeds with custom labeler chocies

Development#

Once you've created a project and installed dependencies with npm install (or pnpm install or yarn), start a development server:

# install dependencies
npm install

# run the development environment
npm run dev

# or start the server and open the app in a new browser tab
npm run dev -- --open

You'll want to set up a .env file with the following variables:

HAPPYVIEW_KEY=<your HappyView API key>

# Optional: where upbot (the separate bot repo that upvotes through mention
# replies) is deployed. Powers the "authorize the bot" link on /settings;
# leave unset and that link just doesn't show.
UPBOT_URL=<upbot's PUBLIC_URL, e.g. https://upbot.upvote.at>

Local AT Protocol environment#

npm run dev runs against the real network (HappyView, plus the public Bluesky AppView for its verification view). To develop against a fully local network instead, start the AT Protocol dev environment (scripts/atproto-dev-env.ts):

npm run dev:atproto

It boots a real PLC directory and a real PDS via @atproto/dev-env, creates fixture accounts holding real records (profiles, verifications, signals, and a standard.site publication + document with a real cover blob), and serves local stand-ins for every service the app consumes; HappyView's published at.upvote.getSignals/at.upvote.getVerifications/at.upvote.getVerifierRecords/ at.upvote.getListMembers/at.upvote.getProfiles/at.upvote.getRecords queries indexed live from the PDS, its on-demand com.atproto.repo.getRecord/com.atproto.identity.resolveHandle proxying, the published at.upvote.resolveDid query, blob bytes (the BLOB_URL_TEMPLATE target), the appview's verification-carrying profile read, and the fixture subjects as real web pages with OpenGraph titles. Point a dev server at it with the same env the E2E suite uses (see the webServer block in playwright.config.ts):

ATPROTO_DEV_ENV=1 HAPPYVIEW_URL=http://127.0.0.1:2581 APPVIEW_URL=http://127.0.0.1:2581 \
BLOB_URL_TEMPLATE='http://127.0.0.1:2581/blob?did={did}&cid={cid}' npm run dev

There's probably some things I've not documented yet. Either feel free to open an issue, submit a PR, or ping me on Bluesky at @bnb.im.

Build#

To create a production version of your app:

npm run build

You can preview the production build with npm run preview.

The build targets adapter-node deliberately: the app's caches are in-memory and warmed once at boot (src/hooks.server.ts), and concurrent feed builds coalesce, all of which assumes one long-lived server process. Deploy the app as a Node.js server (node build); a serverless target would cold-start every request and silently defeat the entire caching architecture.

Testing#

There are two suites:

  • Unit tests leverage Vitest for the domain modules in src/lib/server/ (signals, subjects, identity, verification) and the browser modules (utils, graph, feeds, upvotes, follow, followPreload, the client half of verification, and friends; every *.spec.ts under src/). Fast, no network, no browser; outbound calls (the AppView pulls through at.upvote.getSignals in signals.spec.ts, record/handle resolution in subjects.spec.ts, profile lookups in identity.spec.ts) are mocked.
  • E2E tests use Playwright and exist in e2e/, covering the logged-out surface of every route end-to-end plus a small visual regression suite, all against the local AT Protocol environment above, so every rendered handle, title, and badge traces back to a real record in the dev PDS.
# unit tests
npm run test:unit        # watch mode
npm run test:unit -- --run   # single run

# e2e tests; first time only, download a browser:
npx playwright install chromium

npm run test:e2e         # headless run (auto-starts the atproto dev env and the dev server)
npm run test:e2e:ui      # Playwright's interactive UI, great for debugging

# everything: unit (single run) then e2e; there's no CI yet, so run this before pushing
npm test

A few things worth knowing before adding to the E2E suite:

  • The suite's data lives in the dev environment's PDS as real records; the app's caches (src/lib/server/{signals,subjects,identity,verification}.ts) are in-memory Maps shared by every request the dev server handles. Tests that need a known starting point call resetDatabase() from e2e/helpers.ts, which hits a dev-only POST /api/test/reset endpoint: the dev environment reconciles the PDS records to the fixture (or deletes them all), and the caches are cleared so renders re-read everything through the production code paths. The route 404s in a production build, and refuses to run on a server not wired to the dev environment (ATPROTO_DEV_ENV=1), a dev server pointed at the real network can never be "reset". Because of the shared state, playwright.config.ts pins workers: 1.
  • E2E tests never touch the internet; every service the app consumes is served locally from the dev PDS, which keeps the suite fast, deterministic, and CI-friendly while still exercising the real resolution pipelines (index hydration, on-demand record fetches through the HappyView-shaped proxy, blob serving, OpenGraph titles, verification backlinks and handle-match validation).
  • Since login is real AT Protocol OAuth against a live PDS (there's no dev login), the suite only exercises logged-out behavior; the logged-in flows are unit-tested instead (subject normalization and metadata resolution in subjects.spec.ts, index hydration in signals.spec.ts, the browser-side toggle in upvotes.spec.ts, follows in follow.spec.ts).
  • Screenshot baselines (e2e/**/*-snapshots/) are committed, but are sensitive to the OS/fonts they were generated on. If CI starts failing visual tests that look fine locally, regenerate the baselines on the same platform CI uses (e.g. via the official Playwright Docker image) rather than just re-running --update-snapshots on your own machine:
    docker run --rm -v "$PWD":/work -w /work mcr.microsoft.com/playwright:v1.61.1-noble \
      npx playwright test --update-snapshots