disjecta#

Route Discord messages — full content or just URLs — into atproto-shaped destinations.

A paying user signs up at the website, links their Discord account and an Atmosphere account, and writes rules like "harvest URLs from server X channel Y → my Semble bookmarks" or "mirror full content of #announcements → my PDS." A single shared Discord bot, invited into any server the user wants, evaluates everyone's rules and routes matching items to each rule's destination.

How it works#

Two halves, deliberately decoupled:

1. Discord harvesting. A single shared bot (one Discord application, operated by us) is invited by users into any server. On every messageCreate the bot sees, it queries the rules table for matches and dispatches URLs to each matching rule's destination. The bot is omniscient by virtue of being in the server; gating happens at the rule level, not at the bot level.
2. Web app. Hosts identity linking (Discord OAuth + atproto OAuth), Stripe Checkout, and the rules UI.

URLs themselves are pass-through. Disjecta stores account metadata and aggregate write counts only — never the URLs, message content, channel history, or anything else from Discord.

Bot model#

One global bot user. Anyone can invite it via the install link; processing only happens for messages that match rules belonging to active (paid) users. Two paying users in the same server route to their own destinations independently.

The bot needs the Message Content privileged intent. Below 100 servers Discord doesn't require verification; past that, we'll need to verify the application.

Auto-leave for inactive guilds. A free-rider server (high message volume, no paying customer) costs us gateway bandwidth even though no work happens downstream. Defense: the bot leaves any guild that has no active rule pointing at it after a grace period (default 7 days, configurable via GUILD_GRACE_HOURS). The reaper runs hourly. Re-invitation is fine; just configure a rule first.

The hot path is also pre-filtered in memory: messageCreate does a single Set.has() lookup against the cache of guilds-with-active-rules before any DB or content access. Messages from inactive guilds are dropped before extraction.

Rules#

A rule is one row in the rules table:

guild_id is required and must be a server the user is a member of (and where the bot is present). Filters are AND'd within a rule. NULL on channel_id / author_id = wildcard.

Multiple paying users can have rules in the same server, each routing to their own destination — the rule itself is the correlation between user and server, not a separate claim.

Common patterns:

• "everything I post in server X" → guild_id=X, author_id='self'
• "all URLs in #links of server X" → guild_id=X, channel_id=#links
• "Boris's posts in server X" → guild_id=X, author_id=<boris_discord_id>
• "everything in server X" → guild_id=X

A message can match multiple rules (same owner, different filters) → fans out to multiple destinations → counts as multiple writes.

Backfill#

A rule can replay a channel's history through the same match → extract → dispatch pipeline as live messageCreate. One row per channel in backfill_jobs; a single in-process worker (backfill.js) drains them FIFO, ~1s per 100-message page. A wildcard (server-wide) rule fans out one job per readable channel via POST /rules/:id/backfill/all.

• Idempotent. rkeys are deterministic (sha256(messageId|value)), so re-running a channel overwrites in place — no duplicates. This is what makes every retry and repair safe.
• Paced + rate-limit-aware. Target ~60 PDS writes/min (BACKFILL_WRITES_PER_MIN). The PDS limit (eurosky ~5000 writes/rolling-hour) is shared with live dispatch, so 429 backoffs are expected — the worker sleeps out ratelimit-reset. A flat scanned/written count across checks during a backoff is normal, not a hang.
• Resumable. Progress is cursor-based (oldest message processed). On restart, requeueOrphanedBackfillJobs() flips running → pending preserving the cursor, so the worker resumes mid-channel.
• Never loses a record (by construction). Dispatch retries every non-permanent error — 5xx, socket resets (UND_ERR_SOCKET), thrown calls — with backoff; only a non-429 4xx is permanent. A job that still ends with error_count > 0 dropped that many records; recover by re-running the channel (idempotent) until every channel's latest job is done + error_count 0. Don't treat "0 pending/running" as complete on its own.
• Targeted repair. scripts/repair-message.mjs <rule_id> <channel_id> <message_id> re-dispatches one message (gateway-free REST fetch — no second bot login) instead of re-walking a whole channel.
• Permission failures (messages.fetch failed: Missing Access) are an acceptable skip, not a loss — the bot just lacks read access to that channel.

Backfill is gated to an allowlisted Discord account (BACKFILL_ALLOWED_DISCORD_ID) while the feature is shaken out.

Pricing#

Flat $2/month today while we monitor real usage. No hard cap in v1. Per-job write counts are recorded monthly in write_counters purely for observability — to inform future tier design, not to gate dispatch.

A revised model is being drafted in docs/pricing-model.md: a per-rule monthly fee plus one-time per-channel and per-guild backfill fees (prices and open decisions still TBD).

Storage#

SQLite on a Railway volume (/data/disjecta.db in prod). Tables:

• users — (discord_id, stripe_customer_id, active, …). One row per Discord account (the billing entity).
• jobs — (user_id, did, did_handle, did_avatar). One Discord account → N atproto identities (DIDs). Rules and write counters hang off a job, not the user directly.
• rules — see above.
• write_counters — (job_id, period 'YYYY-MM', count). Aggregate only.
• backfill_jobs — one row per channel backfill; status/cursor/scanned/written/error_count (see Backfill).
• oauth_state — short-lived atproto OAuth handshake state.
• oauth_session — long-lived atproto OAuth sessions (DPoP + refresh token), indexed by DID. The destination adapter restores these to write records on the user's behalf.
• webhook_events — Stripe event ids, for idempotent webhook delivery.

Pass-through is enforced by what the schema can't hold: there's no URLs table, no message log, no content column. (backfill_jobs keeps counts and a cursor, never message content.)

Project layout#

disjecta/
  src/
    index.js              entry — boots bot, web, and backfill worker in one process
    bot.js                Discord client + messageCreate handler
    web.js                Hono app — OAuth, rules UI, backfill UI, Stripe, webhooks
    db.js                 sqlite + tables + prepared statements
    backfill.js           in-process backfill worker (history replay; see Backfill)
    extract.js            URL extraction
    match.js              per-rule content matchers (substring / regex)
    active-guilds.js      cheap "any active rule in this guild?" pre-filter
    atproto-identity.js   DID / handle resolution helpers
    cookies.js            HMAC signed-cookie session
    keys.js               JOSE keyset for atproto OAuth client auth
    oauth.js              atproto OAuth client singleton (SQLite-backed stores)
    discord-oauth.js      Discord OAuth helpers
    stripe.js             Stripe SDK singleton
    secret-store.js       AES-256-GCM encrypt/decrypt for OAuth sessions at rest
    destinations/
      index.js            dispatch by destination_kind
      stdout.js           debug destination (logs JSON to stdout)
      atproto-record.js   writes a record to the rule owner's PDS
      bookmark.js         writes bookmark cards (community lexicon + Semble)
  scripts/
    user-add.js           upsert a user row (dev)
    rule-add.js           insert a rule row (dev)
    repair-message.mjs    targeted re-dispatch of one message (backfill repair)
    diag-*.mjs            backfill / job / dupe diagnostics
  data/                   sqlite file lives here (gitignored)

Local dev#

cp .env.example .env       # see Environment below
npm install
npm run keygen             # paste output into OAUTH_PRIVATE_KEY
npm start

npm start boots both the bot and the web server (Hono) in one process and creates the SQLite schema if it doesn't exist. Visit PUBLIC_URL to link Discord + Atmosphere accounts and create rules.

For a bot-only quick test (no web/auth), seed by hand:

npm run user:add -- <your_discord_id>
npm run rule:add -- <your_discord_id> --author self

Then post a URL where the bot can see it; you'll get a JSON line on stdout.

atproto OAuth caveat#

atproto OAuth rejects localhost. Full end-to-end testing of the web flow requires a public HTTPS URL — either a tunnel (cloudflared/ngrok) with PUBLIC_URL set to the tunnel host, or a Railway deploy.

Scripts#

• npm run user:add -- <discord_id> [--did <did>] [--inactive] — upsert a user
• npm run rule:add -- <discord_id> [--guild ID] [--channel ID] [--author ID|self] [--dest kind] [--config JSON] [--disabled] — add a rule
• node scripts/repair-message.mjs <rule_id> <channel_id> <message_id> — re-dispatch one message to recover a record a transient error dropped (idempotent; gateway-free)
• node scripts/diag-job-show.mjs <job_id>, diag-backfill-window.mjs, diag-cleanup-dupes.mjs — backfill/job diagnostics. On prod, run any of these via railway ssh --service disjecta node scripts/<name> ….

Inspecting the DB#

sqlite3 data/disjecta.db "SELECT * FROM rules;"
# write_counters key off job_id (a job = one DID under a user), so join through jobs:
sqlite3 data/disjecta.db "SELECT u.discord_id, j.did, w.period, w.count FROM write_counters w JOIN jobs j ON j.id = w.job_id JOIN users u ON u.id = j.user_id;"

The prod DB lives at /data/disjecta.db inside the Railway container — query it read-only with:

railway ssh --service disjecta node -e "'import(\"./src/db.js\").then(m => console.log(JSON.stringify(m.db.prepare(\"SELECT status, COUNT(*) n FROM backfill_jobs GROUP BY status\").all())))'"

(Wrap the JS in literal single quotes inside outer double quotes; the remote runs sh -c, so parens must stay quoted.)

Environment#

Discord setup#

1. https://discord.com/developers/applications → New Application.
2. Bot tab → reveal token → set DISCORD_TOKEN.
3. Bot tab → enable Message Content Intent (privileged).
4. Installation tab:
   • Authorization Methods: Guild Install (User Install optional).
   • Install Link: Discord Provided Link.
   • Default Install Settings → Guild Install:
     • Scopes: bot, applications.commands
     • Permissions: View Channels, Read Message History, Send Messages, Use Slash Commands
5. The Install Link at the top of the Installation tab is the invite URL.
6. OAuth2 tab → Redirects: add ${PUBLIC_URL}/discord/callback. Reset the client secret if you haven't already and copy it into DISCORD_CLIENT_SECRET.

Stripe setup#

1. Create a Product with a recurring Price; copy the price_… id into STRIPE_PRICE_ID. The Subscribe button label is rendered from the resolved price, not the source — change the price in Stripe and the button updates on next restart.
2. Enable the Customer Portal in the Stripe dashboard (Settings → Customer Portal). Allow subscription cancellation.
3. Add a webhook endpoint pointing at ${PUBLIC_URL}/webhooks/stripe. Pick any event to subscribe to initially (Stripe requires at least one) — disjecta auto-extends enabled_events at startup to match EVENTS_HANDLED in src/stripe.js, so adding a new event case in code is the only step needed going forward.
4. Copy the webhook signing secret into STRIPE_WEBHOOK_SECRET.

Billing state model#

• BILLING_DISABLED=1 (current test period): identity-complete users are auto-activated, no Stripe call required.
• BILLING_DISABLED=0 + GRACE_UNTIL=<ISO date>: existing test-period users keep firing rules until that date; new signups must subscribe.
• BILLING_DISABLED=0 + no GRACE_UNTIL: only users with an active Stripe subscription fire rules.
• Unsubscribing (or customer.deleted) flips the user inactive and disables every rule they own. Re-subscribing flips active back on but rules stay disabled — the user re-enables them deliberately in the UI.
• Webhook deliveries are idempotent (de-duped by event.id in the webhook_events table).

Testing webhooks locally#

The atproto OAuth flow doesn't work against loopback (so most of disjecta can't be exercised locally), but the Stripe webhook path can:

1. Install the Stripe CLI and stripe login.

2. Forward your test webhook to the local server:

   stripe listen --forward-to localhost:3000/webhooks/stripe
   

3. Copy the whsec_… secret it prints into STRIPE_WEBHOOK_SECRET for your local env.

4. Set STRIPE_SECRET_KEY to a test-mode key (sk_test_…) and STRIPE_PRICE_ID to a test-mode price. Boot disjecta.

5. Trigger events: stripe trigger checkout.session.completed, stripe trigger customer.subscription.deleted, etc. The handler logs success / signature failures; the webhook_events table fills up so retries become no-ops.

Deploy#

Railway (project/service disjecta). SQLite file lives on a mounted volume at /data/disjecta.db (set DB_PATH=/data/disjecta.db).

• Ship code: railway up --service disjecta builds from the working tree and swaps in a new container. (railway redeploy only restarts the current image — it will not include new commits.)
• Restart / unstick the worker: railway redeploy --service disjecta --yes. Graceful shutdown force-exits after 1s; on boot the backfill worker requeues any orphaned running job from its saved cursor (idempotent, no data loss).

Status#

• Bot — multi-server, server-agnostic, rule-driven dispatch
• Rules engine + pass-through harvesting
• Write counters
• stdout destination
• Web — Discord OAuth + atproto OAuth + identity link
• Web — rules CRUD UI with guild-membership guard
• Stripe Checkout + Customer Portal + webhooks → active flag
• Persistent OAuth sessions (SQLite-backed)
• atproto-record destination — writes to user's PDS under a configured NSID
• OAuth state/sessions encrypted at rest (AES-256-GCM, key derived from SESSION_SECRET)
• Bot slash commands /disjecta connect and /disjecta status
• Atmosphere unlink button on landing page
• Usage panel on rules page (this-month + recent-months write counts)
• Multi-user per server — multiple paying users can have rules in the same server, each routed to their own destination
• bookmark destination — community bookmark lexicon + Semble cards (network.cosmik.card + collectionLink)
• Backfill — channel-history replay with pacing, 429 backoff, transient-error retry, and idempotent repair
• One Discord account → N atproto identities (jobs table)
• Deploy at disjecta.portable.agency