zzstoatzz.io
This repository has no description
initial pensieve mvp
author
zzstoatzz
date
(Jul 2, 2026, 1:20 AM -0500)
commit
ca3f21cf
ca3f21cff95ee7387f364b2e697e9de108602795
+6348
+18
.gitignore
+18
.gitignore
+66
AGENTS.md
+66
AGENTS.md
···
1
+
# Pensieve Agent Notes
2
+
3
+
Pensieve is a Cloudflare Worker app for public ATProto memory search. Work from
4
+
this repository root.
5
+
6
+
## Current Shape
7
+
8
+
- `src/worker.js` is the live app: Worker routes, indexing pipeline, artifact
9
+
extraction, frontend, and client-side behavior are all in one file.
10
+
- `src/*.zig` is an earlier/prototype extractor that walks repo CARs and emits
11
+
NDJSON. Keep it building, but do not assume it is the deployed path.
12
+
- `scripts/` contains smoke/demo helpers. They are not part of the Worker
13
+
runtime.
14
+
15
+
## Commands
16
+
17
+
```sh
18
+
npm install
19
+
node --check src/worker.js
20
+
npm run dry-run
21
+
npm run deploy
22
+
zig build test
23
+
```
24
+
25
+
Use `npm run dry-run` before deploys. For frontend changes, use Playwright or a
26
+
browser to inspect desktop and mobile layouts against the live or local Worker.
27
+
28
+
## Secrets
29
+
30
+
Never print or commit secret values. The Worker needs:
31
+
32
+
- `VOYAGE_API_KEY`
33
+
- `TURBOPUFFER_API_KEY`
34
+
35
+
Store deployed values with Cloudflare Worker secrets (`wrangler secret put`).
36
+
For local work, prefer `op run` or `.dev.vars`; `.env` and `.dev.vars` are
37
+
ignored.
38
+
39
+
## Product Constraints
40
+
41
+
- The user-facing app should feel like Pensieve, not an ATProto debugging tool.
42
+
- Avoid hardcoding app-specific collection allowlists as the main semantic
43
+
strategy. The important subproblem is extracting a useful artifact envelope
44
+
from arbitrary records.
45
+
- Native links beat PDSLS. Use PDSLS as "details" when a better destination is
46
+
available.
47
+
- The handle input uses the sibling typeahead service at
48
+
`https://typeahead.waow.tech/xrpc/app.bsky.actor.searchActorsTypeahead`.
49
+
50
+
## Indexing Notes
51
+
52
+
Actor indexes live in Turbopuffer namespaces named:
53
+
54
+
```text
55
+
${TURBOPUFFER_NAMESPACE_PREFIX}-${did with punctuation replaced}
56
+
```
57
+
58
+
`/api/search` uses existing namespaces. `/api/update` deletes and rebuilds the
59
+
actor namespace, streams progress, then searches the fresh index.
60
+
61
+
## Before Handing Back
62
+
63
+
- Run `node --check src/worker.js`.
64
+
- Run `npm run dry-run`.
65
+
- If UI changed, check at least one desktop and one mobile viewport.
66
+
- If deployed, include the Worker URL and the deployed version ID when useful.
+104
README.md
+104
README.md
···
1
+
# Pensieve
2
+
3
+
Pensieve is public memory search for ATProto repos. Give it a handle and a
4
+
half-remembered phrase; it walks that person's public repo, extracts semantic
5
+
artifacts from arbitrary record shapes, embeds them with Voyage, stores them in
6
+
Turbopuffer, and returns a small human-readable digest with links back to the
7
+
native thing when Pensieve can infer one.
8
+
9
+
Live Worker:
10
+
11
+
```text
12
+
https://pensieve.n8-3e9.workers.dev
13
+
```
14
+
15
+
## What Exists
16
+
17
+
- A Cloudflare Worker in `src/worker.js`.
18
+
- A single-page frontend rendered by the Worker.
19
+
- A Zig prototype in `src/*.zig` for CAR walking and NDJSON extraction.
20
+
- Small Python smoke/demo scripts in `scripts/`.
21
+
22
+
The production-ish MVP is the Worker. The Zig code is useful reference and a
23
+
future extraction path, but the deployed app currently uses `@atcute/repo` in
24
+
the Worker to walk repo CARs.
25
+
26
+
## How It Works
27
+
28
+
Search is two-stage:
29
+
30
+
1. `/api/search` resolves the actor, embeds the query, and searches that actor's
31
+
Turbopuffer namespace.
32
+
2. If the namespace is missing, the frontend calls `/api/update`, which streams
33
+
progress while Pensieve fetches `com.atproto.sync.getRepo`, walks the CAR,
34
+
infers artifacts, embeds up to `INDEX_LIMIT` records, writes them to
35
+
Turbopuffer, and then runs the query.
36
+
37
+
Second and later searches for the same actor are fast because they reuse the
38
+
existing Turbopuffer namespace. Re-indexing currently happens only through
39
+
`/api/update`, which deletes and rebuilds that actor namespace.
40
+
41
+
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the data flow and
42
+
[docs/OPERATIONS.md](docs/OPERATIONS.md) for deployment and secrets.
43
+
44
+
## Development
45
+
46
+
```sh
47
+
npm install
48
+
npm run dev
49
+
```
50
+
51
+
Useful checks:
52
+
53
+
```sh
54
+
node --check src/worker.js
55
+
npm run dry-run
56
+
```
57
+
58
+
Deploy:
59
+
60
+
```sh
61
+
npm run deploy
62
+
```
63
+
64
+
Zig prototype:
65
+
66
+
```sh
67
+
zig build test
68
+
zig build run -- extract --actor zzstoatzz.io --out corpus.ndjson
69
+
zig build run -- demo --actor zzstoatzz.io --media --dir /tmp/pensieve-demo
70
+
```
71
+
72
+
## Secrets
73
+
74
+
Do not commit secrets. The deployed Worker expects these Cloudflare Worker
75
+
secrets:
76
+
77
+
- `VOYAGE_API_KEY`
78
+
- `TURBOPUFFER_API_KEY`
79
+
80
+
The non-secret namespace prefix lives in `wrangler.jsonc` as
81
+
`TURBOPUFFER_NAMESPACE_PREFIX`.
82
+
83
+
Local development can use `wrangler secret put`, `.dev.vars`, or `op run` with
84
+
1Password. The local `.env` / `.dev.vars` files are intentionally ignored.
85
+
86
+
## Project Map
87
+
88
+
```text
89
+
src/worker.js Cloudflare Worker, API, indexing, frontend HTML/CSS/JS
90
+
src/*.zig Zig CAR extraction prototype
91
+
scripts/tpuf_demo.py Python smoke script for NDJSON -> Voyage -> Turbopuffer
92
+
wrangler.jsonc Cloudflare Worker config
93
+
docs/ Architecture and operations notes
94
+
```
95
+
96
+
## Contributing Notes
97
+
98
+
- Keep product copy non-jargony. Avoid surfacing "CAR", "repo", "collection",
99
+
or implementation terms in the UI unless they are hidden behind an inspection
100
+
affordance.
101
+
- Prefer deterministic artifact extraction before adding model calls.
102
+
- Keep PDSLS as a details/inspection fallback, not the primary destination when
103
+
a native artifact URL is known.
104
+
- Verify UI changes with a browser at desktop and mobile widths.
+41
build.zig
+41
build.zig
···
1
+
const std = @import("std");
2
+
3
+
pub fn build(b: *std.Build) void {
4
+
const target = b.standardTargetOptions(.{});
5
+
const optimize = b.standardOptimizeOption(.{});
6
+
7
+
const zat = b.dependency("zat", .{ .target = target, .optimize = optimize });
8
+
9
+
const pensieve_mod = b.createModule(.{
10
+
.root_source_file = b.path("src/root.zig"),
11
+
.target = target,
12
+
.optimize = optimize,
13
+
.imports = &.{
14
+
.{ .name = "zat", .module = zat.module("zat") },
15
+
},
16
+
});
17
+
18
+
const exe_mod = b.createModule(.{
19
+
.root_source_file = b.path("src/main.zig"),
20
+
.target = target,
21
+
.optimize = optimize,
22
+
.imports = &.{
23
+
.{ .name = "pensieve", .module = pensieve_mod },
24
+
.{ .name = "zat", .module = zat.module("zat") },
25
+
},
26
+
});
27
+
28
+
const exe = b.addExecutable(.{
29
+
.name = "pensieve",
30
+
.root_module = exe_mod,
31
+
});
32
+
b.installArtifact(exe);
33
+
34
+
const run = b.addRunArtifact(exe);
35
+
if (b.args) |args| run.addArgs(args);
36
+
b.step("run", "Run pensieve").dependOn(&run.step);
37
+
38
+
const tests = b.addTest(.{ .root_module = pensieve_mod });
39
+
const run_tests = b.addRunArtifact(tests);
40
+
b.step("test", "Run tests").dependOn(&run_tests.step);
41
+
}
+18
build.zig.zon
+18
build.zig.zon
···
1
+
.{
2
+
.name = .pensieve,
3
+
.version = "0.0.1",
4
+
.fingerprint = 0xd0ff236ba064c0dd,
5
+
.minimum_zig_version = "0.16.0",
6
+
.dependencies = .{
7
+
.zat = .{
8
+
.url = "https://tangled.org/zat.dev/zat/archive/v0.3.8.tar.gz",
9
+
.hash = "zat-0.3.8-5PuC7glgCgCVn18R0k6AuXiNok-jjqDRGgaBFj5JsI_N",
10
+
},
11
+
},
12
+
.paths = .{
13
+
"build.zig",
14
+
"build.zig.zon",
15
+
"src",
16
+
"README.md",
17
+
},
18
+
}
+125
docs/ARCHITECTURE.md
+125
docs/ARCHITECTURE.md
···
1
+
# Architecture
2
+
3
+
Pensieve has one deployed runtime: a Cloudflare Worker in `src/worker.js`.
4
+
5
+
```text
6
+
browser
7
+
|
8
+
| /api/actors -> typeahead.waow.tech
9
+
| /api/search -> resolve actor -> Voyage query embedding -> Turbopuffer
10
+
| /api/update -> resolve actor -> PDS getRepo -> CAR walk -> Voyage docs -> Turbopuffer
11
+
v
12
+
Cloudflare Worker
13
+
```
14
+
15
+
## Request Flow
16
+
17
+
### `/`
18
+
19
+
Returns the single-page app. The frontend is inline HTML/CSS/JS generated by
20
+
`renderIndex()`.
21
+
22
+
### `/api/actors`
23
+
24
+
Proxies handle search to the sibling typeahead service:
25
+
26
+
```text
27
+
https://typeahead.waow.tech/xrpc/app.bsky.actor.searchActorsTypeahead
28
+
```
29
+
30
+
This keeps handle selection consistent with the user's other ATProto tools.
31
+
32
+
### `/api/search`
33
+
34
+
1. Normalizes the actor handle.
35
+
2. Resolves it to DID + PDS with `@atcute/identity-resolver`.
36
+
3. Embeds the query with Voyage.
37
+
4. Queries the actor's Turbopuffer namespace twice:
38
+
- vector ANN over `vector`
39
+
- BM25 over `text`
40
+
5. Fuses ranked rows, dedupes equivalent artifacts, and returns display-ready
41
+
results.
42
+
43
+
If both Turbopuffer queries return 404, the response sets `needsIndex: true`.
44
+
The frontend then calls `/api/update`.
45
+
46
+
### `/api/update`
47
+
48
+
Streams newline-delimited JSON progress events.
49
+
50
+
1. Resolves the actor.
51
+
2. Deletes the actor's Turbopuffer namespace when `replace: true`.
52
+
3. Fetches the actor repo through `com.atproto.sync.getRepo` from the actor's
53
+
PDS.
54
+
4. Walks the CAR with `@atcute/repo`.
55
+
5. Infers an artifact envelope from each record.
56
+
6. Embeds up to `INDEX_LIMIT` text-bearing artifacts with Voyage.
57
+
7. Upserts rows into Turbopuffer.
58
+
8. Runs the query against the rebuilt namespace and emits a final `done` event.
59
+
60
+
## Artifact Envelope
61
+
62
+
Pensieve indexes arbitrary records by trying to infer a generic artifact:
63
+
64
+
```js
65
+
{
66
+
kind,
67
+
title,
68
+
body,
69
+
url,
70
+
media,
71
+
refs,
72
+
date,
73
+
confidence
74
+
}
75
+
```
76
+
77
+
This is deliberately not a curated collection allowlist. The extraction pass
78
+
walks the record, drops obvious protocol noise, scores likely title/body/url/date
79
+
fields, and stores both the searchable text and the display attributes in
80
+
Turbopuffer.
81
+
82
+
The UI then renders a small digest from the envelope:
83
+
84
+
- platform/app badge and name
85
+
- artifact kind
86
+
- date and duplicate count
87
+
- title/body
88
+
- primary native URL when known
89
+
- `details` link to PDSLS for raw inspection
90
+
91
+
## Link Hierarchy
92
+
93
+
Primary clicks should take the user to the most useful human destination:
94
+
95
+
1. `artifact.url` when the record contains one.
96
+
2. Known native URL builders, currently Bluesky posts and some Tangled records.
97
+
3. PDSLS as fallback.
98
+
99
+
PDSLS is still useful for inspecting raw records, but it should not be the main
100
+
destination when Pensieve already knows a better app URL.
101
+
102
+
## Caching / Reuse
103
+
104
+
There is no separate cache layer. Turbopuffer namespaces are the cache:
105
+
106
+
- first search for an actor can be slow because it builds the namespace
107
+
- later searches for that actor are fast because `/api/search` reuses it
108
+
- `/api/update` is a rebuild path and deletes the namespace first
109
+
110
+
Namespace naming:
111
+
112
+
```text
113
+
${TURBOPUFFER_NAMESPACE_PREFIX}-${did with punctuation replaced}
114
+
```
115
+
116
+
## Zig Prototype
117
+
118
+
The Zig code is an earlier extractor and still useful reference:
119
+
120
+
- `src/repo_walk.zig` streams `com.atproto.sync.getRepo` to a temp file, mmaps
121
+
the CAR, indexes blocks, and walks the MST.
122
+
- `src/record_text.zig` flattens records into embedding-ready text.
123
+
- `src/main.zig` exposes `extract` and `demo`.
124
+
125
+
The deployed Worker does not call the Zig binary today.
+103
docs/OPERATIONS.md
+103
docs/OPERATIONS.md
···
1
+
# Operations
2
+
3
+
## Deployment
4
+
5
+
Pensieve deploys as a Cloudflare Worker.
6
+
7
+
```sh
8
+
npm install
9
+
npm run dry-run
10
+
npm run deploy
11
+
```
12
+
13
+
The Worker config is in `wrangler.jsonc`.
14
+
15
+
Live URL:
16
+
17
+
```text
18
+
https://pensieve.n8-3e9.workers.dev
19
+
```
20
+
21
+
## Required Secrets
22
+
23
+
Cloudflare Worker secrets:
24
+
25
+
- `VOYAGE_API_KEY` - used for query and document embeddings.
26
+
- `TURBOPUFFER_API_KEY` - used for namespace delete/upsert/query.
27
+
28
+
Set them with:
29
+
30
+
```sh
31
+
npx wrangler secret put VOYAGE_API_KEY
32
+
npx wrangler secret put TURBOPUFFER_API_KEY
33
+
```
34
+
35
+
Do not commit local secret files. `.env` and `.dev.vars` are ignored.
36
+
37
+
For local development, either:
38
+
39
+
```sh
40
+
npx wrangler dev
41
+
```
42
+
43
+
with `.dev.vars`, or run through 1Password:
44
+
45
+
```sh
46
+
op run --env-file .env -- npm run dev
47
+
```
48
+
49
+
Prefer 1Password or Wrangler secrets over printing values in the terminal.
50
+
51
+
## Non-Secret Configuration
52
+
53
+
`wrangler.jsonc` defines:
54
+
55
+
```jsonc
56
+
"TURBOPUFFER_NAMESPACE_PREFIX": "pensieve"
57
+
```
58
+
59
+
Changing this creates a new logical cache namespace. Existing indexed actors in
60
+
the old prefix will not be reused.
61
+
62
+
## Rebuilding An Actor
63
+
64
+
The app rebuilds an actor automatically when `/api/search` reports a missing
65
+
namespace. To force a rebuild manually:
66
+
67
+
```sh
68
+
curl -N "https://pensieve.n8-3e9.workers.dev/api/update?actor=zzstoatzz.io&q=bufo&top_k=8"
69
+
```
70
+
71
+
`/api/update` streams NDJSON progress and deletes the actor namespace before
72
+
re-indexing.
73
+
74
+
## Smoke Checks
75
+
76
+
Syntax and bundle:
77
+
78
+
```sh
79
+
node --check src/worker.js
80
+
npm run dry-run
81
+
```
82
+
83
+
Live API:
84
+
85
+
```sh
86
+
curl "https://pensieve.n8-3e9.workers.dev/health"
87
+
curl "https://pensieve.n8-3e9.workers.dev/api/search?actor=zzstoatzz.io&q=bufo"
88
+
```
89
+
90
+
Expected search behavior:
91
+
92
+
- if the actor has already been indexed, results return quickly
93
+
- if not, `needsIndex: true` tells the frontend to call `/api/update`
94
+
95
+
## Known Limits
96
+
97
+
- `INDEX_LIMIT` currently caps indexing at 5,000 searchable records per actor.
98
+
- No background freshness job exists yet. A namespace can get stale until
99
+
`/api/update` is called.
100
+
- Artifact extraction is heuristic. It intentionally favors a generic semantic
101
+
envelope over per-app hardcoded collection logic.
102
+
- The Worker currently owns both backend and frontend in one file. That is fine
103
+
for the MVP but should be split if the UI keeps growing.
+1746
package-lock.json
+1746
package-lock.json
···
1
+
{
2
+
"name": "pensieve",
3
+
"version": "0.1.0",
4
+
"lockfileVersion": 3,
5
+
"requires": true,
6
+
"packages": {
7
+
"": {
8
+
"name": "pensieve",
9
+
"version": "0.1.0",
10
+
"dependencies": {
11
+
"@atcute/atproto": "^4.0.3",
12
+
"@atcute/car": "^6.0.2",
13
+
"@atcute/cbor": "^2.3.5",
14
+
"@atcute/cid": "^2.4.2",
15
+
"@atcute/client": "^5.1.0",
16
+
"@atcute/identity-resolver": "^2.0.1",
17
+
"@atcute/mst": "^1.0.2",
18
+
"@atcute/repo": "^1.0.2"
19
+
},
20
+
"devDependencies": {
21
+
"wrangler": "^4.99.0"
22
+
}
23
+
},
24
+
"node_modules/@atcute/atproto": {
25
+
"version": "4.0.3",
26
+
"resolved": "https://registry.npmjs.org/@atcute/atproto/-/atproto-4.0.3.tgz",
27
+
"integrity": "sha512-BNylfO7nK0yYBpSpnGhOYgrJTeZWrXHPrb6tOQmp9A3Am0epctIWm6/5lPC4ZNPHpUbwr5w/LzH/v7kjAoKEDg==",
28
+
"license": "0BSD",
29
+
"dependencies": {
30
+
"@atcute/lexicons": "^2.0.2"
31
+
},
32
+
"peerDependencies": {
33
+
"@atcute/lexicons": "^2.0.0"
34
+
}
35
+
},
36
+
"node_modules/@atcute/car": {
37
+
"version": "6.0.2",
38
+
"resolved": "https://registry.npmjs.org/@atcute/car/-/car-6.0.2.tgz",
39
+
"integrity": "sha512-6AaLjO0zrFD8R/aK7jwrqHEmLzfVilu/5pv4LAcUjxIYfBrw+nThV2FVLPtA4Jt59tIhjp5tQDP+9sx1eqQp+A==",
40
+
"license": "0BSD",
41
+
"dependencies": {
42
+
"@atcute/cbor": "^2.3.5",
43
+
"@atcute/cid": "^2.4.2",
44
+
"@atcute/uint8array": "^1.1.3",
45
+
"@atcute/varint": "^2.0.1"
46
+
},
47
+
"peerDependencies": {
48
+
"@atcute/cbor": "^2.0.0",
49
+
"@atcute/cid": "^2.0.0"
50
+
}
51
+
},
52
+
"node_modules/@atcute/cbor": {
53
+
"version": "2.3.5",
54
+
"resolved": "https://registry.npmjs.org/@atcute/cbor/-/cbor-2.3.5.tgz",
55
+
"integrity": "sha512-NvgeibMqtfeD6/4NMTnBwhjyB/F945UdnzeZMEsGb2eXSalOBzutgNmpnBPTj5FiN6Sl9VYhE3qDQfyox+P8aw==",
56
+
"license": "0BSD",
57
+
"dependencies": {
58
+
"@atcute/cid": "^2.4.2",
59
+
"@atcute/multibase": "^1.2.4",
60
+
"@atcute/uint8array": "^1.1.3"
61
+
},
62
+
"peerDependencies": {
63
+
"@atcute/cid": "^2.0.0"
64
+
}
65
+
},
66
+
"node_modules/@atcute/cid": {
67
+
"version": "2.4.2",
68
+
"resolved": "https://registry.npmjs.org/@atcute/cid/-/cid-2.4.2.tgz",
69
+
"integrity": "sha512-Uy48yfyo/hPQXF+XMXWIGomF6v8IvX5ErjozXMV7rlfU3EV7PSVX3J7plJXV6MRC3iI1z3PgTZTS7V0drCQVVw==",
70
+
"license": "0BSD",
71
+
"dependencies": {
72
+
"@atcute/multibase": "^1.2.4",
73
+
"@atcute/uint8array": "^1.1.3"
74
+
}
75
+
},
76
+
"node_modules/@atcute/client": {
77
+
"version": "5.1.0",
78
+
"resolved": "https://registry.npmjs.org/@atcute/client/-/client-5.1.0.tgz",
79
+
"integrity": "sha512-l2LYCY43QvrOsvS+q1d959x0yVeXQ5F7haloCB8MLzrTKT3s9fc4S3Kr+8JkgjPtdapgOPIeEdhWcrzP5WNLRg==",
80
+
"license": "0BSD",
81
+
"dependencies": {
82
+
"@atcute/identity": "^2.0.0",
83
+
"@atcute/lexicons": "^2.0.0"
84
+
},
85
+
"peerDependencies": {
86
+
"@atcute/lexicons": "^2.0.0"
87
+
}
88
+
},
89
+
"node_modules/@atcute/crypto": {
90
+
"version": "2.4.2",
91
+
"resolved": "https://registry.npmjs.org/@atcute/crypto/-/crypto-2.4.2.tgz",
92
+
"integrity": "sha512-WKVVstCsgGZaaEJFemC6UPh5MtEB2SMU0tiOi744i9T2a6oInJg3VnbTzQsGkyQqqTNC30IHF7//Dwi4hYnu1g==",
93
+
"license": "0BSD",
94
+
"dependencies": {
95
+
"@atcute/multibase": "^1.2.4",
96
+
"@atcute/uint8array": "^1.1.3",
97
+
"@noble/secp256k1": "^3.1.0"
98
+
}
99
+
},
100
+
"node_modules/@atcute/identity": {
101
+
"version": "2.0.1",
102
+
"resolved": "https://registry.npmjs.org/@atcute/identity/-/identity-2.0.1.tgz",
103
+
"integrity": "sha512-FEURUvl30SyyWWikkvm+MLz0Snuf0OF10L/qxRhWjj6qDB5Ib+XWhiBuwidjvhCkrCepTUNLbj4TlUm/gHaUig==",
104
+
"license": "0BSD",
105
+
"dependencies": {
106
+
"@atcute/lexicons": "^2.0.2",
107
+
"valibot": "^1.4.1"
108
+
},
109
+
"peerDependencies": {
110
+
"@atcute/lexicons": "^2.0.0"
111
+
}
112
+
},
113
+
"node_modules/@atcute/identity-resolver": {
114
+
"version": "2.0.1",
115
+
"resolved": "https://registry.npmjs.org/@atcute/identity-resolver/-/identity-resolver-2.0.1.tgz",
116
+
"integrity": "sha512-0enA9w7XnbbqsZ5Rcl6jXLf7ZZuwFQ9dBmxFq3qOxPHLaCETsqsrQflXDPqiM27TnZwYq8sqCV5D1mFOksggDQ==",
117
+
"license": "0BSD",
118
+
"dependencies": {
119
+
"@atcute/lexicons": "^2.0.2",
120
+
"@atcute/util-fetch": "^2.0.1",
121
+
"valibot": "^1.4.1"
122
+
},
123
+
"peerDependencies": {
124
+
"@atcute/identity": "^2.0.0",
125
+
"@atcute/lexicons": "^2.0.0"
126
+
}
127
+
},
128
+
"node_modules/@atcute/lexicons": {
129
+
"version": "2.0.2",
130
+
"resolved": "https://registry.npmjs.org/@atcute/lexicons/-/lexicons-2.0.2.tgz",
131
+
"integrity": "sha512-ATBADJAy4KQ76NB86BjgYKrRdbDRUo76Cbqna4WIfQAgN105Rcy972MiNKs+BSmcOOM3WakilgTm0CXD4RC0iA==",
132
+
"license": "0BSD",
133
+
"dependencies": {
134
+
"@atcute/uint8array": "^1.1.3",
135
+
"@atcute/util-text": "^1.3.3",
136
+
"@standard-schema/spec": "^1.1.0",
137
+
"esm-env": "^1.2.2"
138
+
}
139
+
},
140
+
"node_modules/@atcute/mst": {
141
+
"version": "1.0.2",
142
+
"resolved": "https://registry.npmjs.org/@atcute/mst/-/mst-1.0.2.tgz",
143
+
"integrity": "sha512-fb98Ygs0PjVAOE4WmtMGqi+PS4Y0Jsip4CIkzWBxyIcbfix8kBGQUwZi54k4rKiGfT9qroCY2rlaPFzwkkAFzQ==",
144
+
"license": "0BSD",
145
+
"dependencies": {
146
+
"@atcute/cbor": "^2.3.5",
147
+
"@atcute/cid": "^2.4.2",
148
+
"@atcute/uint8array": "^1.1.3"
149
+
},
150
+
"peerDependencies": {
151
+
"@atcute/cbor": "^2.0.0",
152
+
"@atcute/cid": "^2.0.0"
153
+
}
154
+
},
155
+
"node_modules/@atcute/multibase": {
156
+
"version": "1.2.4",
157
+
"resolved": "https://registry.npmjs.org/@atcute/multibase/-/multibase-1.2.4.tgz",
158
+
"integrity": "sha512-WeX12hvFZEim6C+cyv7Eqd93w6DzubNWQGmTFBghjsEuXvMe4HbBCYvsti0OUnbA5qLBPlsTyssQUJeLlHCzIw==",
159
+
"license": "0BSD",
160
+
"dependencies": {
161
+
"@atcute/uint8array": "^1.1.3"
162
+
}
163
+
},
164
+
"node_modules/@atcute/repo": {
165
+
"version": "1.0.2",
166
+
"resolved": "https://registry.npmjs.org/@atcute/repo/-/repo-1.0.2.tgz",
167
+
"integrity": "sha512-fsAuGbagOW52nSFtf/TWBx25A6xpZvzxBk0+S14Bp5sGNq7JoQ0zyu40P7b6+L8b/buB+1ACHPntYsKqCI4Awg==",
168
+
"license": "0BSD",
169
+
"dependencies": {
170
+
"@atcute/car": "^6.0.2",
171
+
"@atcute/cbor": "^2.3.5",
172
+
"@atcute/cid": "^2.4.2",
173
+
"@atcute/crypto": "^2.4.2",
174
+
"@atcute/lexicons": "^2.0.2",
175
+
"@atcute/mst": "^1.0.2",
176
+
"@atcute/uint8array": "^1.1.3"
177
+
},
178
+
"peerDependencies": {
179
+
"@atcute/cbor": "^2.0.0",
180
+
"@atcute/cid": "^2.0.0",
181
+
"@atcute/lexicons": "^2.0.0"
182
+
}
183
+
},
184
+
"node_modules/@atcute/uint8array": {
185
+
"version": "1.1.3",
186
+
"resolved": "https://registry.npmjs.org/@atcute/uint8array/-/uint8array-1.1.3.tgz",
187
+
"integrity": "sha512-2KLcMQHUFtntY3tEjdyqqq1tR9hvPFndluWFCa637QY0cMyvq0fHSnhmZeWaSRIXMCwVDY3TLLWHNOHEWFb11g==",
188
+
"license": "0BSD"
189
+
},
190
+
"node_modules/@atcute/util-fetch": {
191
+
"version": "2.0.1",
192
+
"resolved": "https://registry.npmjs.org/@atcute/util-fetch/-/util-fetch-2.0.1.tgz",
193
+
"integrity": "sha512-ugWTOLemA8OxSOj7c8q6ncRmBGFDHSwwE1YinO+PCtaw6WLQFGBfHn+yikQ0e3wTK2t4IPjQ5PxZcRXm961ZVA==",
194
+
"license": "0BSD",
195
+
"dependencies": {
196
+
"valibot": "^1.4.1"
197
+
}
198
+
},
199
+
"node_modules/@atcute/util-text": {
200
+
"version": "1.3.3",
201
+
"resolved": "https://registry.npmjs.org/@atcute/util-text/-/util-text-1.3.3.tgz",
202
+
"integrity": "sha512-WhedTmg/msFhrdwXw9RjnNcDl8Vmisxl4+Vzyf5k3+8Gj5TKQg72dLSDtBNmNLd61RbHjgfQRBgE0ez6q/jciw==",
203
+
"license": "0BSD",
204
+
"dependencies": {
205
+
"unicode-segmenter": "^0.14.5"
206
+
}
207
+
},
208
+
"node_modules/@atcute/varint": {
209
+
"version": "2.0.1",
210
+
"resolved": "https://registry.npmjs.org/@atcute/varint/-/varint-2.0.1.tgz",
211
+
"integrity": "sha512-reTtgQ1VPGVRcaTohKVb1tUbpvE7MpguoNS0gjhsxDWFY3t1yqWWvwKK6cJodAmePML36JombvgFwfsblir9Jg==",
212
+
"license": "0BSD"
213
+
},
214
+
"node_modules/@cloudflare/kv-asset-handler": {
215
+
"version": "0.5.0",
216
+
"resolved": "https://registry.npmjs.org/@cloudflare/kv-asset-handler/-/kv-asset-handler-0.5.0.tgz",
217
+
"integrity": "sha512-jxQYkj8dSIzc0cD6cMMNdOc1UVjqSqu8BZdor5s8cGjW2I8BjODt/kWPVdY+u9zj3ms75Q5qaZgnxUad83+eAg==",
218
+
"dev": true,
219
+
"license": "MIT OR Apache-2.0",
220
+
"engines": {
221
+
"node": ">=22.0.0"
222
+
}
223
+
},
224
+
"node_modules/@cloudflare/unenv-preset": {
225
+
"version": "2.16.1",
226
+
"resolved": "https://registry.npmjs.org/@cloudflare/unenv-preset/-/unenv-preset-2.16.1.tgz",
227
+
"integrity": "sha512-ECxObrMfyTl5bhQf/lZCXwo5G6xX9IAUo+nDMKK4SZ8m4Jvvxp52vilxyySSWh2YTZz8+HQ07qGH/2rEom1vDw==",
228
+
"dev": true,
229
+
"license": "MIT OR Apache-2.0",
230
+
"peerDependencies": {
231
+
"unenv": "2.0.0-rc.24",
232
+
"workerd": ">1.20260305.0 <2.0.0-0"
233
+
},
234
+
"peerDependenciesMeta": {
235
+
"workerd": {
236
+
"optional": true
237
+
}
238
+
}
239
+
},
240
+
"node_modules/@cloudflare/workerd-darwin-64": {
241
+
"version": "1.20260625.1",
242
+
"resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-64/-/workerd-darwin-64-1.20260625.1.tgz",
243
+
"integrity": "sha512-naCfBv0WnnTQIQPTniqMoUlklOIFjrAcSn1X+IAOhY8aFLF/xGYtFjs1eEE8sFib3ZuChGGpU23FFORVczqr0A==",
244
+
"cpu": [
245
+
"x64"
246
+
],
247
+
"dev": true,
248
+
"license": "Apache-2.0",
249
+
"optional": true,
250
+
"os": [
251
+
"darwin"
252
+
],
253
+
"engines": {
254
+
"node": ">=16"
255
+
}
256
+
},
257
+
"node_modules/@cloudflare/workerd-darwin-arm64": {
258
+
"version": "1.20260625.1",
259
+
"resolved": "https://registry.npmjs.org/@cloudflare/workerd-darwin-arm64/-/workerd-darwin-arm64-1.20260625.1.tgz",
260
+
"integrity": "sha512-jmH6zjp6Wrux46+qtFwDwrj+vd7s5bdwEqeGvdnwE0a4IEeAhKs0L42HQOyID+g5lkrHq9m55+AbhtmRAm63Pw==",
261
+
"cpu": [
262
+
"arm64"
263
+
],
264
+
"dev": true,
265
+
"license": "Apache-2.0",
266
+
"optional": true,
267
+
"os": [
268
+
"darwin"
269
+
],
270
+
"engines": {
271
+
"node": ">=16"
272
+
}
273
+
},
274
+
"node_modules/@cloudflare/workerd-linux-64": {
275
+
"version": "1.20260625.1",
276
+
"resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-64/-/workerd-linux-64-1.20260625.1.tgz",
277
+
"integrity": "sha512-MiQkpA/dX8d83Zp64pzHUKfd6ca4cvwxnNobSP6CnXvfESvnNI9pfa+nfwnParla36sPmnYntNkjR7NjRuDeKQ==",
278
+
"cpu": [
279
+
"x64"
280
+
],
281
+
"dev": true,
282
+
"license": "Apache-2.0",
283
+
"optional": true,
284
+
"os": [
285
+
"linux"
286
+
],
287
+
"engines": {
288
+
"node": ">=16"
289
+
}
290
+
},
291
+
"node_modules/@cloudflare/workerd-linux-arm64": {
292
+
"version": "1.20260625.1",
293
+
"resolved": "https://registry.npmjs.org/@cloudflare/workerd-linux-arm64/-/workerd-linux-arm64-1.20260625.1.tgz",
294
+
"integrity": "sha512-LxxW7Qv60Xvv37+w6gUSDpYZziyqMy+cZWd9IvSA5ehVgKAxmzEaYPMiSZlxk32nbIWL9u/tfjXYCOKJ4Lo+XQ==",
295
+
"cpu": [
296
+
"arm64"
297
+
],
298
+
"dev": true,
299
+
"license": "Apache-2.0",
300
+
"optional": true,
301
+
"os": [
302
+
"linux"
303
+
],
304
+
"engines": {
305
+
"node": ">=16"
306
+
}
307
+
},
308
+
"node_modules/@cloudflare/workerd-windows-64": {
309
+
"version": "1.20260625.1",
310
+
"resolved": "https://registry.npmjs.org/@cloudflare/workerd-windows-64/-/workerd-windows-64-1.20260625.1.tgz",
311
+
"integrity": "sha512-LH6iIX1HHaTwVKV5VokDxxUErXJzQoNZFRwVm7Vx/3fB/ApcTcRCUaMqcxI4as94jEUqg+pmX5czOndiveohow==",
312
+
"cpu": [
313
+
"x64"
314
+
],
315
+
"dev": true,
316
+
"license": "Apache-2.0",
317
+
"optional": true,
318
+
"os": [
319
+
"win32"
320
+
],
321
+
"engines": {
322
+
"node": ">=16"
323
+
}
324
+
},
325
+
"node_modules/@cspotcode/source-map-support": {
326
+
"version": "0.8.1",
327
+
"resolved": "https://registry.npmjs.org/@cspotcode/source-map-support/-/source-map-support-0.8.1.tgz",
328
+
"integrity": "sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw==",
329
+
"dev": true,
330
+
"license": "MIT",
331
+
"dependencies": {
332
+
"@jridgewell/trace-mapping": "0.3.9"
333
+
},
334
+
"engines": {
335
+
"node": ">=12"
336
+
}
337
+
},
338
+
"node_modules/@emnapi/runtime": {
339
+
"version": "1.11.1",
340
+
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.1.tgz",
341
+
"integrity": "sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw==",
342
+
"dev": true,
343
+
"license": "MIT",
344
+
"optional": true,
345
+
"dependencies": {
346
+
"tslib": "^2.4.0"
347
+
}
348
+
},
349
+
"node_modules/@esbuild/aix-ppc64": {
350
+
"version": "0.28.1",
351
+
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz",
352
+
"integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==",
353
+
"cpu": [
354
+
"ppc64"
355
+
],
356
+
"dev": true,
357
+
"license": "MIT",
358
+
"optional": true,
359
+
"os": [
360
+
"aix"
361
+
],
362
+
"engines": {
363
+
"node": ">=18"
364
+
}
365
+
},
366
+
"node_modules/@esbuild/android-arm": {
367
+
"version": "0.28.1",
368
+
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz",
369
+
"integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==",
370
+
"cpu": [
371
+
"arm"
372
+
],
373
+
"dev": true,
374
+
"license": "MIT",
375
+
"optional": true,
376
+
"os": [
377
+
"android"
378
+
],
379
+
"engines": {
380
+
"node": ">=18"
381
+
}
382
+
},
383
+
"node_modules/@esbuild/android-arm64": {
384
+
"version": "0.28.1",
385
+
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz",
386
+
"integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==",
387
+
"cpu": [
388
+
"arm64"
389
+
],
390
+
"dev": true,
391
+
"license": "MIT",
392
+
"optional": true,
393
+
"os": [
394
+
"android"
395
+
],
396
+
"engines": {
397
+
"node": ">=18"
398
+
}
399
+
},
400
+
"node_modules/@esbuild/android-x64": {
401
+
"version": "0.28.1",
402
+
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz",
403
+
"integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==",
404
+
"cpu": [
405
+
"x64"
406
+
],
407
+
"dev": true,
408
+
"license": "MIT",
409
+
"optional": true,
410
+
"os": [
411
+
"android"
412
+
],
413
+
"engines": {
414
+
"node": ">=18"
415
+
}
416
+
},
417
+
"node_modules/@esbuild/darwin-arm64": {
418
+
"version": "0.28.1",
419
+
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz",
420
+
"integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==",
421
+
"cpu": [
422
+
"arm64"
423
+
],
424
+
"dev": true,
425
+
"license": "MIT",
426
+
"optional": true,
427
+
"os": [
428
+
"darwin"
429
+
],
430
+
"engines": {
431
+
"node": ">=18"
432
+
}
433
+
},
434
+
"node_modules/@esbuild/darwin-x64": {
435
+
"version": "0.28.1",
436
+
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz",
437
+
"integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==",
438
+
"cpu": [
439
+
"x64"
440
+
],
441
+
"dev": true,
442
+
"license": "MIT",
443
+
"optional": true,
444
+
"os": [
445
+
"darwin"
446
+
],
447
+
"engines": {
448
+
"node": ">=18"
449
+
}
450
+
},
451
+
"node_modules/@esbuild/freebsd-arm64": {
452
+
"version": "0.28.1",
453
+
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz",
454
+
"integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==",
455
+
"cpu": [
456
+
"arm64"
457
+
],
458
+
"dev": true,
459
+
"license": "MIT",
460
+
"optional": true,
461
+
"os": [
462
+
"freebsd"
463
+
],
464
+
"engines": {
465
+
"node": ">=18"
466
+
}
467
+
},
468
+
"node_modules/@esbuild/freebsd-x64": {
469
+
"version": "0.28.1",
470
+
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz",
471
+
"integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==",
472
+
"cpu": [
473
+
"x64"
474
+
],
475
+
"dev": true,
476
+
"license": "MIT",
477
+
"optional": true,
478
+
"os": [
479
+
"freebsd"
480
+
],
481
+
"engines": {
482
+
"node": ">=18"
483
+
}
484
+
},
485
+
"node_modules/@esbuild/linux-arm": {
486
+
"version": "0.28.1",
487
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz",
488
+
"integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==",
489
+
"cpu": [
490
+
"arm"
491
+
],
492
+
"dev": true,
493
+
"license": "MIT",
494
+
"optional": true,
495
+
"os": [
496
+
"linux"
497
+
],
498
+
"engines": {
499
+
"node": ">=18"
500
+
}
501
+
},
502
+
"node_modules/@esbuild/linux-arm64": {
503
+
"version": "0.28.1",
504
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz",
505
+
"integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==",
506
+
"cpu": [
507
+
"arm64"
508
+
],
509
+
"dev": true,
510
+
"license": "MIT",
511
+
"optional": true,
512
+
"os": [
513
+
"linux"
514
+
],
515
+
"engines": {
516
+
"node": ">=18"
517
+
}
518
+
},
519
+
"node_modules/@esbuild/linux-ia32": {
520
+
"version": "0.28.1",
521
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz",
522
+
"integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==",
523
+
"cpu": [
524
+
"ia32"
525
+
],
526
+
"dev": true,
527
+
"license": "MIT",
528
+
"optional": true,
529
+
"os": [
530
+
"linux"
531
+
],
532
+
"engines": {
533
+
"node": ">=18"
534
+
}
535
+
},
536
+
"node_modules/@esbuild/linux-loong64": {
537
+
"version": "0.28.1",
538
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz",
539
+
"integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==",
540
+
"cpu": [
541
+
"loong64"
542
+
],
543
+
"dev": true,
544
+
"license": "MIT",
545
+
"optional": true,
546
+
"os": [
547
+
"linux"
548
+
],
549
+
"engines": {
550
+
"node": ">=18"
551
+
}
552
+
},
553
+
"node_modules/@esbuild/linux-mips64el": {
554
+
"version": "0.28.1",
555
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz",
556
+
"integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==",
557
+
"cpu": [
558
+
"mips64el"
559
+
],
560
+
"dev": true,
561
+
"license": "MIT",
562
+
"optional": true,
563
+
"os": [
564
+
"linux"
565
+
],
566
+
"engines": {
567
+
"node": ">=18"
568
+
}
569
+
},
570
+
"node_modules/@esbuild/linux-ppc64": {
571
+
"version": "0.28.1",
572
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz",
573
+
"integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==",
574
+
"cpu": [
575
+
"ppc64"
576
+
],
577
+
"dev": true,
578
+
"license": "MIT",
579
+
"optional": true,
580
+
"os": [
581
+
"linux"
582
+
],
583
+
"engines": {
584
+
"node": ">=18"
585
+
}
586
+
},
587
+
"node_modules/@esbuild/linux-riscv64": {
588
+
"version": "0.28.1",
589
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz",
590
+
"integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==",
591
+
"cpu": [
592
+
"riscv64"
593
+
],
594
+
"dev": true,
595
+
"license": "MIT",
596
+
"optional": true,
597
+
"os": [
598
+
"linux"
599
+
],
600
+
"engines": {
601
+
"node": ">=18"
602
+
}
603
+
},
604
+
"node_modules/@esbuild/linux-s390x": {
605
+
"version": "0.28.1",
606
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz",
607
+
"integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==",
608
+
"cpu": [
609
+
"s390x"
610
+
],
611
+
"dev": true,
612
+
"license": "MIT",
613
+
"optional": true,
614
+
"os": [
615
+
"linux"
616
+
],
617
+
"engines": {
618
+
"node": ">=18"
619
+
}
620
+
},
621
+
"node_modules/@esbuild/linux-x64": {
622
+
"version": "0.28.1",
623
+
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz",
624
+
"integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==",
625
+
"cpu": [
626
+
"x64"
627
+
],
628
+
"dev": true,
629
+
"license": "MIT",
630
+
"optional": true,
631
+
"os": [
632
+
"linux"
633
+
],
634
+
"engines": {
635
+
"node": ">=18"
636
+
}
637
+
},
638
+
"node_modules/@esbuild/netbsd-arm64": {
639
+
"version": "0.28.1",
640
+
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz",
641
+
"integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==",
642
+
"cpu": [
643
+
"arm64"
644
+
],
645
+
"dev": true,
646
+
"license": "MIT",
647
+
"optional": true,
648
+
"os": [
649
+
"netbsd"
650
+
],
651
+
"engines": {
652
+
"node": ">=18"
653
+
}
654
+
},
655
+
"node_modules/@esbuild/netbsd-x64": {
656
+
"version": "0.28.1",
657
+
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz",
658
+
"integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==",
659
+
"cpu": [
660
+
"x64"
661
+
],
662
+
"dev": true,
663
+
"license": "MIT",
664
+
"optional": true,
665
+
"os": [
666
+
"netbsd"
667
+
],
668
+
"engines": {
669
+
"node": ">=18"
670
+
}
671
+
},
672
+
"node_modules/@esbuild/openbsd-arm64": {
673
+
"version": "0.28.1",
674
+
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz",
675
+
"integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==",
676
+
"cpu": [
677
+
"arm64"
678
+
],
679
+
"dev": true,
680
+
"license": "MIT",
681
+
"optional": true,
682
+
"os": [
683
+
"openbsd"
684
+
],
685
+
"engines": {
686
+
"node": ">=18"
687
+
}
688
+
},
689
+
"node_modules/@esbuild/openbsd-x64": {
690
+
"version": "0.28.1",
691
+
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz",
692
+
"integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==",
693
+
"cpu": [
694
+
"x64"
695
+
],
696
+
"dev": true,
697
+
"license": "MIT",
698
+
"optional": true,
699
+
"os": [
700
+
"openbsd"
701
+
],
702
+
"engines": {
703
+
"node": ">=18"
704
+
}
705
+
},
706
+
"node_modules/@esbuild/openharmony-arm64": {
707
+
"version": "0.28.1",
708
+
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz",
709
+
"integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==",
710
+
"cpu": [
711
+
"arm64"
712
+
],
713
+
"dev": true,
714
+
"license": "MIT",
715
+
"optional": true,
716
+
"os": [
717
+
"openharmony"
718
+
],
719
+
"engines": {
720
+
"node": ">=18"
721
+
}
722
+
},
723
+
"node_modules/@esbuild/sunos-x64": {
724
+
"version": "0.28.1",
725
+
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz",
726
+
"integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==",
727
+
"cpu": [
728
+
"x64"
729
+
],
730
+
"dev": true,
731
+
"license": "MIT",
732
+
"optional": true,
733
+
"os": [
734
+
"sunos"
735
+
],
736
+
"engines": {
737
+
"node": ">=18"
738
+
}
739
+
},
740
+
"node_modules/@esbuild/win32-arm64": {
741
+
"version": "0.28.1",
742
+
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz",
743
+
"integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==",
744
+
"cpu": [
745
+
"arm64"
746
+
],
747
+
"dev": true,
748
+
"license": "MIT",
749
+
"optional": true,
750
+
"os": [
751
+
"win32"
752
+
],
753
+
"engines": {
754
+
"node": ">=18"
755
+
}
756
+
},
757
+
"node_modules/@esbuild/win32-ia32": {
758
+
"version": "0.28.1",
759
+
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz",
760
+
"integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==",
761
+
"cpu": [
762
+
"ia32"
763
+
],
764
+
"dev": true,
765
+
"license": "MIT",
766
+
"optional": true,
767
+
"os": [
768
+
"win32"
769
+
],
770
+
"engines": {
771
+
"node": ">=18"
772
+
}
773
+
},
774
+
"node_modules/@esbuild/win32-x64": {
775
+
"version": "0.28.1",
776
+
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz",
777
+
"integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==",
778
+
"cpu": [
779
+
"x64"
780
+
],
781
+
"dev": true,
782
+
"license": "MIT",
783
+
"optional": true,
784
+
"os": [
785
+
"win32"
786
+
],
787
+
"engines": {
788
+
"node": ">=18"
789
+
}
790
+
},
791
+
"node_modules/@img/colour": {
792
+
"version": "1.1.0",
793
+
"resolved": "https://registry.npmjs.org/@img/colour/-/colour-1.1.0.tgz",
794
+
"integrity": "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==",
795
+
"dev": true,
796
+
"license": "MIT",
797
+
"engines": {
798
+
"node": ">=18"
799
+
}
800
+
},
801
+
"node_modules/@img/sharp-darwin-arm64": {
802
+
"version": "0.34.5",
803
+
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.34.5.tgz",
804
+
"integrity": "sha512-imtQ3WMJXbMY4fxb/Ndp6HBTNVtWCUI0WdobyheGf5+ad6xX8VIDO8u2xE4qc/fr08CKG/7dDseFtn6M6g/r3w==",
805
+
"cpu": [
806
+
"arm64"
807
+
],
808
+
"dev": true,
809
+
"license": "Apache-2.0",
810
+
"optional": true,
811
+
"os": [
812
+
"darwin"
813
+
],
814
+
"engines": {
815
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
816
+
},
817
+
"funding": {
818
+
"url": "https://opencollective.com/libvips"
819
+
},
820
+
"optionalDependencies": {
821
+
"@img/sharp-libvips-darwin-arm64": "1.2.4"
822
+
}
823
+
},
824
+
"node_modules/@img/sharp-darwin-x64": {
825
+
"version": "0.34.5",
826
+
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.34.5.tgz",
827
+
"integrity": "sha512-YNEFAF/4KQ/PeW0N+r+aVVsoIY0/qxxikF2SWdp+NRkmMB7y9LBZAVqQ4yhGCm/H3H270OSykqmQMKLBhBJDEw==",
828
+
"cpu": [
829
+
"x64"
830
+
],
831
+
"dev": true,
832
+
"license": "Apache-2.0",
833
+
"optional": true,
834
+
"os": [
835
+
"darwin"
836
+
],
837
+
"engines": {
838
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
839
+
},
840
+
"funding": {
841
+
"url": "https://opencollective.com/libvips"
842
+
},
843
+
"optionalDependencies": {
844
+
"@img/sharp-libvips-darwin-x64": "1.2.4"
845
+
}
846
+
},
847
+
"node_modules/@img/sharp-libvips-darwin-arm64": {
848
+
"version": "1.2.4",
849
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.2.4.tgz",
850
+
"integrity": "sha512-zqjjo7RatFfFoP0MkQ51jfuFZBnVE2pRiaydKJ1G/rHZvnsrHAOcQALIi9sA5co5xenQdTugCvtb1cuf78Vf4g==",
851
+
"cpu": [
852
+
"arm64"
853
+
],
854
+
"dev": true,
855
+
"license": "LGPL-3.0-or-later",
856
+
"optional": true,
857
+
"os": [
858
+
"darwin"
859
+
],
860
+
"funding": {
861
+
"url": "https://opencollective.com/libvips"
862
+
}
863
+
},
864
+
"node_modules/@img/sharp-libvips-darwin-x64": {
865
+
"version": "1.2.4",
866
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.2.4.tgz",
867
+
"integrity": "sha512-1IOd5xfVhlGwX+zXv2N93k0yMONvUlANylbJw1eTah8K/Jtpi15KC+WSiaX/nBmbm2HxRM1gZ0nSdjSsrZbGKg==",
868
+
"cpu": [
869
+
"x64"
870
+
],
871
+
"dev": true,
872
+
"license": "LGPL-3.0-or-later",
873
+
"optional": true,
874
+
"os": [
875
+
"darwin"
876
+
],
877
+
"funding": {
878
+
"url": "https://opencollective.com/libvips"
879
+
}
880
+
},
881
+
"node_modules/@img/sharp-libvips-linux-arm": {
882
+
"version": "1.2.4",
883
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.2.4.tgz",
884
+
"integrity": "sha512-bFI7xcKFELdiNCVov8e44Ia4u2byA+l3XtsAj+Q8tfCwO6BQ8iDojYdvoPMqsKDkuoOo+X6HZA0s0q11ANMQ8A==",
885
+
"cpu": [
886
+
"arm"
887
+
],
888
+
"dev": true,
889
+
"license": "LGPL-3.0-or-later",
890
+
"optional": true,
891
+
"os": [
892
+
"linux"
893
+
],
894
+
"funding": {
895
+
"url": "https://opencollective.com/libvips"
896
+
}
897
+
},
898
+
"node_modules/@img/sharp-libvips-linux-arm64": {
899
+
"version": "1.2.4",
900
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.2.4.tgz",
901
+
"integrity": "sha512-excjX8DfsIcJ10x1Kzr4RcWe1edC9PquDRRPx3YVCvQv+U5p7Yin2s32ftzikXojb1PIFc/9Mt28/y+iRklkrw==",
902
+
"cpu": [
903
+
"arm64"
904
+
],
905
+
"dev": true,
906
+
"license": "LGPL-3.0-or-later",
907
+
"optional": true,
908
+
"os": [
909
+
"linux"
910
+
],
911
+
"funding": {
912
+
"url": "https://opencollective.com/libvips"
913
+
}
914
+
},
915
+
"node_modules/@img/sharp-libvips-linux-ppc64": {
916
+
"version": "1.2.4",
917
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-ppc64/-/sharp-libvips-linux-ppc64-1.2.4.tgz",
918
+
"integrity": "sha512-FMuvGijLDYG6lW+b/UvyilUWu5Ayu+3r2d1S8notiGCIyYU/76eig1UfMmkZ7vwgOrzKzlQbFSuQfgm7GYUPpA==",
919
+
"cpu": [
920
+
"ppc64"
921
+
],
922
+
"dev": true,
923
+
"license": "LGPL-3.0-or-later",
924
+
"optional": true,
925
+
"os": [
926
+
"linux"
927
+
],
928
+
"funding": {
929
+
"url": "https://opencollective.com/libvips"
930
+
}
931
+
},
932
+
"node_modules/@img/sharp-libvips-linux-riscv64": {
933
+
"version": "1.2.4",
934
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-riscv64/-/sharp-libvips-linux-riscv64-1.2.4.tgz",
935
+
"integrity": "sha512-oVDbcR4zUC0ce82teubSm+x6ETixtKZBh/qbREIOcI3cULzDyb18Sr/Wcyx7NRQeQzOiHTNbZFF1UwPS2scyGA==",
936
+
"cpu": [
937
+
"riscv64"
938
+
],
939
+
"dev": true,
940
+
"license": "LGPL-3.0-or-later",
941
+
"optional": true,
942
+
"os": [
943
+
"linux"
944
+
],
945
+
"funding": {
946
+
"url": "https://opencollective.com/libvips"
947
+
}
948
+
},
949
+
"node_modules/@img/sharp-libvips-linux-s390x": {
950
+
"version": "1.2.4",
951
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-s390x/-/sharp-libvips-linux-s390x-1.2.4.tgz",
952
+
"integrity": "sha512-qmp9VrzgPgMoGZyPvrQHqk02uyjA0/QrTO26Tqk6l4ZV0MPWIW6LTkqOIov+J1yEu7MbFQaDpwdwJKhbJvuRxQ==",
953
+
"cpu": [
954
+
"s390x"
955
+
],
956
+
"dev": true,
957
+
"license": "LGPL-3.0-or-later",
958
+
"optional": true,
959
+
"os": [
960
+
"linux"
961
+
],
962
+
"funding": {
963
+
"url": "https://opencollective.com/libvips"
964
+
}
965
+
},
966
+
"node_modules/@img/sharp-libvips-linux-x64": {
967
+
"version": "1.2.4",
968
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.2.4.tgz",
969
+
"integrity": "sha512-tJxiiLsmHc9Ax1bz3oaOYBURTXGIRDODBqhveVHonrHJ9/+k89qbLl0bcJns+e4t4rvaNBxaEZsFtSfAdquPrw==",
970
+
"cpu": [
971
+
"x64"
972
+
],
973
+
"dev": true,
974
+
"license": "LGPL-3.0-or-later",
975
+
"optional": true,
976
+
"os": [
977
+
"linux"
978
+
],
979
+
"funding": {
980
+
"url": "https://opencollective.com/libvips"
981
+
}
982
+
},
983
+
"node_modules/@img/sharp-libvips-linuxmusl-arm64": {
984
+
"version": "1.2.4",
985
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.2.4.tgz",
986
+
"integrity": "sha512-FVQHuwx1IIuNow9QAbYUzJ+En8KcVm9Lk5+uGUQJHaZmMECZmOlix9HnH7n1TRkXMS0pGxIJokIVB9SuqZGGXw==",
987
+
"cpu": [
988
+
"arm64"
989
+
],
990
+
"dev": true,
991
+
"license": "LGPL-3.0-or-later",
992
+
"optional": true,
993
+
"os": [
994
+
"linux"
995
+
],
996
+
"funding": {
997
+
"url": "https://opencollective.com/libvips"
998
+
}
999
+
},
1000
+
"node_modules/@img/sharp-libvips-linuxmusl-x64": {
1001
+
"version": "1.2.4",
1002
+
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.2.4.tgz",
1003
+
"integrity": "sha512-+LpyBk7L44ZIXwz/VYfglaX/okxezESc6UxDSoyo2Ks6Jxc4Y7sGjpgU9s4PMgqgjj1gZCylTieNamqA1MF7Dg==",
1004
+
"cpu": [
1005
+
"x64"
1006
+
],
1007
+
"dev": true,
1008
+
"license": "LGPL-3.0-or-later",
1009
+
"optional": true,
1010
+
"os": [
1011
+
"linux"
1012
+
],
1013
+
"funding": {
1014
+
"url": "https://opencollective.com/libvips"
1015
+
}
1016
+
},
1017
+
"node_modules/@img/sharp-linux-arm": {
1018
+
"version": "0.34.5",
1019
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.34.5.tgz",
1020
+
"integrity": "sha512-9dLqsvwtg1uuXBGZKsxem9595+ujv0sJ6Vi8wcTANSFpwV/GONat5eCkzQo/1O6zRIkh0m/8+5BjrRr7jDUSZw==",
1021
+
"cpu": [
1022
+
"arm"
1023
+
],
1024
+
"dev": true,
1025
+
"license": "Apache-2.0",
1026
+
"optional": true,
1027
+
"os": [
1028
+
"linux"
1029
+
],
1030
+
"engines": {
1031
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1032
+
},
1033
+
"funding": {
1034
+
"url": "https://opencollective.com/libvips"
1035
+
},
1036
+
"optionalDependencies": {
1037
+
"@img/sharp-libvips-linux-arm": "1.2.4"
1038
+
}
1039
+
},
1040
+
"node_modules/@img/sharp-linux-arm64": {
1041
+
"version": "0.34.5",
1042
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.34.5.tgz",
1043
+
"integrity": "sha512-bKQzaJRY/bkPOXyKx5EVup7qkaojECG6NLYswgktOZjaXecSAeCWiZwwiFf3/Y+O1HrauiE3FVsGxFg8c24rZg==",
1044
+
"cpu": [
1045
+
"arm64"
1046
+
],
1047
+
"dev": true,
1048
+
"license": "Apache-2.0",
1049
+
"optional": true,
1050
+
"os": [
1051
+
"linux"
1052
+
],
1053
+
"engines": {
1054
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1055
+
},
1056
+
"funding": {
1057
+
"url": "https://opencollective.com/libvips"
1058
+
},
1059
+
"optionalDependencies": {
1060
+
"@img/sharp-libvips-linux-arm64": "1.2.4"
1061
+
}
1062
+
},
1063
+
"node_modules/@img/sharp-linux-ppc64": {
1064
+
"version": "0.34.5",
1065
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-ppc64/-/sharp-linux-ppc64-0.34.5.tgz",
1066
+
"integrity": "sha512-7zznwNaqW6YtsfrGGDA6BRkISKAAE1Jo0QdpNYXNMHu2+0dTrPflTLNkpc8l7MUP5M16ZJcUvysVWWrMefZquA==",
1067
+
"cpu": [
1068
+
"ppc64"
1069
+
],
1070
+
"dev": true,
1071
+
"license": "Apache-2.0",
1072
+
"optional": true,
1073
+
"os": [
1074
+
"linux"
1075
+
],
1076
+
"engines": {
1077
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1078
+
},
1079
+
"funding": {
1080
+
"url": "https://opencollective.com/libvips"
1081
+
},
1082
+
"optionalDependencies": {
1083
+
"@img/sharp-libvips-linux-ppc64": "1.2.4"
1084
+
}
1085
+
},
1086
+
"node_modules/@img/sharp-linux-riscv64": {
1087
+
"version": "0.34.5",
1088
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-riscv64/-/sharp-linux-riscv64-0.34.5.tgz",
1089
+
"integrity": "sha512-51gJuLPTKa7piYPaVs8GmByo7/U7/7TZOq+cnXJIHZKavIRHAP77e3N2HEl3dgiqdD/w0yUfiJnII77PuDDFdw==",
1090
+
"cpu": [
1091
+
"riscv64"
1092
+
],
1093
+
"dev": true,
1094
+
"license": "Apache-2.0",
1095
+
"optional": true,
1096
+
"os": [
1097
+
"linux"
1098
+
],
1099
+
"engines": {
1100
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1101
+
},
1102
+
"funding": {
1103
+
"url": "https://opencollective.com/libvips"
1104
+
},
1105
+
"optionalDependencies": {
1106
+
"@img/sharp-libvips-linux-riscv64": "1.2.4"
1107
+
}
1108
+
},
1109
+
"node_modules/@img/sharp-linux-s390x": {
1110
+
"version": "0.34.5",
1111
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-s390x/-/sharp-linux-s390x-0.34.5.tgz",
1112
+
"integrity": "sha512-nQtCk0PdKfho3eC5MrbQoigJ2gd1CgddUMkabUj+rBevs8tZ2cULOx46E7oyX+04WGfABgIwmMC0VqieTiR4jg==",
1113
+
"cpu": [
1114
+
"s390x"
1115
+
],
1116
+
"dev": true,
1117
+
"license": "Apache-2.0",
1118
+
"optional": true,
1119
+
"os": [
1120
+
"linux"
1121
+
],
1122
+
"engines": {
1123
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1124
+
},
1125
+
"funding": {
1126
+
"url": "https://opencollective.com/libvips"
1127
+
},
1128
+
"optionalDependencies": {
1129
+
"@img/sharp-libvips-linux-s390x": "1.2.4"
1130
+
}
1131
+
},
1132
+
"node_modules/@img/sharp-linux-x64": {
1133
+
"version": "0.34.5",
1134
+
"resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.34.5.tgz",
1135
+
"integrity": "sha512-MEzd8HPKxVxVenwAa+JRPwEC7QFjoPWuS5NZnBt6B3pu7EG2Ge0id1oLHZpPJdn3OQK+BQDiw9zStiHBTJQQQQ==",
1136
+
"cpu": [
1137
+
"x64"
1138
+
],
1139
+
"dev": true,
1140
+
"license": "Apache-2.0",
1141
+
"optional": true,
1142
+
"os": [
1143
+
"linux"
1144
+
],
1145
+
"engines": {
1146
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1147
+
},
1148
+
"funding": {
1149
+
"url": "https://opencollective.com/libvips"
1150
+
},
1151
+
"optionalDependencies": {
1152
+
"@img/sharp-libvips-linux-x64": "1.2.4"
1153
+
}
1154
+
},
1155
+
"node_modules/@img/sharp-linuxmusl-arm64": {
1156
+
"version": "0.34.5",
1157
+
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.34.5.tgz",
1158
+
"integrity": "sha512-fprJR6GtRsMt6Kyfq44IsChVZeGN97gTD331weR1ex1c1rypDEABN6Tm2xa1wE6lYb5DdEnk03NZPqA7Id21yg==",
1159
+
"cpu": [
1160
+
"arm64"
1161
+
],
1162
+
"dev": true,
1163
+
"license": "Apache-2.0",
1164
+
"optional": true,
1165
+
"os": [
1166
+
"linux"
1167
+
],
1168
+
"engines": {
1169
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1170
+
},
1171
+
"funding": {
1172
+
"url": "https://opencollective.com/libvips"
1173
+
},
1174
+
"optionalDependencies": {
1175
+
"@img/sharp-libvips-linuxmusl-arm64": "1.2.4"
1176
+
}
1177
+
},
1178
+
"node_modules/@img/sharp-linuxmusl-x64": {
1179
+
"version": "0.34.5",
1180
+
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.34.5.tgz",
1181
+
"integrity": "sha512-Jg8wNT1MUzIvhBFxViqrEhWDGzqymo3sV7z7ZsaWbZNDLXRJZoRGrjulp60YYtV4wfY8VIKcWidjojlLcWrd8Q==",
1182
+
"cpu": [
1183
+
"x64"
1184
+
],
1185
+
"dev": true,
1186
+
"license": "Apache-2.0",
1187
+
"optional": true,
1188
+
"os": [
1189
+
"linux"
1190
+
],
1191
+
"engines": {
1192
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1193
+
},
1194
+
"funding": {
1195
+
"url": "https://opencollective.com/libvips"
1196
+
},
1197
+
"optionalDependencies": {
1198
+
"@img/sharp-libvips-linuxmusl-x64": "1.2.4"
1199
+
}
1200
+
},
1201
+
"node_modules/@img/sharp-wasm32": {
1202
+
"version": "0.34.5",
1203
+
"resolved": "https://registry.npmjs.org/@img/sharp-wasm32/-/sharp-wasm32-0.34.5.tgz",
1204
+
"integrity": "sha512-OdWTEiVkY2PHwqkbBI8frFxQQFekHaSSkUIJkwzclWZe64O1X4UlUjqqqLaPbUpMOQk6FBu/HtlGXNblIs0huw==",
1205
+
"cpu": [
1206
+
"wasm32"
1207
+
],
1208
+
"dev": true,
1209
+
"license": "Apache-2.0 AND LGPL-3.0-or-later AND MIT",
1210
+
"optional": true,
1211
+
"dependencies": {
1212
+
"@emnapi/runtime": "^1.7.0"
1213
+
},
1214
+
"engines": {
1215
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1216
+
},
1217
+
"funding": {
1218
+
"url": "https://opencollective.com/libvips"
1219
+
}
1220
+
},
1221
+
"node_modules/@img/sharp-win32-arm64": {
1222
+
"version": "0.34.5",
1223
+
"resolved": "https://registry.npmjs.org/@img/sharp-win32-arm64/-/sharp-win32-arm64-0.34.5.tgz",
1224
+
"integrity": "sha512-WQ3AgWCWYSb2yt+IG8mnC6Jdk9Whs7O0gxphblsLvdhSpSTtmu69ZG1Gkb6NuvxsNACwiPV6cNSZNzt0KPsw7g==",
1225
+
"cpu": [
1226
+
"arm64"
1227
+
],
1228
+
"dev": true,
1229
+
"license": "Apache-2.0 AND LGPL-3.0-or-later",
1230
+
"optional": true,
1231
+
"os": [
1232
+
"win32"
1233
+
],
1234
+
"engines": {
1235
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1236
+
},
1237
+
"funding": {
1238
+
"url": "https://opencollective.com/libvips"
1239
+
}
1240
+
},
1241
+
"node_modules/@img/sharp-win32-ia32": {
1242
+
"version": "0.34.5",
1243
+
"resolved": "https://registry.npmjs.org/@img/sharp-win32-ia32/-/sharp-win32-ia32-0.34.5.tgz",
1244
+
"integrity": "sha512-FV9m/7NmeCmSHDD5j4+4pNI8Cp3aW+JvLoXcTUo0IqyjSfAZJ8dIUmijx1qaJsIiU+Hosw6xM5KijAWRJCSgNg==",
1245
+
"cpu": [
1246
+
"ia32"
1247
+
],
1248
+
"dev": true,
1249
+
"license": "Apache-2.0 AND LGPL-3.0-or-later",
1250
+
"optional": true,
1251
+
"os": [
1252
+
"win32"
1253
+
],
1254
+
"engines": {
1255
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1256
+
},
1257
+
"funding": {
1258
+
"url": "https://opencollective.com/libvips"
1259
+
}
1260
+
},
1261
+
"node_modules/@img/sharp-win32-x64": {
1262
+
"version": "0.34.5",
1263
+
"resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.34.5.tgz",
1264
+
"integrity": "sha512-+29YMsqY2/9eFEiW93eqWnuLcWcufowXewwSNIT6UwZdUUCrM3oFjMWH/Z6/TMmb4hlFenmfAVbpWeup2jryCw==",
1265
+
"cpu": [
1266
+
"x64"
1267
+
],
1268
+
"dev": true,
1269
+
"license": "Apache-2.0 AND LGPL-3.0-or-later",
1270
+
"optional": true,
1271
+
"os": [
1272
+
"win32"
1273
+
],
1274
+
"engines": {
1275
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1276
+
},
1277
+
"funding": {
1278
+
"url": "https://opencollective.com/libvips"
1279
+
}
1280
+
},
1281
+
"node_modules/@jridgewell/resolve-uri": {
1282
+
"version": "3.1.2",
1283
+
"resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz",
1284
+
"integrity": "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==",
1285
+
"dev": true,
1286
+
"license": "MIT",
1287
+
"engines": {
1288
+
"node": ">=6.0.0"
1289
+
}
1290
+
},
1291
+
"node_modules/@jridgewell/sourcemap-codec": {
1292
+
"version": "1.5.5",
1293
+
"resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz",
1294
+
"integrity": "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==",
1295
+
"dev": true,
1296
+
"license": "MIT"
1297
+
},
1298
+
"node_modules/@jridgewell/trace-mapping": {
1299
+
"version": "0.3.9",
1300
+
"resolved": "https://registry.npmjs.org/@jridgewell/trace-mapping/-/trace-mapping-0.3.9.tgz",
1301
+
"integrity": "sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ==",
1302
+
"dev": true,
1303
+
"license": "MIT",
1304
+
"dependencies": {
1305
+
"@jridgewell/resolve-uri": "^3.0.3",
1306
+
"@jridgewell/sourcemap-codec": "^1.4.10"
1307
+
}
1308
+
},
1309
+
"node_modules/@noble/secp256k1": {
1310
+
"version": "3.1.0",
1311
+
"resolved": "https://registry.npmjs.org/@noble/secp256k1/-/secp256k1-3.1.0.tgz",
1312
+
"integrity": "sha512-+F7iS7tUMaNGXcc9X3PjmjvuQnXEuSjCRNzVVA2xAcKXgCaP0dHYz4SFyt4FKNHef7sOP//xihowcySSS7PK9g==",
1313
+
"license": "MIT",
1314
+
"funding": {
1315
+
"url": "https://paulmillr.com/funding/"
1316
+
}
1317
+
},
1318
+
"node_modules/@poppinss/colors": {
1319
+
"version": "4.1.6",
1320
+
"resolved": "https://registry.npmjs.org/@poppinss/colors/-/colors-4.1.6.tgz",
1321
+
"integrity": "sha512-H9xkIdFswbS8n1d6vmRd8+c10t2Qe+rZITbbDHHkQixH5+2x1FDGmi/0K+WgWiqQFKPSlIYB7jlH6Kpfn6Fleg==",
1322
+
"dev": true,
1323
+
"license": "MIT",
1324
+
"dependencies": {
1325
+
"kleur": "^4.1.5"
1326
+
}
1327
+
},
1328
+
"node_modules/@poppinss/dumper": {
1329
+
"version": "0.6.5",
1330
+
"resolved": "https://registry.npmjs.org/@poppinss/dumper/-/dumper-0.6.5.tgz",
1331
+
"integrity": "sha512-NBdYIb90J7LfOI32dOewKI1r7wnkiH6m920puQ3qHUeZkxNkQiFnXVWoE6YtFSv6QOiPPf7ys6i+HWWecDz7sw==",
1332
+
"dev": true,
1333
+
"license": "MIT",
1334
+
"dependencies": {
1335
+
"@poppinss/colors": "^4.1.5",
1336
+
"@sindresorhus/is": "^7.0.2",
1337
+
"supports-color": "^10.0.0"
1338
+
}
1339
+
},
1340
+
"node_modules/@poppinss/exception": {
1341
+
"version": "1.2.3",
1342
+
"resolved": "https://registry.npmjs.org/@poppinss/exception/-/exception-1.2.3.tgz",
1343
+
"integrity": "sha512-dCED+QRChTVatE9ibtoaxc+WkdzOSjYTKi/+uacHWIsfodVfpsueo3+DKpgU5Px8qXjgmXkSvhXvSCz3fnP9lw==",
1344
+
"dev": true,
1345
+
"license": "MIT"
1346
+
},
1347
+
"node_modules/@sindresorhus/is": {
1348
+
"version": "7.2.0",
1349
+
"resolved": "https://registry.npmjs.org/@sindresorhus/is/-/is-7.2.0.tgz",
1350
+
"integrity": "sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw==",
1351
+
"dev": true,
1352
+
"license": "MIT",
1353
+
"engines": {
1354
+
"node": ">=18"
1355
+
},
1356
+
"funding": {
1357
+
"url": "https://github.com/sindresorhus/is?sponsor=1"
1358
+
}
1359
+
},
1360
+
"node_modules/@speed-highlight/core": {
1361
+
"version": "1.2.17",
1362
+
"resolved": "https://registry.npmjs.org/@speed-highlight/core/-/core-1.2.17.tgz",
1363
+
"integrity": "sha512-Z92FwKpCtfaW1V0jTU/fh3QzYEZN8wDwrzRIBoADCJfn4mJCNcJN/XegifX7BDrQ8/h9Xh/JnbyMchL0FqXrkg==",
1364
+
"dev": true,
1365
+
"license": "CC0-1.0"
1366
+
},
1367
+
"node_modules/@standard-schema/spec": {
1368
+
"version": "1.1.0",
1369
+
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
1370
+
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
1371
+
"license": "MIT"
1372
+
},
1373
+
"node_modules/blake3-wasm": {
1374
+
"version": "2.1.5",
1375
+
"resolved": "https://registry.npmjs.org/blake3-wasm/-/blake3-wasm-2.1.5.tgz",
1376
+
"integrity": "sha512-F1+K8EbfOZE49dtoPtmxUQrpXaBIl3ICvasLh+nJta0xkz+9kF/7uet9fLnwKqhDrmj6g+6K3Tw9yQPUg2ka5g==",
1377
+
"dev": true,
1378
+
"license": "MIT"
1379
+
},
1380
+
"node_modules/cookie": {
1381
+
"version": "1.1.1",
1382
+
"resolved": "https://registry.npmjs.org/cookie/-/cookie-1.1.1.tgz",
1383
+
"integrity": "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==",
1384
+
"dev": true,
1385
+
"license": "MIT",
1386
+
"engines": {
1387
+
"node": ">=18"
1388
+
},
1389
+
"funding": {
1390
+
"type": "opencollective",
1391
+
"url": "https://opencollective.com/express"
1392
+
}
1393
+
},
1394
+
"node_modules/detect-libc": {
1395
+
"version": "2.1.2",
1396
+
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
1397
+
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
1398
+
"dev": true,
1399
+
"license": "Apache-2.0",
1400
+
"engines": {
1401
+
"node": ">=8"
1402
+
}
1403
+
},
1404
+
"node_modules/error-stack-parser-es": {
1405
+
"version": "1.0.5",
1406
+
"resolved": "https://registry.npmjs.org/error-stack-parser-es/-/error-stack-parser-es-1.0.5.tgz",
1407
+
"integrity": "sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==",
1408
+
"dev": true,
1409
+
"license": "MIT",
1410
+
"funding": {
1411
+
"url": "https://github.com/sponsors/antfu"
1412
+
}
1413
+
},
1414
+
"node_modules/esbuild": {
1415
+
"version": "0.28.1",
1416
+
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz",
1417
+
"integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==",
1418
+
"dev": true,
1419
+
"hasInstallScript": true,
1420
+
"license": "MIT",
1421
+
"bin": {
1422
+
"esbuild": "bin/esbuild"
1423
+
},
1424
+
"engines": {
1425
+
"node": ">=18"
1426
+
},
1427
+
"optionalDependencies": {
1428
+
"@esbuild/aix-ppc64": "0.28.1",
1429
+
"@esbuild/android-arm": "0.28.1",
1430
+
"@esbuild/android-arm64": "0.28.1",
1431
+
"@esbuild/android-x64": "0.28.1",
1432
+
"@esbuild/darwin-arm64": "0.28.1",
1433
+
"@esbuild/darwin-x64": "0.28.1",
1434
+
"@esbuild/freebsd-arm64": "0.28.1",
1435
+
"@esbuild/freebsd-x64": "0.28.1",
1436
+
"@esbuild/linux-arm": "0.28.1",
1437
+
"@esbuild/linux-arm64": "0.28.1",
1438
+
"@esbuild/linux-ia32": "0.28.1",
1439
+
"@esbuild/linux-loong64": "0.28.1",
1440
+
"@esbuild/linux-mips64el": "0.28.1",
1441
+
"@esbuild/linux-ppc64": "0.28.1",
1442
+
"@esbuild/linux-riscv64": "0.28.1",
1443
+
"@esbuild/linux-s390x": "0.28.1",
1444
+
"@esbuild/linux-x64": "0.28.1",
1445
+
"@esbuild/netbsd-arm64": "0.28.1",
1446
+
"@esbuild/netbsd-x64": "0.28.1",
1447
+
"@esbuild/openbsd-arm64": "0.28.1",
1448
+
"@esbuild/openbsd-x64": "0.28.1",
1449
+
"@esbuild/openharmony-arm64": "0.28.1",
1450
+
"@esbuild/sunos-x64": "0.28.1",
1451
+
"@esbuild/win32-arm64": "0.28.1",
1452
+
"@esbuild/win32-ia32": "0.28.1",
1453
+
"@esbuild/win32-x64": "0.28.1"
1454
+
}
1455
+
},
1456
+
"node_modules/esm-env": {
1457
+
"version": "1.2.2",
1458
+
"resolved": "https://registry.npmjs.org/esm-env/-/esm-env-1.2.2.tgz",
1459
+
"integrity": "sha512-Epxrv+Nr/CaL4ZcFGPJIYLWFom+YeV1DqMLHJoEd9SYRxNbaFruBwfEX/kkHUJf55j2+TUbmDcmuilbP1TmXHA==",
1460
+
"license": "MIT"
1461
+
},
1462
+
"node_modules/fsevents": {
1463
+
"version": "2.3.3",
1464
+
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
1465
+
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
1466
+
"dev": true,
1467
+
"hasInstallScript": true,
1468
+
"license": "MIT",
1469
+
"optional": true,
1470
+
"os": [
1471
+
"darwin"
1472
+
],
1473
+
"engines": {
1474
+
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
1475
+
}
1476
+
},
1477
+
"node_modules/kleur": {
1478
+
"version": "4.1.5",
1479
+
"resolved": "https://registry.npmjs.org/kleur/-/kleur-4.1.5.tgz",
1480
+
"integrity": "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==",
1481
+
"dev": true,
1482
+
"license": "MIT",
1483
+
"engines": {
1484
+
"node": ">=6"
1485
+
}
1486
+
},
1487
+
"node_modules/miniflare": {
1488
+
"version": "4.20260625.0",
1489
+
"resolved": "https://registry.npmjs.org/miniflare/-/miniflare-4.20260625.0.tgz",
1490
+
"integrity": "sha512-3kKXwRUObJsnBYPBgR0NiNZYKF/yv8GFyha1cx2EeAEraxNODgRVcyeRo+F1ok1tg5Mg7iUpOWSkknQTHuFhwA==",
1491
+
"dev": true,
1492
+
"license": "MIT",
1493
+
"dependencies": {
1494
+
"@cspotcode/source-map-support": "0.8.1",
1495
+
"sharp": "0.34.5",
1496
+
"undici": "7.28.0",
1497
+
"workerd": "1.20260625.1",
1498
+
"ws": "8.21.0",
1499
+
"youch": "4.1.0-beta.10"
1500
+
},
1501
+
"bin": {
1502
+
"miniflare": "bootstrap.js"
1503
+
},
1504
+
"engines": {
1505
+
"node": ">=22.0.0"
1506
+
}
1507
+
},
1508
+
"node_modules/path-to-regexp": {
1509
+
"version": "6.3.0",
1510
+
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-6.3.0.tgz",
1511
+
"integrity": "sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==",
1512
+
"dev": true,
1513
+
"license": "MIT"
1514
+
},
1515
+
"node_modules/pathe": {
1516
+
"version": "2.0.3",
1517
+
"resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz",
1518
+
"integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==",
1519
+
"dev": true,
1520
+
"license": "MIT"
1521
+
},
1522
+
"node_modules/semver": {
1523
+
"version": "7.8.5",
1524
+
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
1525
+
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
1526
+
"dev": true,
1527
+
"license": "ISC",
1528
+
"bin": {
1529
+
"semver": "bin/semver.js"
1530
+
},
1531
+
"engines": {
1532
+
"node": ">=10"
1533
+
}
1534
+
},
1535
+
"node_modules/sharp": {
1536
+
"version": "0.34.5",
1537
+
"resolved": "https://registry.npmjs.org/sharp/-/sharp-0.34.5.tgz",
1538
+
"integrity": "sha512-Ou9I5Ft9WNcCbXrU9cMgPBcCK8LiwLqcbywW3t4oDV37n1pzpuNLsYiAV8eODnjbtQlSDwZ2cUEeQz4E54Hltg==",
1539
+
"dev": true,
1540
+
"hasInstallScript": true,
1541
+
"license": "Apache-2.0",
1542
+
"dependencies": {
1543
+
"@img/colour": "^1.0.0",
1544
+
"detect-libc": "^2.1.2",
1545
+
"semver": "^7.7.3"
1546
+
},
1547
+
"engines": {
1548
+
"node": "^18.17.0 || ^20.3.0 || >=21.0.0"
1549
+
},
1550
+
"funding": {
1551
+
"url": "https://opencollective.com/libvips"
1552
+
},
1553
+
"optionalDependencies": {
1554
+
"@img/sharp-darwin-arm64": "0.34.5",
1555
+
"@img/sharp-darwin-x64": "0.34.5",
1556
+
"@img/sharp-libvips-darwin-arm64": "1.2.4",
1557
+
"@img/sharp-libvips-darwin-x64": "1.2.4",
1558
+
"@img/sharp-libvips-linux-arm": "1.2.4",
1559
+
"@img/sharp-libvips-linux-arm64": "1.2.4",
1560
+
"@img/sharp-libvips-linux-ppc64": "1.2.4",
1561
+
"@img/sharp-libvips-linux-riscv64": "1.2.4",
1562
+
"@img/sharp-libvips-linux-s390x": "1.2.4",
1563
+
"@img/sharp-libvips-linux-x64": "1.2.4",
1564
+
"@img/sharp-libvips-linuxmusl-arm64": "1.2.4",
1565
+
"@img/sharp-libvips-linuxmusl-x64": "1.2.4",
1566
+
"@img/sharp-linux-arm": "0.34.5",
1567
+
"@img/sharp-linux-arm64": "0.34.5",
1568
+
"@img/sharp-linux-ppc64": "0.34.5",
1569
+
"@img/sharp-linux-riscv64": "0.34.5",
1570
+
"@img/sharp-linux-s390x": "0.34.5",
1571
+
"@img/sharp-linux-x64": "0.34.5",
1572
+
"@img/sharp-linuxmusl-arm64": "0.34.5",
1573
+
"@img/sharp-linuxmusl-x64": "0.34.5",
1574
+
"@img/sharp-wasm32": "0.34.5",
1575
+
"@img/sharp-win32-arm64": "0.34.5",
1576
+
"@img/sharp-win32-ia32": "0.34.5",
1577
+
"@img/sharp-win32-x64": "0.34.5"
1578
+
}
1579
+
},
1580
+
"node_modules/supports-color": {
1581
+
"version": "10.2.2",
1582
+
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-10.2.2.tgz",
1583
+
"integrity": "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g==",
1584
+
"dev": true,
1585
+
"license": "MIT",
1586
+
"engines": {
1587
+
"node": ">=18"
1588
+
},
1589
+
"funding": {
1590
+
"url": "https://github.com/chalk/supports-color?sponsor=1"
1591
+
}
1592
+
},
1593
+
"node_modules/tslib": {
1594
+
"version": "2.8.1",
1595
+
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
1596
+
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
1597
+
"dev": true,
1598
+
"license": "0BSD",
1599
+
"optional": true
1600
+
},
1601
+
"node_modules/undici": {
1602
+
"version": "7.28.0",
1603
+
"resolved": "https://registry.npmjs.org/undici/-/undici-7.28.0.tgz",
1604
+
"integrity": "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==",
1605
+
"dev": true,
1606
+
"license": "MIT",
1607
+
"engines": {
1608
+
"node": ">=20.18.1"
1609
+
}
1610
+
},
1611
+
"node_modules/unenv": {
1612
+
"version": "2.0.0-rc.24",
1613
+
"resolved": "https://registry.npmjs.org/unenv/-/unenv-2.0.0-rc.24.tgz",
1614
+
"integrity": "sha512-i7qRCmY42zmCwnYlh9H2SvLEypEFGye5iRmEMKjcGi7zk9UquigRjFtTLz0TYqr0ZGLZhaMHl/foy1bZR+Cwlw==",
1615
+
"dev": true,
1616
+
"license": "MIT",
1617
+
"dependencies": {
1618
+
"pathe": "^2.0.3"
1619
+
}
1620
+
},
1621
+
"node_modules/unicode-segmenter": {
1622
+
"version": "0.14.5",
1623
+
"resolved": "https://registry.npmjs.org/unicode-segmenter/-/unicode-segmenter-0.14.5.tgz",
1624
+
"integrity": "sha512-jHGmj2LUuqDcX3hqY12Ql+uhUTn8huuxNZGq7GvtF6bSybzH3aFgedYu/KTzQStEgt1Ra2F3HxadNXsNjb3m3g==",
1625
+
"license": "MIT"
1626
+
},
1627
+
"node_modules/valibot": {
1628
+
"version": "1.4.2",
1629
+
"resolved": "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz",
1630
+
"integrity": "sha512-gjdCvJ6d3RyHAneqxMYMW9QMCwYMb3jpOO0IyHZV1bnRHFBHrX3VkIILt5XYR0WhwHiH7Mty8ovuPZ/O3gamrg==",
1631
+
"license": "MIT",
1632
+
"peerDependencies": {
1633
+
"typescript": ">=5"
1634
+
},
1635
+
"peerDependenciesMeta": {
1636
+
"typescript": {
1637
+
"optional": true
1638
+
}
1639
+
}
1640
+
},
1641
+
"node_modules/workerd": {
1642
+
"version": "1.20260625.1",
1643
+
"resolved": "https://registry.npmjs.org/workerd/-/workerd-1.20260625.1.tgz",
1644
+
"integrity": "sha512-GApQvFX52SDM6L4u0+RRnUDB1wJOnEwoXjinkmOPtIyofWBxrlZckdegJSYc1leg++lLZ3+DQ4zMVmBqYVtzfA==",
1645
+
"dev": true,
1646
+
"hasInstallScript": true,
1647
+
"license": "Apache-2.0",
1648
+
"bin": {
1649
+
"workerd": "bin/workerd"
1650
+
},
1651
+
"engines": {
1652
+
"node": ">=16"
1653
+
},
1654
+
"optionalDependencies": {
1655
+
"@cloudflare/workerd-darwin-64": "1.20260625.1",
1656
+
"@cloudflare/workerd-darwin-arm64": "1.20260625.1",
1657
+
"@cloudflare/workerd-linux-64": "1.20260625.1",
1658
+
"@cloudflare/workerd-linux-arm64": "1.20260625.1",
1659
+
"@cloudflare/workerd-windows-64": "1.20260625.1"
1660
+
}
1661
+
},
1662
+
"node_modules/wrangler": {
1663
+
"version": "4.105.0",
1664
+
"resolved": "https://registry.npmjs.org/wrangler/-/wrangler-4.105.0.tgz",
1665
+
"integrity": "sha512-7dXFH6OLj1Fv0y6ZeRPUxFTkp+duWD7/xxVi/1c0vfOeEYwIFKWB7cdqnY05DvY1Ta3BnqAwRkXfLs8PDj538g==",
1666
+
"dev": true,
1667
+
"license": "MIT OR Apache-2.0",
1668
+
"dependencies": {
1669
+
"@cloudflare/kv-asset-handler": "0.5.0",
1670
+
"@cloudflare/unenv-preset": "2.16.1",
1671
+
"blake3-wasm": "2.1.5",
1672
+
"esbuild": "0.28.1",
1673
+
"miniflare": "4.20260625.0",
1674
+
"path-to-regexp": "6.3.0",
1675
+
"unenv": "2.0.0-rc.24",
1676
+
"workerd": "1.20260625.1"
1677
+
},
1678
+
"bin": {
1679
+
"cf-wrangler": "bin/cf-wrangler.js",
1680
+
"wrangler": "bin/wrangler.js",
1681
+
"wrangler2": "bin/wrangler.js"
1682
+
},
1683
+
"engines": {
1684
+
"node": ">=22.0.0"
1685
+
},
1686
+
"optionalDependencies": {
1687
+
"fsevents": "2.3.3"
1688
+
},
1689
+
"peerDependencies": {
1690
+
"@cloudflare/workers-types": "^4.20260625.1"
1691
+
},
1692
+
"peerDependenciesMeta": {
1693
+
"@cloudflare/workers-types": {
1694
+
"optional": true
1695
+
}
1696
+
}
1697
+
},
1698
+
"node_modules/ws": {
1699
+
"version": "8.21.0",
1700
+
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz",
1701
+
"integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==",
1702
+
"dev": true,
1703
+
"license": "MIT",
1704
+
"engines": {
1705
+
"node": ">=10.0.0"
1706
+
},
1707
+
"peerDependencies": {
1708
+
"bufferutil": "^4.0.1",
1709
+
"utf-8-validate": ">=5.0.2"
1710
+
},
1711
+
"peerDependenciesMeta": {
1712
+
"bufferutil": {
1713
+
"optional": true
1714
+
},
1715
+
"utf-8-validate": {
1716
+
"optional": true
1717
+
}
1718
+
}
1719
+
},
1720
+
"node_modules/youch": {
1721
+
"version": "4.1.0-beta.10",
1722
+
"resolved": "https://registry.npmjs.org/youch/-/youch-4.1.0-beta.10.tgz",
1723
+
"integrity": "sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ==",
1724
+
"dev": true,
1725
+
"license": "MIT",
1726
+
"dependencies": {
1727
+
"@poppinss/colors": "^4.1.5",
1728
+
"@poppinss/dumper": "^0.6.4",
1729
+
"@speed-highlight/core": "^1.2.7",
1730
+
"cookie": "^1.0.2",
1731
+
"youch-core": "^0.3.3"
1732
+
}
1733
+
},
1734
+
"node_modules/youch-core": {
1735
+
"version": "0.3.3",
1736
+
"resolved": "https://registry.npmjs.org/youch-core/-/youch-core-0.3.3.tgz",
1737
+
"integrity": "sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA==",
1738
+
"dev": true,
1739
+
"license": "MIT",
1740
+
"dependencies": {
1741
+
"@poppinss/exception": "^1.2.2",
1742
+
"error-stack-parser-es": "^1.0.5"
1743
+
}
1744
+
}
1745
+
}
1746
+
}
+24
package.json
+24
package.json
···
1
+
{
2
+
"name": "pensieve",
3
+
"version": "0.1.0",
4
+
"private": true,
5
+
"type": "module",
6
+
"scripts": {
7
+
"dev": "wrangler dev",
8
+
"deploy": "wrangler deploy",
9
+
"dry-run": "wrangler deploy --dry-run"
10
+
},
11
+
"devDependencies": {
12
+
"wrangler": "^4.99.0"
13
+
},
14
+
"dependencies": {
15
+
"@atcute/atproto": "^4.0.3",
16
+
"@atcute/car": "^6.0.2",
17
+
"@atcute/cbor": "^2.3.5",
18
+
"@atcute/cid": "^2.4.2",
19
+
"@atcute/client": "^5.1.0",
20
+
"@atcute/identity-resolver": "^2.0.1",
21
+
"@atcute/mst": "^1.0.2",
22
+
"@atcute/repo": "^1.0.2"
23
+
}
24
+
}
+149
scripts/render_demo_report.py
+149
scripts/render_demo_report.py
···
1
+
#!/usr/bin/env python3
2
+
"""Render the Pensieve demo as a user-facing inspection page."""
3
+
4
+
from __future__ import annotations
5
+
6
+
import html
7
+
import json
8
+
from pathlib import Path
9
+
10
+
11
+
BASE = Path("/tmp/pensieve-demo")
12
+
13
+
14
+
def h(value: object) -> str:
15
+
return html.escape(str(value), quote=True)
16
+
17
+
18
+
def load(name: str):
19
+
return json.loads((BASE / name).read_text())
20
+
21
+
22
+
def stat(label: str, value: object) -> str:
23
+
return f"<div class='metric'><b>{h(value)}</b><span>{h(label)}</span></div>"
24
+
25
+
26
+
def result(row: dict) -> str:
27
+
text = row.get("text", "")
28
+
if len(text) > 1200:
29
+
text = text[:1200] + "\n..."
30
+
return f"""
31
+
<article class="result">
32
+
<div class="result-top">
33
+
<code>{h(row.get("collection", ""))}</code>
34
+
<span>distance {float(row.get("$dist", 0)):.3f}</span>
35
+
</div>
36
+
<a href="https://bsky.app/profile/{h(row.get("authorDid", ""))}/post/{h(row.get("uri", "").rsplit("/", 1)[-1])}">{h(row.get("uri", ""))}</a>
37
+
<pre>{h(text)}</pre>
38
+
</article>
39
+
"""
40
+
41
+
42
+
def query_section(title: str, filename: str) -> str:
43
+
data = load(filename)
44
+
rows = "\n".join(result(row) for row in data["rows"])
45
+
return f"""
46
+
<section>
47
+
<div class="section-head">
48
+
<h2>{h(title)}</h2>
49
+
<p>Semantic query: <code>{h(data["query"])}</code></p>
50
+
</div>
51
+
{rows}
52
+
</section>
53
+
"""
54
+
55
+
56
+
def main() -> None:
57
+
manifest = load("manifest.json")
58
+
top_collections = "".join(
59
+
f"<li><span>{h(row['collection'])}</span><b>{h(row['count'])}</b></li>"
60
+
for row in manifest["topCollections"][:10]
61
+
)
62
+
63
+
page = f"""<!doctype html>
64
+
<html lang="en">
65
+
<head>
66
+
<meta charset="utf-8">
67
+
<meta name="viewport" content="width=device-width, initial-scale=1">
68
+
<title>Pensieve</title>
69
+
<style>
70
+
:root {{
71
+
color-scheme: light dark;
72
+
font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
73
+
background: #f6f5f1;
74
+
color: #181816;
75
+
}}
76
+
body {{ margin: 0; background: #f6f5f1; }}
77
+
main {{ max-width: 1180px; margin: 0 auto; padding: 30px; }}
78
+
header {{ border-bottom: 1px solid #d8d5cc; padding: 8px 0 24px; }}
79
+
h1 {{ margin: 0; font-size: 42px; line-height: 1; letter-spacing: 0; }}
80
+
h2 {{ margin: 0; font-size: 21px; }}
81
+
p {{ margin: 8px 0 0; color: #5c5a52; }}
82
+
code {{ font-size: 12px; color: #57544e; }}
83
+
.metrics {{ display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 10px; margin: 22px 0; }}
84
+
.metric, .panel, .result {{ background: #fff; border: 1px solid #dedbd2; border-radius: 8px; }}
85
+
.metric {{ padding: 14px; }}
86
+
.metric b {{ display: block; font-size: 24px; }}
87
+
.metric span {{ display: block; margin-top: 3px; color: #6b675f; font-size: 12px; text-transform: uppercase; }}
88
+
.layout {{ display: grid; grid-template-columns: minmax(260px, 340px) 1fr; gap: 18px; align-items: start; }}
89
+
.panel {{ padding: 16px; position: sticky; top: 20px; }}
90
+
.panel h2 {{ margin-bottom: 12px; }}
91
+
ul {{ list-style: none; padding: 0; margin: 0; }}
92
+
li {{ display: flex; justify-content: space-between; gap: 12px; padding: 8px 0; border-top: 1px solid #ece9df; }}
93
+
li:first-child {{ border-top: 0; }}
94
+
li span {{ overflow-wrap: anywhere; }}
95
+
section {{ margin-bottom: 30px; }}
96
+
.section-head {{ margin-bottom: 12px; }}
97
+
.result {{ padding: 14px; margin: 10px 0; }}
98
+
.result-top {{ display: flex; justify-content: space-between; gap: 14px; align-items: baseline; }}
99
+
.result-top span {{ color: #177062; font-variant-numeric: tabular-nums; }}
100
+
a {{ display: inline-block; margin-top: 8px; color: #17695d; overflow-wrap: anywhere; }}
101
+
pre {{ white-space: pre-wrap; overflow-wrap: anywhere; font-size: 13px; line-height: 1.45; margin: 10px 0 0; }}
102
+
.files {{ margin-top: 16px; color: #5c5a52; font-size: 13px; }}
103
+
@media (max-width: 820px) {{ .layout {{ grid-template-columns: 1fr; }} .panel {{ position: static; }} }}
104
+
@media (prefers-color-scheme: dark) {{
105
+
:root, body {{ background: #171717; color: #f2efe7; }}
106
+
header {{ border-color: #383630; }}
107
+
p, .files {{ color: #b9b3a6; }}
108
+
.metric, .panel, .result {{ background: #20201f; border-color: #383630; }}
109
+
code {{ color: #bdb7aa; }}
110
+
li {{ border-color: #35332d; }}
111
+
a, .result-top span {{ color: #65b8a8; }}
112
+
}}
113
+
</style>
114
+
</head>
115
+
<body>
116
+
<main>
117
+
<header>
118
+
<h1>Pensieve</h1>
119
+
<p>ATProto memory search over an extracted repository corpus.</p>
120
+
</header>
121
+
<div class="metrics">
122
+
{stat("repo records", f"{manifest['records']:,}")}
123
+
{stat("extracted memories", f"{manifest['items']:,}")}
124
+
{stat("indexed sample", "200")}
125
+
{stat("CAR size", f"{manifest['carBytes']:,} bytes")}
126
+
</div>
127
+
<div class="layout">
128
+
<aside class="panel">
129
+
<h2>Corpus</h2>
130
+
<p><code>{h(manifest['did'])}</code></p>
131
+
<p>{h(manifest['pds'])}</p>
132
+
<h2 style="margin-top:22px">Top Collections</h2>
133
+
<ul>{top_collections}</ul>
134
+
<p class="files">Files: <code>corpus.ndjson</code>, <code>manifest.json</code>, and query result JSON in <code>/tmp/pensieve-demo</code>.</p>
135
+
</aside>
136
+
<div>
137
+
{query_section("Structured Outputs Lineage", "query-structured-outputs.json")}
138
+
{query_section("ATProto Repository Search", "query-atproto-search.json")}
139
+
</div>
140
+
</div>
141
+
</main>
142
+
</body>
143
+
</html>
144
+
"""
145
+
(BASE / "search.html").write_text(page)
146
+
147
+
148
+
if __name__ == "__main__":
149
+
main()
+165
scripts/tpuf_demo.py
+165
scripts/tpuf_demo.py
···
1
+
#!/usr/bin/env python3
2
+
"""Embed Pensieve NDJSON with Voyage and store/query it in Turbopuffer."""
3
+
4
+
from __future__ import annotations
5
+
6
+
import argparse
7
+
import hashlib
8
+
import json
9
+
import os
10
+
import sys
11
+
import urllib.error
12
+
import urllib.request
13
+
from pathlib import Path
14
+
from typing import Any
15
+
16
+
17
+
VOYAGE_URL = "https://api.voyageai.com/v1/embeddings"
18
+
TPUF_BASE = "https://api.turbopuffer.com/v2/namespaces"
19
+
MODEL = "voyage-4-lite"
20
+
DIMENSIONS = 1024
21
+
22
+
23
+
def env(name: str) -> str:
24
+
value = os.environ.get(name)
25
+
if not value:
26
+
raise SystemExit(f"missing {name}")
27
+
return value
28
+
29
+
30
+
def post_json(url: str, body: dict[str, Any], headers: dict[str, str]) -> dict[str, Any]:
31
+
payload = json.dumps(body, separators=(",", ":")).encode()
32
+
request = urllib.request.Request(
33
+
url,
34
+
data=payload,
35
+
headers={"content-type": "application/json", **headers},
36
+
method="POST",
37
+
)
38
+
try:
39
+
with urllib.request.urlopen(request, timeout=60) as response:
40
+
return json.loads(response.read())
41
+
except urllib.error.HTTPError as exc:
42
+
detail = exc.read().decode(errors="replace")[:500]
43
+
raise SystemExit(f"{url} failed: HTTP {exc.code}: {detail}") from exc
44
+
45
+
46
+
def embed(texts: list[str], input_type: str) -> list[list[float]]:
47
+
voyage_key = env("VOYAGE_API_KEY")
48
+
response = post_json(
49
+
VOYAGE_URL,
50
+
{
51
+
"model": MODEL,
52
+
"input_type": input_type,
53
+
"output_dimension": DIMENSIONS,
54
+
"input": texts,
55
+
},
56
+
{"authorization": f"Bearer {voyage_key}"},
57
+
)
58
+
return [row["embedding"] for row in response["data"]]
59
+
60
+
61
+
def row_id(uri: str) -> str:
62
+
return hashlib.sha256(uri.encode()).hexdigest()[:32]
63
+
64
+
65
+
def load_items(path: Path, limit: int) -> list[dict[str, Any]]:
66
+
items: list[dict[str, Any]] = []
67
+
with path.open() as f:
68
+
for line in f:
69
+
item = json.loads(line)
70
+
if item.get("kind") != "text" or not item.get("text"):
71
+
continue
72
+
items.append(item)
73
+
if len(items) >= limit:
74
+
break
75
+
return items
76
+
77
+
78
+
def chunks(items: list[dict[str, Any]], size: int):
79
+
for i in range(0, len(items), size):
80
+
yield items[i : i + size]
81
+
82
+
83
+
def tpuf_url(namespace: str, suffix: str = "") -> str:
84
+
return f"{TPUF_BASE}/{namespace}{suffix}"
85
+
86
+
87
+
def upsert(namespace: str, items: list[dict[str, Any]], batch_size: int) -> None:
88
+
tpuf_key = env("TURBOPUFFER_API_KEY")
89
+
total = 0
90
+
for batch in chunks(items, batch_size):
91
+
vectors = embed([item["text"] for item in batch], "document")
92
+
rows = []
93
+
for item, vector in zip(batch, vectors, strict=True):
94
+
rows.append(
95
+
{
96
+
"id": row_id(item["uri"]),
97
+
"vector": vector,
98
+
"uri": item["uri"],
99
+
"cid": item["cid"],
100
+
"collection": item["collection"],
101
+
"authorDid": item["authorDid"],
102
+
"createdAt": item.get("createdAt", ""),
103
+
"text": item["text"][:800],
104
+
}
105
+
)
106
+
post_json(
107
+
tpuf_url(namespace),
108
+
{"distance_metric": "cosine_distance", "upsert_rows": rows},
109
+
{"authorization": f"Bearer {tpuf_key}"},
110
+
)
111
+
total += len(rows)
112
+
print(f"upserted {total}/{len(items)}", file=sys.stderr)
113
+
114
+
115
+
def query(namespace: str, text: str, top_k: int) -> list[dict[str, Any]]:
116
+
tpuf_key = env("TURBOPUFFER_API_KEY")
117
+
vector = embed([text], "query")[0]
118
+
response = post_json(
119
+
tpuf_url(namespace, "/query"),
120
+
{
121
+
"rank_by": ["vector", "ANN", vector],
122
+
"top_k": top_k,
123
+
"include_attributes": ["uri", "collection", "authorDid", "createdAt", "text"],
124
+
},
125
+
{"authorization": f"Bearer {tpuf_key}"},
126
+
)
127
+
return response.get("rows", [])
128
+
129
+
130
+
def cmd_index(args: argparse.Namespace) -> None:
131
+
items = load_items(Path(args.corpus), args.limit)
132
+
if not items:
133
+
raise SystemExit("no text items found")
134
+
upsert(args.namespace, items, args.batch_size)
135
+
print(json.dumps({"namespace": args.namespace, "indexed": len(items)}, indent=2))
136
+
137
+
138
+
def cmd_query(args: argparse.Namespace) -> None:
139
+
rows = query(args.namespace, args.text, args.top_k)
140
+
print(json.dumps({"namespace": args.namespace, "query": args.text, "rows": rows}, indent=2))
141
+
142
+
143
+
def main() -> None:
144
+
parser = argparse.ArgumentParser()
145
+
sub = parser.add_subparsers(dest="command", required=True)
146
+
147
+
index = sub.add_parser("index")
148
+
index.add_argument("--corpus", required=True)
149
+
index.add_argument("--namespace", default=os.environ.get("TURBOPUFFER_NAMESPACE", "pensieve-demo"))
150
+
index.add_argument("--limit", type=int, default=200)
151
+
index.add_argument("--batch-size", type=int, default=32)
152
+
index.set_defaults(func=cmd_index)
153
+
154
+
q = sub.add_parser("query")
155
+
q.add_argument("--namespace", default=os.environ.get("TURBOPUFFER_NAMESPACE", "pensieve-demo"))
156
+
q.add_argument("--text", required=True)
157
+
q.add_argument("--top-k", type=int, default=5)
158
+
q.set_defaults(func=cmd_query)
159
+
160
+
args = parser.parse_args()
161
+
args.func(args)
162
+
163
+
164
+
if __name__ == "__main__":
165
+
main()
+362
src/main.zig
+362
src/main.zig
···
1
+
const std = @import("std");
2
+
const pensieve = @import("pensieve");
3
+
const zat = @import("zat");
4
+
5
+
const usage =
6
+
\\usage:
7
+
\\ pensieve extract --actor <handle-or-did> [--media] [--out <path>]
8
+
\\ pensieve demo --actor <handle-or-did> [--media] [--dir <path>]
9
+
\\
10
+
\\examples:
11
+
\\ pensieve extract --actor zzstoatzz.io
12
+
\\ pensieve extract --actor did:plc:... --media --out corpus.ndjson
13
+
\\ pensieve demo --actor zzstoatzz.io --dir /tmp/pensieve-demo
14
+
\\
15
+
;
16
+
17
+
pub fn main(init: std.process.Init) !void {
18
+
const args = try init.minimal.args.toSlice(init.arena.allocator());
19
+
20
+
var arena_impl = std.heap.ArenaAllocator.init(init.gpa);
21
+
defer arena_impl.deinit();
22
+
const arena = arena_impl.allocator();
23
+
24
+
if (args.len < 2) return fail(usage);
25
+
const cmd = args[1];
26
+
const is_extract = std.mem.eql(u8, cmd, "extract");
27
+
const is_demo = std.mem.eql(u8, cmd, "demo");
28
+
if (!is_extract and !is_demo) return fail(usage);
29
+
30
+
var actor: ?[]const u8 = null;
31
+
var out_path: ?[]const u8 = null;
32
+
var demo_dir: []const u8 = "/tmp/pensieve-demo";
33
+
var options: pensieve.ExtractOptions = .{};
34
+
35
+
var i: usize = 2;
36
+
while (i < args.len) : (i += 1) {
37
+
const arg = args[i];
38
+
if (std.mem.eql(u8, arg, "--actor")) {
39
+
i += 1;
40
+
if (i >= args.len) return fail("missing value for --actor\n");
41
+
actor = args[i];
42
+
} else if (std.mem.eql(u8, arg, "--media")) {
43
+
options.include_media_refs = true;
44
+
} else if (std.mem.eql(u8, arg, "--out")) {
45
+
i += 1;
46
+
if (i >= args.len) return fail("missing value for --out\n");
47
+
out_path = args[i];
48
+
} else if (std.mem.eql(u8, arg, "--dir")) {
49
+
i += 1;
50
+
if (i >= args.len) return fail("missing value for --dir\n");
51
+
demo_dir = args[i];
52
+
} else if (std.mem.eql(u8, arg, "--help") or std.mem.eql(u8, arg, "-h")) {
53
+
return fail(usage);
54
+
} else {
55
+
return fail(usage);
56
+
}
57
+
}
58
+
59
+
const target = actor orelse return fail("missing --actor\n");
60
+
61
+
const io = init.io;
62
+
63
+
var transport = zat.HttpTransport.init(io, arena);
64
+
const identity = try pensieve.pds.resolveIdentity(arena, &transport, target);
65
+
var opened = try pensieve.repo_walk.openRepo(arena, io, identity.pds, identity.did);
66
+
defer opened.close(io);
67
+
68
+
const result = try pensieve.repo_walk.walkOpened(arena, &opened, identity.did, pensieve.filterFromOptions(options));
69
+
const items = try pensieve.extractRecords(arena, identity.did, result.records, options);
70
+
71
+
if (is_demo) {
72
+
try writeDemo(arena, io, demo_dir, target, identity, result, items, options);
73
+
} else {
74
+
var file: std.Io.File = if (out_path) |path|
75
+
try std.Io.Dir.createFileAbsolute(io, path, .{ .read = true, .truncate = true })
76
+
else
77
+
std.Io.File.stdout();
78
+
defer if (out_path != null) file.close(io);
79
+
80
+
var buf: [64 * 1024]u8 = undefined;
81
+
var writer = std.Io.File.Writer.initStreaming(file, io, &buf);
82
+
try writeNdjson(&writer.interface, items);
83
+
try writer.end();
84
+
}
85
+
86
+
std.log.info("extracted {d} items from {d} records ({d} CAR bytes)", .{
87
+
items.len,
88
+
result.records.len,
89
+
result.car_bytes,
90
+
});
91
+
}
92
+
93
+
fn writeDemo(
94
+
arena: std.mem.Allocator,
95
+
io: std.Io,
96
+
dir_path: []const u8,
97
+
actor: []const u8,
98
+
identity: pensieve.pds.Identity,
99
+
result: pensieve.repo_walk.WalkResult,
100
+
items: []const pensieve.EmbeddableItem,
101
+
options: pensieve.ExtractOptions,
102
+
) !void {
103
+
std.Io.Dir.createDirPath(std.Io.Dir.cwd(), io, dir_path) catch |err| switch (err) {
104
+
error.PathAlreadyExists => {},
105
+
else => return err,
106
+
};
107
+
108
+
const corpus_path = try std.fs.path.join(arena, &.{ dir_path, "corpus.ndjson" });
109
+
const manifest_path = try std.fs.path.join(arena, &.{ dir_path, "manifest.json" });
110
+
const report_path = try std.fs.path.join(arena, &.{ dir_path, "index.html" });
111
+
112
+
{
113
+
var file = try std.Io.Dir.createFileAbsolute(io, corpus_path, .{ .read = true, .truncate = true });
114
+
defer file.close(io);
115
+
var buf: [64 * 1024]u8 = undefined;
116
+
var writer = std.Io.File.Writer.initStreaming(file, io, &buf);
117
+
try writeNdjson(&writer.interface, items);
118
+
try writer.end();
119
+
}
120
+
121
+
const summary = try summarize(arena, items);
122
+
try writeManifest(io, manifest_path, actor, identity, result, items, summary, options);
123
+
try writeReport(io, report_path, actor, identity, result, items, summary, corpus_path, manifest_path, options);
124
+
125
+
std.debug.print("Pensieve demo ready:\n report: {s}\n corpus: {s}\n manifest: {s}\n", .{
126
+
report_path,
127
+
corpus_path,
128
+
manifest_path,
129
+
});
130
+
}
131
+
132
+
const CollectionCount = struct {
133
+
name: []const u8,
134
+
count: usize,
135
+
};
136
+
137
+
const Summary = struct {
138
+
text_items: usize = 0,
139
+
image_refs: usize = 0,
140
+
video_refs: usize = 0,
141
+
collections: []CollectionCount,
142
+
};
143
+
144
+
fn summarize(arena: std.mem.Allocator, items: []const pensieve.EmbeddableItem) !Summary {
145
+
var collections: std.ArrayList(CollectionCount) = .empty;
146
+
var summary: Summary = .{ .collections = &.{} };
147
+
for (items) |item| {
148
+
switch (item.kind) {
149
+
.text => summary.text_items += 1,
150
+
.image_ref => summary.image_refs += 1,
151
+
.video_ref => summary.video_refs += 1,
152
+
}
153
+
var found = false;
154
+
for (collections.items) |*entry| {
155
+
if (std.mem.eql(u8, entry.name, item.collection)) {
156
+
entry.count += 1;
157
+
found = true;
158
+
break;
159
+
}
160
+
}
161
+
if (!found) try collections.append(arena, .{ .name = item.collection, .count = 1 });
162
+
}
163
+
std.mem.sort(CollectionCount, collections.items, {}, struct {
164
+
fn lessThan(_: void, a: CollectionCount, b: CollectionCount) bool {
165
+
if (a.count == b.count) return std.mem.lessThan(u8, a.name, b.name);
166
+
return a.count > b.count;
167
+
}
168
+
}.lessThan);
169
+
summary.collections = try collections.toOwnedSlice(arena);
170
+
return summary;
171
+
}
172
+
173
+
fn writeManifest(
174
+
io: std.Io,
175
+
path: []const u8,
176
+
actor: []const u8,
177
+
identity: pensieve.pds.Identity,
178
+
result: pensieve.repo_walk.WalkResult,
179
+
items: []const pensieve.EmbeddableItem,
180
+
summary: Summary,
181
+
options: pensieve.ExtractOptions,
182
+
) !void {
183
+
var file = try std.Io.Dir.createFileAbsolute(io, path, .{ .read = true, .truncate = true });
184
+
defer file.close(io);
185
+
var buf: [16 * 1024]u8 = undefined;
186
+
var writer = std.Io.File.Writer.initStreaming(file, io, &buf);
187
+
var jw: std.json.Stringify = .{ .writer = &writer.interface };
188
+
try jw.beginObject();
189
+
try field(&jw, "actor", actor);
190
+
try field(&jw, "did", identity.did);
191
+
try field(&jw, "pds", identity.pds);
192
+
try jw.objectField("carBytes");
193
+
try jw.write(result.car_bytes);
194
+
try jw.objectField("records");
195
+
try jw.write(result.records.len);
196
+
try jw.objectField("items");
197
+
try jw.write(items.len);
198
+
try jw.objectField("textItems");
199
+
try jw.write(summary.text_items);
200
+
try jw.objectField("imageRefs");
201
+
try jw.write(summary.image_refs);
202
+
try jw.objectField("videoRefs");
203
+
try jw.write(summary.video_refs);
204
+
try jw.objectField("mediaRefsIncluded");
205
+
try jw.write(options.include_media_refs);
206
+
try jw.objectField("topCollections");
207
+
try jw.beginArray();
208
+
for (summary.collections[0..@min(summary.collections.len, 20)]) |entry| {
209
+
try jw.beginObject();
210
+
try field(&jw, "collection", entry.name);
211
+
try jw.objectField("count");
212
+
try jw.write(entry.count);
213
+
try jw.endObject();
214
+
}
215
+
try jw.endArray();
216
+
try jw.endObject();
217
+
try writer.interface.writeByte('\n');
218
+
try writer.end();
219
+
}
220
+
221
+
fn writeReport(
222
+
io: std.Io,
223
+
path: []const u8,
224
+
actor: []const u8,
225
+
identity: pensieve.pds.Identity,
226
+
result: pensieve.repo_walk.WalkResult,
227
+
items: []const pensieve.EmbeddableItem,
228
+
summary: Summary,
229
+
corpus_path: []const u8,
230
+
manifest_path: []const u8,
231
+
options: pensieve.ExtractOptions,
232
+
) !void {
233
+
var file = try std.Io.Dir.createFileAbsolute(io, path, .{ .read = true, .truncate = true });
234
+
defer file.close(io);
235
+
var buf: [64 * 1024]u8 = undefined;
236
+
var writer = std.Io.File.Writer.initStreaming(file, io, &buf);
237
+
const w = &writer.interface;
238
+
239
+
try w.writeAll(
240
+
\\<!doctype html><html lang="en"><head><meta charset="utf-8">
241
+
\\<meta name="viewport" content="width=device-width,initial-scale=1">
242
+
\\<title>Pensieve Demo</title>
243
+
\\<style>
244
+
\\:root{color-scheme:light dark;font-family:ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif}
245
+
\\body{margin:0;background:#f7f7f4;color:#171717}
246
+
\\main{max-width:1120px;margin:0 auto;padding:28px}
247
+
\\header{display:grid;gap:10px;padding:22px 0;border-bottom:1px solid #d8d6ce}
248
+
\\h1{font-size:34px;line-height:1.05;margin:0;font-weight:760}
249
+
\\h2{font-size:18px;margin:30px 0 12px}
250
+
\\p{margin:0;color:#4a4a44}
251
+
\\.stats{display:grid;grid-template-columns:repeat(auto-fit,minmax(150px,1fr));gap:10px;margin:22px 0}
252
+
\\.stat,.sample,.bar{background:#fff;border:1px solid #dedbd2;border-radius:8px}
253
+
\\.stat{padding:14px}.stat b{display:block;font-size:24px}.stat span{font-size:12px;color:#686861;text-transform:uppercase}
254
+
\\.bar{display:grid;grid-template-columns:minmax(160px,260px) 1fr auto;gap:10px;align-items:center;padding:9px 11px;margin:6px 0}
255
+
\\.meter{height:9px;background:#e7e3d8;border-radius:99px;overflow:hidden}.meter i{display:block;height:100%;background:#317c6f}
256
+
\\.sample{padding:14px;margin:10px 0}.sample code{font-size:12px;color:#555}.sample pre{white-space:pre-wrap;overflow-wrap:anywhere;font-size:13px;line-height:1.45;margin:10px 0 0;color:#222}
257
+
\\a{color:#17695d} @media (prefers-color-scheme:dark){body{background:#171717;color:#f1efe7}p{color:#beb8aa}.stat,.sample,.bar{background:#20201f;border-color:#383630}.meter{background:#33322d}.sample pre{color:#eee}.sample code{color:#bdb7aa}}
258
+
\\</style></head><body><main>
259
+
);
260
+
261
+
try w.writeAll("<header><h1>Pensieve</h1><p>");
262
+
try html(w, actor);
263
+
try w.writeAll(" -> ");
264
+
try html(w, identity.did);
265
+
try w.writeAll("</p><p>Raw extracted corpus ready for embedding and backfills.</p></header><section class=\"stats\">");
266
+
try stat(w, "CAR bytes", result.car_bytes);
267
+
try stat(w, "records", result.records.len);
268
+
try stat(w, "items", items.len);
269
+
try stat(w, "text", summary.text_items);
270
+
try stat(w, "image refs", summary.image_refs);
271
+
try stat(w, "video refs", summary.video_refs);
272
+
try w.writeAll("</section><p>Wrote <code>");
273
+
try html(w, corpus_path);
274
+
try w.writeAll("</code> and <code>");
275
+
try html(w, manifest_path);
276
+
try w.writeAll("</code>. Media refs ");
277
+
try w.writeAll(if (options.include_media_refs) "included." else "not included; run with --media to include blob references.");
278
+
try w.writeAll("</p><h2>Top Collections</h2>");
279
+
280
+
const max_count = if (summary.collections.len > 0) summary.collections[0].count else 1;
281
+
for (summary.collections[0..@min(summary.collections.len, 18)]) |entry| {
282
+
const pct = @max(@as(usize, 1), entry.count * 100 / max_count);
283
+
try w.writeAll("<div class=\"bar\"><code>");
284
+
try html(w, entry.name);
285
+
try w.writeAll("</code><div class=\"meter\"><i style=\"width:");
286
+
try w.print("{d}", .{pct});
287
+
try w.writeAll("%\"></i></div><b>");
288
+
try w.print("{d}", .{entry.count});
289
+
try w.writeAll("</b></div>");
290
+
}
291
+
292
+
try w.writeAll("<h2>Sample Items</h2>");
293
+
var shown: usize = 0;
294
+
for (items) |item| {
295
+
if (item.kind != .text or item.text == null) continue;
296
+
try w.writeAll("<article class=\"sample\"><code>");
297
+
try html(w, item.uri);
298
+
try w.writeAll("</code><pre>");
299
+
const text = item.text.?;
300
+
try html(w, text[0..@min(text.len, 900)]);
301
+
if (text.len > 900) try w.writeAll("\n...");
302
+
try w.writeAll("</pre></article>");
303
+
shown += 1;
304
+
if (shown >= 8) break;
305
+
}
306
+
307
+
try w.writeAll("</main></body></html>");
308
+
try writer.end();
309
+
}
310
+
311
+
fn stat(w: *std.Io.Writer, label: []const u8, value: usize) !void {
312
+
try w.writeAll("<div class=\"stat\"><b>");
313
+
try w.print("{d}", .{value});
314
+
try w.writeAll("</b><span>");
315
+
try html(w, label);
316
+
try w.writeAll("</span></div>");
317
+
}
318
+
319
+
fn html(w: *std.Io.Writer, text: []const u8) !void {
320
+
for (text) |c| switch (c) {
321
+
'&' => try w.writeAll("&"),
322
+
'<' => try w.writeAll("<"),
323
+
'>' => try w.writeAll(">"),
324
+
'"' => try w.writeAll("""),
325
+
'\'' => try w.writeAll("'"),
326
+
else => try w.writeByte(c),
327
+
};
328
+
}
329
+
330
+
fn writeNdjson(writer: *std.Io.Writer, items: []const pensieve.EmbeddableItem) !void {
331
+
for (items) |item| {
332
+
var jw: std.json.Stringify = .{ .writer = writer };
333
+
try jw.beginObject();
334
+
try field(&jw, "kind", item.kind.jsonString());
335
+
try field(&jw, "uri", item.uri);
336
+
try field(&jw, "cid", item.cid);
337
+
try field(&jw, "collection", item.collection);
338
+
try field(&jw, "authorDid", item.author_did);
339
+
if (item.created_at) |created_at| try field(&jw, "createdAt", created_at);
340
+
if (item.text) |text| try field(&jw, "text", text);
341
+
if (item.blob) |blob| {
342
+
try jw.objectField("blob");
343
+
try jw.beginObject();
344
+
try field(&jw, "cid", blob.cid);
345
+
if (blob.mime_type) |mime| try field(&jw, "mimeType", mime);
346
+
if (blob.alt) |alt| try field(&jw, "alt", alt);
347
+
try jw.endObject();
348
+
}
349
+
try jw.endObject();
350
+
try writer.writeByte('\n');
351
+
}
352
+
}
353
+
354
+
fn field(jw: *std.json.Stringify, name: []const u8, value: []const u8) !void {
355
+
try jw.objectField(name);
356
+
try jw.write(value);
357
+
}
358
+
359
+
fn fail(message: []const u8) !void {
360
+
std.debug.print("{s}", .{message});
361
+
return error.InvalidArgs;
362
+
}
+246
src/pds.zig
+246
src/pds.zig
···
1
+
//! thin PDS client: handle → DID, DID → PDS URL, describeRepo, listRecords.
2
+
//!
3
+
//! for v1 all reads go through bsky's public appview (resolveHandle) and
4
+
//! plc.directory (DID document). no auth required — we're only reading
5
+
//! public data to build an in-memory search index.
6
+
7
+
const std = @import("std");
8
+
const json = std.json;
9
+
const Allocator = std.mem.Allocator;
10
+
const zat = @import("zat");
11
+
12
+
const PUBLIC_APPVIEW = "https://public.api.bsky.app";
13
+
const PLC_DIRECTORY = "https://plc.directory";
14
+
15
+
pub const ResolveError = error{
16
+
ResolveFailed,
17
+
NoPdsEndpoint,
18
+
RequestFailed,
19
+
OutOfMemory,
20
+
ParseFailed,
21
+
};
22
+
23
+
pub const Identity = struct {
24
+
did: []const u8,
25
+
pds: []const u8,
26
+
};
27
+
28
+
/// resolve a handle or DID to {did, pds_url}. allocates into `arena`.
29
+
pub fn resolveIdentity(
30
+
arena: Allocator,
31
+
transport: *zat.HttpTransport,
32
+
actor: []const u8,
33
+
) ResolveError!Identity {
34
+
// step 1: if `actor` already looks like a DID, use it directly; otherwise
35
+
// resolve via bsky's public appview.
36
+
const did: []const u8 = if (std.mem.startsWith(u8, actor, "did:"))
37
+
try arena.dupe(u8, actor)
38
+
else
39
+
try resolveHandle(arena, transport, actor);
40
+
41
+
// step 2: DID document → PDS service endpoint
42
+
const pds = try resolvePdsFromDid(arena, transport, did);
43
+
return .{ .did = did, .pds = pds };
44
+
}
45
+
46
+
fn resolveHandle(
47
+
arena: Allocator,
48
+
transport: *zat.HttpTransport,
49
+
handle: []const u8,
50
+
) ResolveError![]const u8 {
51
+
const url = try std.fmt.allocPrint(
52
+
arena,
53
+
"{s}/xrpc/com.atproto.identity.resolveHandle?handle={s}",
54
+
.{ PUBLIC_APPVIEW, handle },
55
+
);
56
+
57
+
const result = transport.fetch(.{ .url = url }) catch return error.RequestFailed;
58
+
if (result.status != .ok) return error.ResolveFailed;
59
+
60
+
const parsed = json.parseFromSliceLeaky(json.Value, arena, result.body, .{}) catch
61
+
return error.ParseFailed;
62
+
63
+
const did_val = parsed.object.get("did") orelse return error.ResolveFailed;
64
+
if (did_val != .string) return error.ResolveFailed;
65
+
return try arena.dupe(u8, did_val.string);
66
+
}
67
+
68
+
fn resolvePdsFromDid(
69
+
arena: Allocator,
70
+
transport: *zat.HttpTransport,
71
+
did: []const u8,
72
+
) ResolveError![]const u8 {
73
+
const url = if (std.mem.startsWith(u8, did, "did:plc:"))
74
+
try std.fmt.allocPrint(arena, "{s}/{s}", .{ PLC_DIRECTORY, did })
75
+
else if (std.mem.startsWith(u8, did, "did:web:")) blk: {
76
+
const domain = did["did:web:".len..];
77
+
break :blk try std.fmt.allocPrint(arena, "https://{s}/.well-known/did.json", .{domain});
78
+
} else return error.ResolveFailed;
79
+
80
+
const result = transport.fetch(.{ .url = url }) catch return error.RequestFailed;
81
+
if (result.status != .ok) return error.ResolveFailed;
82
+
83
+
const parsed = json.parseFromSliceLeaky(json.Value, arena, result.body, .{}) catch
84
+
return error.ParseFailed;
85
+
86
+
const services = parsed.object.get("service") orelse return error.NoPdsEndpoint;
87
+
if (services != .array) return error.NoPdsEndpoint;
88
+
89
+
for (services.array.items) |svc| {
90
+
if (svc != .object) continue;
91
+
const svc_type = svc.object.get("type") orelse continue;
92
+
if (svc_type != .string) continue;
93
+
if (!std.mem.eql(u8, svc_type.string, "AtprotoPersonalDataServer")) continue;
94
+
const endpoint = svc.object.get("serviceEndpoint") orelse continue;
95
+
if (endpoint != .string) continue;
96
+
return try arena.dupe(u8, endpoint.string);
97
+
}
98
+
return error.NoPdsEndpoint;
99
+
}
100
+
101
+
/// get the list of collection NSIDs present in a repo.
102
+
pub fn describeRepo(
103
+
arena: Allocator,
104
+
transport: *zat.HttpTransport,
105
+
pds: []const u8,
106
+
did: []const u8,
107
+
) ResolveError![]const []const u8 {
108
+
const url = try std.fmt.allocPrint(
109
+
arena,
110
+
"{s}/xrpc/com.atproto.repo.describeRepo?repo={s}",
111
+
.{ pds, did },
112
+
);
113
+
114
+
const result = transport.fetch(.{ .url = url }) catch return error.RequestFailed;
115
+
if (result.status != .ok) return error.ResolveFailed;
116
+
117
+
const parsed = json.parseFromSliceLeaky(json.Value, arena, result.body, .{}) catch
118
+
return error.ParseFailed;
119
+
120
+
const collections = parsed.object.get("collections") orelse return error.ParseFailed;
121
+
if (collections != .array) return error.ParseFailed;
122
+
123
+
var out: std.ArrayList([]const u8) = .empty;
124
+
for (collections.array.items) |c| {
125
+
if (c != .string) continue;
126
+
try out.append(arena, try arena.dupe(u8, c.string));
127
+
}
128
+
return out.items;
129
+
}
130
+
131
+
pub const Record = struct {
132
+
uri: []const u8,
133
+
cid: []const u8,
134
+
value: json.Value, // arena-backed, walked by record_text.zig
135
+
};
136
+
137
+
pub const ListPage = struct {
138
+
records: []Record,
139
+
cursor: ?[]const u8,
140
+
};
141
+
142
+
/// one page of listRecords for a given collection. caller loops until cursor
143
+
/// is null to get everything. `limit` max is 100 per the spec.
144
+
pub fn listRecords(
145
+
arena: Allocator,
146
+
transport: *zat.HttpTransport,
147
+
pds: []const u8,
148
+
did: []const u8,
149
+
collection: []const u8,
150
+
cursor: ?[]const u8,
151
+
limit: u32,
152
+
) ResolveError!ListPage {
153
+
const url = if (cursor) |c|
154
+
try std.fmt.allocPrint(
155
+
arena,
156
+
"{s}/xrpc/com.atproto.repo.listRecords?repo={s}&collection={s}&limit={d}&cursor={s}",
157
+
.{ pds, did, collection, limit, c },
158
+
)
159
+
else
160
+
try std.fmt.allocPrint(
161
+
arena,
162
+
"{s}/xrpc/com.atproto.repo.listRecords?repo={s}&collection={s}&limit={d}",
163
+
.{ pds, did, collection, limit },
164
+
);
165
+
166
+
const result = transport.fetch(.{ .url = url }) catch return error.RequestFailed;
167
+
if (result.status != .ok) return error.ResolveFailed;
168
+
169
+
const parsed = json.parseFromSliceLeaky(json.Value, arena, result.body, .{}) catch
170
+
return error.ParseFailed;
171
+
172
+
var out_records: std.ArrayList(Record) = .empty;
173
+
174
+
if (parsed.object.get("records")) |records_val| {
175
+
if (records_val == .array) {
176
+
for (records_val.array.items) |r| {
177
+
if (r != .object) continue;
178
+
const uri = (r.object.get("uri") orelse continue);
179
+
const cid = (r.object.get("cid") orelse continue);
180
+
const value = r.object.get("value") orelse continue;
181
+
if (uri != .string or cid != .string) continue;
182
+
try out_records.append(arena, .{
183
+
.uri = try arena.dupe(u8, uri.string),
184
+
.cid = try arena.dupe(u8, cid.string),
185
+
.value = value,
186
+
});
187
+
}
188
+
}
189
+
}
190
+
191
+
const cursor_out: ?[]const u8 = blk: {
192
+
const v = parsed.object.get("cursor") orelse break :blk null;
193
+
if (v != .string) break :blk null;
194
+
if (v.string.len == 0) break :blk null;
195
+
break :blk try arena.dupe(u8, v.string);
196
+
};
197
+
198
+
return .{ .records = out_records.items, .cursor = cursor_out };
199
+
}
200
+
201
+
/// fetch a single record by collection + rkey. unauthenticated. returns the
202
+
/// record value (json.Value) or null if the record doesn't exist (404/400).
203
+
pub fn getRecord(
204
+
arena: Allocator,
205
+
transport: *zat.HttpTransport,
206
+
pds: []const u8,
207
+
did: []const u8,
208
+
collection: []const u8,
209
+
rkey: []const u8,
210
+
) ?json.Value {
211
+
const url = std.fmt.allocPrint(
212
+
arena,
213
+
"{s}/xrpc/com.atproto.repo.getRecord?repo={s}&collection={s}&rkey={s}",
214
+
.{ pds, did, collection, rkey },
215
+
) catch return null;
216
+
217
+
const result = transport.fetch(.{ .url = url }) catch return null;
218
+
if (result.status != .ok) return null;
219
+
220
+
const parsed = json.parseFromSliceLeaky(json.Value, arena, result.body, .{}) catch
221
+
return null;
222
+
223
+
return parsed.object.get("value");
224
+
}
225
+
226
+
/// fetch a blob by cid via com.atproto.sync.getBlob. unauthenticated — blobs
227
+
/// in a public PDS are readable by anyone. returns the raw bytes owned by
228
+
/// `arena`.
229
+
pub fn getBlob(
230
+
arena: Allocator,
231
+
transport: *zat.HttpTransport,
232
+
pds: []const u8,
233
+
did: []const u8,
234
+
cid: []const u8,
235
+
) ResolveError![]const u8 {
236
+
const url = try std.fmt.allocPrint(
237
+
arena,
238
+
"{s}/xrpc/com.atproto.sync.getBlob?did={s}&cid={s}",
239
+
.{ pds, did, cid },
240
+
);
241
+
const result = transport.fetch(.{ .url = url }) catch return error.RequestFailed;
242
+
if (result.status != .ok) return error.ResolveFailed;
243
+
// result.body is transport-owned; dup into the caller's arena so it
244
+
// outlives this call.
245
+
return try arena.dupe(u8, result.body);
246
+
}
+189
src/record_text.zig
+189
src/record_text.zig
···
1
+
//! record_to_text heuristic — port of embedder/build_pack.py.
2
+
//!
3
+
//! walks arbitrary atproto record JSON and produces embedding-ready text.
4
+
//! rules:
5
+
//! - prepend `collection: <nsid>` so the lex name is in the vector
6
+
//! - drop atproto plumbing keys ($type, cid, rev, sig, prev, version, $link)
7
+
//! - drop identifier-shaped strings (DIDs, at-uris, CIDs, TID rkeys, hex hashes)
8
+
//! - convert iso timestamps to year only (keeps temporal signal, not noise)
9
+
//! - render as flat `key.path: value` lines
10
+
//! - strongRefs and DIDs are currently dropped (v2: deref via slingshot)
11
+
//! - truncate final text at 4000 chars
12
+
13
+
const std = @import("std");
14
+
const json = std.json;
15
+
const Allocator = std.mem.Allocator;
16
+
17
+
const MAX_TEXT_LEN = 4000;
18
+
19
+
const NOISE_KEYS = [_][]const u8{
20
+
"$type", "cid", "rev", "sig", "prev", "version", "$link",
21
+
};
22
+
23
+
fn isNoiseKey(k: []const u8) bool {
24
+
if (k.len == 0) return false;
25
+
if (k[0] == '$') return true;
26
+
for (NOISE_KEYS) |n| if (std.mem.eql(u8, k, n)) return true;
27
+
return false;
28
+
}
29
+
30
+
fn isIdentifier(s: []const u8) bool {
31
+
if (s.len == 0) return false;
32
+
if (std.mem.startsWith(u8, s, "did:plc:")) return true;
33
+
if (std.mem.startsWith(u8, s, "did:web:")) return true;
34
+
if (std.mem.startsWith(u8, s, "at://")) return true;
35
+
// CID v1 raw/dag-cbor
36
+
if (std.mem.startsWith(u8, s, "bafy")) return true;
37
+
if (std.mem.startsWith(u8, s, "bafk")) return true;
38
+
// TID format: exactly 13 chars, lowercase base32 subset [2-7a-z]
39
+
if (s.len == 13) {
40
+
var all_tid = true;
41
+
for (s) |ch| {
42
+
const ok = (ch >= 'a' and ch <= 'z') or (ch >= '2' and ch <= '7');
43
+
if (!ok) {
44
+
all_tid = false;
45
+
break;
46
+
}
47
+
}
48
+
if (all_tid) return true;
49
+
}
50
+
// hex hash of 32+ chars
51
+
if (s.len >= 32) {
52
+
var all_hex = true;
53
+
for (s) |ch| {
54
+
const ok = (ch >= '0' and ch <= '9') or (ch >= 'a' and ch <= 'f');
55
+
if (!ok) {
56
+
all_hex = false;
57
+
break;
58
+
}
59
+
}
60
+
if (all_hex) return true;
61
+
}
62
+
return false;
63
+
}
64
+
65
+
fn looksLikeIsoTimestamp(s: []const u8) bool {
66
+
// YYYY-MM-DDT... pattern (loose)
67
+
if (s.len < 11) return false;
68
+
for (0..4) |i| if (!std.ascii.isDigit(s[i])) return false;
69
+
if (s[4] != '-') return false;
70
+
for (5..7) |i| if (!std.ascii.isDigit(s[i])) return false;
71
+
if (s[7] != '-') return false;
72
+
for (8..10) |i| if (!std.ascii.isDigit(s[i])) return false;
73
+
return s[10] == 'T' or s[10] == ' ';
74
+
}
75
+
76
+
/// shape check: is this a strongRef {uri: "at://...", cid: "bafy..."}?
77
+
fn isStrongRef(v: json.Value) bool {
78
+
if (v != .object) return false;
79
+
const uri_v = v.object.get("uri") orelse return false;
80
+
const cid_v = v.object.get("cid") orelse return false;
81
+
if (uri_v != .string or cid_v != .string) return false;
82
+
return std.mem.startsWith(u8, uri_v.string, "at://");
83
+
}
84
+
85
+
pub const Builder = struct {
86
+
buf: std.ArrayList(u8),
87
+
allocator: Allocator,
88
+
89
+
pub fn init(allocator: Allocator) Builder {
90
+
return .{ .buf = .empty, .allocator = allocator };
91
+
}
92
+
93
+
fn appendLine(self: *Builder, path: []const u8, value: []const u8) !void {
94
+
if (self.buf.items.len >= MAX_TEXT_LEN) return;
95
+
if (self.buf.items.len > 0) try self.buf.append(self.allocator, '\n');
96
+
if (path.len > 0) {
97
+
try self.buf.appendSlice(self.allocator, path);
98
+
try self.buf.appendSlice(self.allocator, ": ");
99
+
}
100
+
try self.buf.appendSlice(self.allocator, value);
101
+
}
102
+
};
103
+
104
+
/// walk a parsed record value, appending flattened text to `builder`.
105
+
fn walk(builder: *Builder, node: json.Value, path: []const u8) anyerror!void {
106
+
if (builder.buf.items.len >= MAX_TEXT_LEN) return;
107
+
108
+
// strongRef short-circuit: drop (v2: inline dereffed content)
109
+
if (isStrongRef(node)) return;
110
+
111
+
switch (node) {
112
+
.object => |obj| {
113
+
var it = obj.iterator();
114
+
while (it.next()) |entry| {
115
+
const k = entry.key_ptr.*;
116
+
if (isNoiseKey(k)) continue;
117
+
118
+
const next_path = if (path.len == 0)
119
+
try builder.allocator.dupe(u8, k)
120
+
else
121
+
try std.fmt.allocPrint(builder.allocator, "{s}.{s}", .{ path, k });
122
+
123
+
try walk(builder, entry.value_ptr.*, next_path);
124
+
}
125
+
},
126
+
.array => |arr| {
127
+
// pure string arrays → comma-joined
128
+
var all_strings = true;
129
+
for (arr.items) |item| {
130
+
if (item != .string) {
131
+
all_strings = false;
132
+
break;
133
+
}
134
+
}
135
+
if (all_strings) {
136
+
var kept: std.ArrayList([]const u8) = .empty;
137
+
for (arr.items) |item| {
138
+
if (!isIdentifier(item.string)) {
139
+
try kept.append(builder.allocator, item.string);
140
+
}
141
+
}
142
+
if (kept.items.len > 0) {
143
+
const joined = try std.mem.join(builder.allocator, ", ", kept.items);
144
+
try builder.appendLine(path, joined);
145
+
}
146
+
} else {
147
+
for (arr.items) |item| try walk(builder, item, path);
148
+
}
149
+
},
150
+
.string => |s| {
151
+
// DID reference → drop (v2: inline profile)
152
+
if (std.mem.startsWith(u8, s, "did:plc:") or std.mem.startsWith(u8, s, "did:web:")) return;
153
+
if (isIdentifier(s)) return;
154
+
if (looksLikeIsoTimestamp(s)) {
155
+
try builder.appendLine(path, s[0..4]); // keep year only
156
+
return;
157
+
}
158
+
try builder.appendLine(path, s);
159
+
},
160
+
.integer => |i| {
161
+
if (i == 0) return;
162
+
const s = try std.fmt.allocPrint(builder.allocator, "{d}", .{i});
163
+
try builder.appendLine(path, s);
164
+
},
165
+
.float => |f| {
166
+
if (f == 0.0) return;
167
+
const s = try std.fmt.allocPrint(builder.allocator, "{d}", .{f});
168
+
try builder.appendLine(path, s);
169
+
},
170
+
.bool => return, // rarely meaningful for retrieval
171
+
.null => return,
172
+
.number_string => |s| try builder.appendLine(path, s),
173
+
}
174
+
}
175
+
176
+
/// produce embedding-ready text for a record. result is allocated in `arena`
177
+
/// and truncated at MAX_TEXT_LEN.
178
+
pub fn recordToText(
179
+
arena: Allocator,
180
+
collection: []const u8,
181
+
value: json.Value,
182
+
) ![]const u8 {
183
+
var b = Builder.init(arena);
184
+
// prepend the NSID — strongest structural signal
185
+
try b.appendLine("collection", collection);
186
+
try walk(&b, value, "");
187
+
const len = @min(b.buf.items.len, MAX_TEXT_LEN);
188
+
return b.buf.items[0..len];
189
+
}
+548
src/repo_walk.zig
+548
src/repo_walk.zig
···
1
+
//! single-request repo walk via `com.atproto.sync.getRepo`.
2
+
//!
3
+
//! the previous walker used paginated `listRecords` on every collection —
4
+
//! ~172 HTTP round trips for a 17k-record repo. this does one request,
5
+
//! pulls the entire repo as a CAR file (DAG-CBOR blocks + MST structure),
6
+
//! and yields records by walking the MST locally.
7
+
//!
8
+
//! three-stage, streaming-friendly fetch + parse:
9
+
//! 1. HTTP GET /xrpc/com.atproto.sync.getRepo, draining the response body
10
+
//! directly into a temp file in /tmp. no intermediate heap buffer, so
11
+
//! peak RSS is bounded regardless of repo size.
12
+
//! 2. mmap the temp file read-only and feed the slice to
13
+
//! `zat.car.streamBlocks`, building a `CID → byte range` index.
14
+
//! 3. MST walk starts at the commit root, looks up each child CID by
15
+
//! range, decodes the MST/record block on demand, and yields records.
16
+
//!
17
+
//! the mmap + the index are the only things resident beyond the arena's
18
+
//! per-record decode churn. kernel handles paging for the CAR bytes. temp
19
+
//! file is deleted before walkRepo returns (success or failure).
20
+
//!
21
+
//! this unblocks very large repos (pfrazee.com sits at ~248k blocks / 72 MB,
22
+
//! well past the heap-based approach on a 4 GB fly machine running two
23
+
//! concurrent indexes).
24
+
//!
25
+
//! we talk to std.http.Client directly for this one call instead of going
26
+
//! through zat.HttpTransport.fetch — the latter always buffers the whole
27
+
//! body via std.Io.Writer.Allocating, and for multi-hundred-MB repos that
28
+
//! defeats the point. every other ken call still uses the zat helper.
29
+
//!
30
+
//! relies on zat for the protocol primitives:
31
+
//! - zat.car.streamBlocks for CAR parsing (zero-alloc iterator)
32
+
//! - zat.mst.decodeMstNode for MST node decoding
33
+
//! - zat.cbor.decodeAll for per-block DAG-CBOR decoding
34
+
//! - zat.multibase.base32lower.encode for CID → "bafy..." stringification
35
+
//!
36
+
//! records come out with the same (uri, cid, value) shape as pds.listRecords
37
+
//! so indexer.doIndex can swap the two walks behind a try-CAR-fallback-to-
38
+
//! listRecords flow.
39
+
40
+
const std = @import("std");
41
+
const Io = std.Io;
42
+
const Allocator = std.mem.Allocator;
43
+
const json = std.json;
44
+
const zat = @import("zat");
45
+
46
+
pub const Record = struct {
47
+
/// at://{did}/{collection}/{rkey}
48
+
uri: []const u8,
49
+
/// base32 multibase CID string ("bafy...")
50
+
cid: []const u8,
51
+
/// NSID, derived from the MST key's collection prefix
52
+
collection: []const u8,
53
+
/// parsed record body, converted from DAG-CBOR to std.json.Value so it
54
+
/// plugs into the existing record_text / display pipelines
55
+
value: json.Value,
56
+
};
57
+
58
+
pub const WalkResult = struct {
59
+
records: []Record,
60
+
/// total CAR size in bytes (for logging / diagnostics)
61
+
car_bytes: usize,
62
+
skipped_by_collection: usize,
63
+
skipped_by_time: usize,
64
+
};
65
+
66
+
/// skip predicate applied per-record during the MST walk. filtering happens
67
+
/// before value decode/dupe, so skipped records cost roughly the MST entry
68
+
/// iteration and nothing else.
69
+
pub const WalkFilter = struct {
70
+
/// exact NSID matches to drop (e.g. "app.bsky.feed.like"). comparison
71
+
/// is O(n*m) on entry count × skip list size — both are small in
72
+
/// practice, not worth hashing.
73
+
skip_collections: []const []const u8 = &.{},
74
+
/// drop records whose rkey decodes to a TID timestamp older than this
75
+
/// cutoff (unix microseconds). records with non-TID rkeys (e.g.
76
+
/// `self`, profile) are always kept — time cutoff is a best-effort
77
+
/// filter, not a hard wall.
78
+
min_rkey_tid_us: ?i64 = null,
79
+
};
80
+
81
+
pub const CountResult = struct {
82
+
/// records that would be kept under the filter
83
+
kept: usize,
84
+
skipped_by_collection: usize,
85
+
skipped_by_time: usize,
86
+
};
87
+
88
+
pub const WalkError = error{
89
+
FetchFailed,
90
+
EmptyCar,
91
+
NoCommitBlock,
92
+
InvalidCommit,
93
+
NoDataField,
94
+
InvalidDataField,
95
+
BlockNotFound,
96
+
InvalidMstNode,
97
+
InvalidRecordCbor,
98
+
OutOfMemory,
99
+
};
100
+
101
+
/// entry in the CID → byte-range index. `off` / `len` point into the CAR
102
+
/// response buffer. 16 bytes per block instead of 48+ in a StringHashMap
103
+
/// of slices.
104
+
const BlockRange = packed struct {
105
+
off: u64,
106
+
len: u32,
107
+
_pad: u32 = 0,
108
+
};
109
+
110
+
/// CID bytes → byte range inside `car_bytes`. keys borrow from the CAR
111
+
/// buffer (zat's BlockIterator yields zero-copy cid slices), so this is
112
+
/// ~16 bytes of value + a few bytes of hashmap overhead per block.
113
+
const BlockIndex = std.StringHashMapUnmanaged(BlockRange);
114
+
115
+
/// process-local counter used to generate unique temp-file names for
116
+
/// concurrent `fetchRepoToTempFile` calls.
117
+
var fetch_seq: std.atomic.Value(u64) = .init(0);
118
+
119
+
/// A repo that has been fetched + mmap'd and had its block index built.
120
+
/// Holds resources (temp file, mmap) that must be released via `close`.
121
+
/// Callers typically do `openRepo` → zero or more `walkOpened`/`countOpened`
122
+
/// passes → `close`. The two-pass pattern (count → decide filter → walk)
123
+
/// avoids re-fetching the CAR.
124
+
pub const OpenedRepo = struct {
125
+
fetched: FetchedCar,
126
+
car_bytes: []const u8,
127
+
index: BlockIndex,
128
+
/// raw bytes of the MST root CID (entry point for walks). borrows
129
+
/// from the arena the repo was opened with.
130
+
data_cid: []const u8,
131
+
132
+
pub fn close(self: *OpenedRepo, io: Io) void {
133
+
self.fetched.destroy(io);
134
+
}
135
+
};
136
+
137
+
/// Fetch the repo CAR, stream-index its blocks, and locate the MST root.
138
+
/// Returns an `OpenedRepo` that can be walked (or counted) any number of
139
+
/// times without re-fetching. Caller must `close` when done.
140
+
pub fn openRepo(
141
+
arena: Allocator,
142
+
io: Io,
143
+
pds_url: []const u8,
144
+
did: []const u8,
145
+
) WalkError!OpenedRepo {
146
+
// 1. stream the HTTP response body straight into a temp file in /tmp,
147
+
// then mmap it read-only. peak RSS stays bounded no matter how big
148
+
// the repo is (kernel pages in what we touch, evicts what we don't).
149
+
var fetched = fetchRepoToTempFile(arena, io, pds_url, did) catch |err| switch (err) {
150
+
error.OutOfMemory => return error.OutOfMemory,
151
+
else => return error.FetchFailed,
152
+
};
153
+
errdefer fetched.destroy(io);
154
+
155
+
const car_bytes: []const u8 = fetched.bytes;
156
+
if (car_bytes.len == 0) return error.EmptyCar;
157
+
158
+
// 2. stream the CAR once to extract roots + build a CID → byte-range
159
+
// index. passing max_size = car_bytes.len disables zat's 2 MB guard
160
+
// for trusted input we fetched ourselves. no max_blocks cap — the
161
+
// iterator is O(1) space per block, so block count is bounded only
162
+
// by CAR size (max_block_size still protects us from malformed
163
+
// over-long blocks).
164
+
var stream = zat.car.streamBlocks(arena, car_bytes, .{
165
+
.max_size = car_bytes.len,
166
+
}) catch return error.EmptyCar;
167
+
if (stream.roots.len == 0) return error.EmptyCar;
168
+
169
+
var index: BlockIndex = .empty;
170
+
// reserve in proportion to the CAR size — avg block is ~300 bytes in
171
+
// practice, so car_bytes.len / 256 is a decent upper-bound estimate
172
+
// that keeps us in a single rehash on all but the densest repos.
173
+
try index.ensureTotalCapacity(arena, @intCast(@min(car_bytes.len / 256, std.math.maxInt(u32))));
174
+
175
+
while (true) {
176
+
const block = stream.iter.next() catch |err| switch (err) {
177
+
error.OutOfMemory => return error.OutOfMemory,
178
+
else => return error.EmptyCar,
179
+
} orelse break;
180
+
// block.data is a zero-copy slice into car_bytes — recover its
181
+
// byte range directly from pointer arithmetic, no extra state.
182
+
const data_off: u64 = @intCast(@intFromPtr(block.data.ptr) - @intFromPtr(car_bytes.ptr));
183
+
try index.put(arena, block.cid_raw, .{
184
+
.off = data_off,
185
+
.len = @intCast(block.data.len),
186
+
});
187
+
}
188
+
189
+
// 3. commit block → data CID (MST root)
190
+
const commit_cid_raw = stream.roots[0].raw;
191
+
const commit_data = indexLookup(car_bytes, index, commit_cid_raw) orelse return error.NoCommitBlock;
192
+
const commit_value = zat.cbor.decodeAll(arena, commit_data) catch return error.InvalidCommit;
193
+
const data_cbor = commit_value.get("data") orelse return error.NoDataField;
194
+
const data_cid_raw = switch (data_cbor) {
195
+
.cid => |c| c.raw,
196
+
else => return error.InvalidDataField,
197
+
};
198
+
199
+
return .{
200
+
.fetched = fetched,
201
+
.car_bytes = car_bytes,
202
+
.index = index,
203
+
.data_cid = data_cid_raw,
204
+
};
205
+
}
206
+
207
+
/// MST walk that yields records through `arena`. Uses `filter` to drop
208
+
/// records before value decode — skipped records cost only the MST entry
209
+
/// iteration, not a CBOR roundtrip.
210
+
pub fn walkOpened(
211
+
arena: Allocator,
212
+
opened: *const OpenedRepo,
213
+
did: []const u8,
214
+
filter: WalkFilter,
215
+
) WalkError!WalkResult {
216
+
var records: std.ArrayList(Record) = .empty;
217
+
var ctx: WalkCtx = .{
218
+
.arena = arena,
219
+
.car_bytes = opened.car_bytes,
220
+
.index = opened.index,
221
+
.did = did,
222
+
.filter = filter,
223
+
.out = &records,
224
+
};
225
+
try walkNode(&ctx, opened.data_cid);
226
+
227
+
return .{
228
+
.records = try records.toOwnedSlice(arena),
229
+
.car_bytes = opened.car_bytes.len,
230
+
.skipped_by_collection = ctx.skipped_by_collection,
231
+
.skipped_by_time = ctx.skipped_by_time,
232
+
};
233
+
}
234
+
235
+
/// Count-only MST walk. Applies the filter and counts kept records + skip
236
+
/// buckets, but does not decode record values — cheap enough to run before
237
+
/// the real walk as a "how many records would we embed?" probe.
238
+
pub fn countOpened(
239
+
arena: Allocator,
240
+
opened: *const OpenedRepo,
241
+
filter: WalkFilter,
242
+
) WalkError!CountResult {
243
+
var ctx: WalkCtx = .{
244
+
.arena = arena,
245
+
.car_bytes = opened.car_bytes,
246
+
.index = opened.index,
247
+
.did = "",
248
+
.filter = filter,
249
+
.out = null,
250
+
};
251
+
try walkNode(&ctx, opened.data_cid);
252
+
return .{
253
+
.kept = ctx.kept,
254
+
.skipped_by_collection = ctx.skipped_by_collection,
255
+
.skipped_by_time = ctx.skipped_by_time,
256
+
};
257
+
}
258
+
259
+
/// Convenience: open → walk → close in one call. Equivalent to the
260
+
/// pre-filter walkRepo API; used by the smoke test and any caller that
261
+
/// doesn't need the count-then-walk staging.
262
+
pub fn walkRepo(
263
+
arena: Allocator,
264
+
io: Io,
265
+
pds_url: []const u8,
266
+
did: []const u8,
267
+
filter: WalkFilter,
268
+
) WalkError!WalkResult {
269
+
var opened = try openRepo(arena, io, pds_url, did);
270
+
defer opened.close(io);
271
+
return walkOpened(arena, &opened, did, filter);
272
+
}
273
+
274
+
/// owns a temp file + its mmap. deletes the temp file on destroy.
275
+
const FetchedCar = struct {
276
+
/// absolute path in /tmp, so we can unlink on destroy
277
+
path: []const u8,
278
+
file: std.Io.File,
279
+
mmap: std.Io.File.MemoryMap,
280
+
bytes: []const u8,
281
+
282
+
fn destroy(self: *FetchedCar, io: Io) void {
283
+
self.mmap.destroy(io);
284
+
self.file.close(io);
285
+
// best-effort delete; leaking a /tmp file on shutdown is fine
286
+
std.Io.Dir.deleteFileAbsolute(io, self.path) catch {};
287
+
}
288
+
};
289
+
290
+
/// GET /xrpc/com.atproto.sync.getRepo and drain the response body directly
291
+
/// into a temp file, then mmap the file read-only. never holds the full
292
+
/// CAR in heap. `path` is owned by `arena`.
293
+
fn fetchRepoToTempFile(
294
+
arena: Allocator,
295
+
io: Io,
296
+
pds_url: []const u8,
297
+
did: []const u8,
298
+
) !FetchedCar {
299
+
const url = try std.fmt.allocPrint(arena, "{s}/xrpc/com.atproto.sync.getRepo?did={s}", .{ pds_url, did });
300
+
301
+
// path: /tmp/ken-car-{seq}-{did-tail}.car — a process-local monotonic
302
+
// counter is enough to disambiguate concurrent indexes (ken is a
303
+
// single-process server); did-tail is purely for human debugging
304
+
// when poking at /tmp by hand.
305
+
const did_tail_start = if (std.mem.lastIndexOfScalar(u8, did, ':')) |i| i + 1 else 0;
306
+
const did_tail = did[did_tail_start..@min(did_tail_start + 16, did.len)];
307
+
const seq = fetch_seq.fetchAdd(1, .monotonic);
308
+
const path = try std.fmt.allocPrint(arena, "/tmp/ken-car-{d}-{s}.car", .{ seq, did_tail });
309
+
310
+
// --- create the temp file + wrap it in a buffered writer ---
311
+
var out_file = try std.Io.Dir.createFileAbsolute(io, path, .{ .read = true });
312
+
errdefer {
313
+
out_file.close(io);
314
+
std.Io.Dir.deleteFileAbsolute(io, path) catch {};
315
+
}
316
+
317
+
var write_buf: [64 * 1024]u8 = undefined;
318
+
var out_writer = std.Io.File.Writer.initStreaming(out_file, io, &write_buf);
319
+
320
+
// --- issue the HTTP request ---
321
+
var client: std.http.Client = .{ .allocator = arena, .io = io };
322
+
defer client.deinit();
323
+
324
+
var req = try client.request(.GET, try std.Uri.parse(url), .{
325
+
// see notes in oauth.zig:pdsAuthedRequest — std.http.Client low-level
326
+
// path doesn't transparently decompress, so force identity encoding
327
+
// via the typed header slot (extra_headers is additive and produces
328
+
// conflicting Accept-Encoding values).
329
+
.headers = .{ .accept_encoding = .{ .override = "identity" } },
330
+
});
331
+
defer req.deinit();
332
+
try req.sendBodiless();
333
+
334
+
// 2 KB is enough for any reasonable redirect target — bsky.network's
335
+
// sync.getRepo redirects to the user's actual PDS hostname, which is
336
+
// at most a couple hundred bytes; matching oauth.zig's 4096-byte
337
+
// auth_hdr_buf is overkill but cheap and future-proof.
338
+
var redirect_buf: [2048]u8 = undefined;
339
+
var response = try req.receiveHead(&redirect_buf);
340
+
if (response.head.status != .ok) return error.FetchFailed;
341
+
342
+
// --- drain body into the file ---
343
+
const reader = response.reader(&.{});
344
+
_ = try reader.streamRemaining(&out_writer.interface);
345
+
try out_writer.end();
346
+
347
+
const file_len = try out_file.length(io);
348
+
if (file_len == 0) {
349
+
out_file.close(io);
350
+
std.Io.Dir.deleteFileAbsolute(io, path) catch {};
351
+
return error.EmptyCar;
352
+
}
353
+
354
+
// --- mmap the file read-only ---
355
+
var mmap = try std.Io.File.MemoryMap.create(io, out_file, .{
356
+
.len = @intCast(file_len),
357
+
.protection = .{ .read = true, .write = false },
358
+
.offset = 0,
359
+
});
360
+
errdefer mmap.destroy(io);
361
+
362
+
return .{
363
+
.path = path,
364
+
.file = out_file,
365
+
.mmap = mmap,
366
+
.bytes = mmap.memory[0..@intCast(file_len)],
367
+
};
368
+
}
369
+
370
+
fn indexLookup(car_bytes: []const u8, index: BlockIndex, cid: []const u8) ?[]const u8 {
371
+
const range = index.get(cid) orelse return null;
372
+
return car_bytes[range.off..][0..range.len];
373
+
}
374
+
375
+
/// shared state for recursive MST traversal. `out == null` means
376
+
/// count-only mode: apply filter, track stats, but skip CBOR value decode
377
+
/// and record materialization.
378
+
const WalkCtx = struct {
379
+
arena: Allocator,
380
+
car_bytes: []const u8,
381
+
index: BlockIndex,
382
+
did: []const u8,
383
+
filter: WalkFilter,
384
+
out: ?*std.ArrayList(Record),
385
+
kept: usize = 0,
386
+
skipped_by_collection: usize = 0,
387
+
skipped_by_time: usize = 0,
388
+
};
389
+
390
+
const FilterDecision = enum { kept, skipped_collection, skipped_time };
391
+
392
+
fn applyFilter(filter: WalkFilter, collection_name: []const u8, rkey: []const u8) FilterDecision {
393
+
for (filter.skip_collections) |sc| {
394
+
if (std.mem.eql(u8, collection_name, sc)) return .skipped_collection;
395
+
}
396
+
if (filter.min_rkey_tid_us) |cutoff| {
397
+
if (decodeTidMicros(rkey)) |tid_us| {
398
+
if (tid_us < cutoff) return .skipped_time;
399
+
}
400
+
// non-TID rkeys fall through as kept — profile `self`, etc.
401
+
}
402
+
return .kept;
403
+
}
404
+
405
+
fn walkNode(ctx: *WalkCtx, node_cid: []const u8) WalkError!void {
406
+
const node_data = indexLookup(ctx.car_bytes, ctx.index, node_cid) orelse return error.BlockNotFound;
407
+
const node = zat.mst.decodeMstNode(ctx.arena, node_data) catch return error.InvalidMstNode;
408
+
409
+
// MST invariant: left subtree holds keys strictly less than every key
410
+
// in this node. walk it first so the output ends up in lexicographic
411
+
// order (same contract as pds.listRecords, alphabetical by collection).
412
+
if (node.left) |left_cid| try walkNode(ctx, left_cid);
413
+
414
+
// prefix-compressed keys: entries[i].key = entries[i-1].key[0..prefix_len] ++ suffix.
415
+
// a 512-byte reconstruction buffer is plenty — atproto MST keys are
416
+
// `{collection}/{rkey}`, which maxes out around 128 bytes in practice.
417
+
var key_buf: [512]u8 = undefined;
418
+
for (node.entries) |entry| {
419
+
if (entry.prefix_len + entry.key_suffix.len > key_buf.len) continue;
420
+
@memcpy(key_buf[entry.prefix_len..][0..entry.key_suffix.len], entry.key_suffix);
421
+
const key = key_buf[0 .. entry.prefix_len + entry.key_suffix.len];
422
+
423
+
const slash = std.mem.indexOfScalar(u8, key, '/') orelse continue;
424
+
const collection_name = key[0..slash];
425
+
const rkey = key[slash + 1 ..];
426
+
427
+
switch (applyFilter(ctx.filter, collection_name, rkey)) {
428
+
.skipped_collection => ctx.skipped_by_collection += 1,
429
+
.skipped_time => ctx.skipped_by_time += 1,
430
+
.kept => {
431
+
ctx.kept += 1;
432
+
if (ctx.out) |out| {
433
+
// full emit: decode value, dupe strings, append. on
434
+
// decode failure preserve old behavior and skip the
435
+
// right subtree — `continue` bypasses the tree walk
436
+
// at the bottom of the loop.
437
+
const collection = ctx.arena.dupe(u8, collection_name) catch return error.OutOfMemory;
438
+
const value_data = indexLookup(ctx.car_bytes, ctx.index, entry.value) orelse continue;
439
+
const value_cbor = zat.cbor.decodeAll(ctx.arena, value_data) catch continue;
440
+
const value_json = try cborToJson(ctx.arena, value_cbor);
441
+
442
+
const uri = std.fmt.allocPrint(ctx.arena, "at://{s}/{s}/{s}", .{ ctx.did, collection, rkey }) catch return error.OutOfMemory;
443
+
const cid_str = zat.multibase.encode(ctx.arena, .base32lower, entry.value) catch return error.OutOfMemory;
444
+
445
+
try out.append(ctx.arena, .{
446
+
.uri = uri,
447
+
.cid = cid_str,
448
+
.collection = collection,
449
+
.value = value_json,
450
+
});
451
+
}
452
+
},
453
+
}
454
+
455
+
// right subtree of THIS entry (keys between this one and the next)
456
+
if (entry.tree) |tree_cid| try walkNode(ctx, tree_cid);
457
+
}
458
+
}
459
+
460
+
/// Decode an atproto TID rkey (13 base32-sortable chars) to a unix
461
+
/// microsecond timestamp. Returns null for non-TID rkeys — profile
462
+
/// `self`, app.bsky.actor.declaration `self`, etc. Callers should treat
463
+
/// a null as "unfilterable, keep by default".
464
+
///
465
+
/// TID layout: 64 bits, top bit always 0 (sort-safe), next 53 bits =
466
+
/// microseconds since unix epoch, bottom 10 bits = clock id.
467
+
/// Alphabet: `234567abcdefghijklmnopqrstuvwxyz`.
468
+
///
469
+
/// public so the listRecords fallback in indexer.zig can apply the
470
+
/// same time cutoff as the CAR walker for large repos.
471
+
pub fn decodeTidMicros(rkey: []const u8) ?i64 {
472
+
if (rkey.len != 13) return null;
473
+
var val: u64 = 0;
474
+
for (rkey) |c| {
475
+
const digit: u64 = switch (c) {
476
+
'2'...'7' => @intCast(c - '2'),
477
+
'a'...'z' => @intCast(@as(u32, c - 'a') + 6),
478
+
else => return null,
479
+
};
480
+
val = (val << 5) | digit;
481
+
}
482
+
if (val >> 63 != 0) return null; // top bit must be 0
483
+
const us = (val >> 10) & ((@as(u64, 1) << 53) - 1);
484
+
return @intCast(us);
485
+
}
486
+
487
+
test "decodeTidMicros: known TID round trip" {
488
+
// 3juj-5m5xfs2v = ??? — use a TID encoding a specific time
489
+
// pick unix epoch 2023-06-01T00:00:00Z = 1685577600 seconds
490
+
// = 1685577600000000 microseconds
491
+
// shift left 10 bits = 1725031526400000000 << 10 actually wait
492
+
// let's encode then decode for self-consistency instead
493
+
const us: u64 = 1685577600000000;
494
+
const clock: u64 = 0;
495
+
var val: u64 = (us << 10) | clock;
496
+
// encode to 13 base32-sortable chars
497
+
var buf: [13]u8 = undefined;
498
+
var i: usize = 13;
499
+
while (i > 0) {
500
+
i -= 1;
501
+
const d: u8 = @intCast(val & 0x1f);
502
+
buf[i] = if (d < 6) '2' + d else 'a' + (d - 6);
503
+
val >>= 5;
504
+
}
505
+
const decoded = decodeTidMicros(&buf) orelse return error.DecodeFailed;
506
+
try std.testing.expectEqual(@as(i64, @intCast(us)), decoded);
507
+
}
508
+
509
+
test "decodeTidMicros: non-TID rkey returns null" {
510
+
try std.testing.expectEqual(@as(?i64, null), decodeTidMicros("self"));
511
+
try std.testing.expectEqual(@as(?i64, null), decodeTidMicros(""));
512
+
try std.testing.expectEqual(@as(?i64, null), decodeTidMicros("too-long-for-a-tid-abc"));
513
+
}
514
+
515
+
/// Convert a zat.cbor.Value into a std.json.Value. This exists so records
516
+
/// pulled through the CAR walker can flow through the same record_text /
517
+
/// display pipeline as records from pds.listRecords (which returns
518
+
/// json.Value already, since listRecords is a JSON XRPC endpoint).
519
+
///
520
+
/// dropped: byte strings and CID refs. record_text treats them as noise
521
+
/// anyway — CID-shaped strings get filtered by isIdentifier, and raw bytes
522
+
/// don't contribute semantic signal.
523
+
fn cborToJson(arena: Allocator, cv: zat.cbor.Value) WalkError!json.Value {
524
+
return switch (cv) {
525
+
.unsigned => |u| .{ .integer = std.math.cast(i64, u) orelse return .{ .null = {} } },
526
+
.negative => |n| .{ .integer = n },
527
+
.text => |t| .{ .string = arena.dupe(u8, t) catch return error.OutOfMemory },
528
+
.boolean => |b| .{ .bool = b },
529
+
.null => .{ .null = {} },
530
+
.bytes => .{ .null = {} },
531
+
.cid => .{ .null = {} },
532
+
.array => |arr| blk: {
533
+
var items: json.Array = .init(arena);
534
+
items.ensureTotalCapacity(arr.len) catch return error.OutOfMemory;
535
+
for (arr) |v| items.appendAssumeCapacity(try cborToJson(arena, v));
536
+
break :blk .{ .array = items };
537
+
},
538
+
.map => |entries| blk: {
539
+
var obj: json.ObjectMap = .{};
540
+
obj.ensureTotalCapacity(arena, @intCast(entries.len)) catch return error.OutOfMemory;
541
+
for (entries) |e| {
542
+
const key_copy = arena.dupe(u8, e.key) catch return error.OutOfMemory;
543
+
obj.putAssumeCapacity(key_copy, try cborToJson(arena, e.value));
544
+
}
545
+
break :blk .{ .object = obj };
546
+
},
547
+
};
548
+
}
+167
src/root.zig
+167
src/root.zig
···
1
+
const std = @import("std");
2
+
3
+
pub const pds = @import("pds.zig");
4
+
pub const repo_walk = @import("repo_walk.zig");
5
+
pub const record_text = @import("record_text.zig");
6
+
7
+
pub const Kind = enum {
8
+
text,
9
+
image_ref,
10
+
video_ref,
11
+
12
+
pub fn jsonString(self: Kind) []const u8 {
13
+
return switch (self) {
14
+
.text => "text",
15
+
.image_ref => "image_ref",
16
+
.video_ref => "video_ref",
17
+
};
18
+
}
19
+
};
20
+
21
+
pub const BlobRef = struct {
22
+
cid: []const u8,
23
+
mime_type: ?[]const u8 = null,
24
+
alt: ?[]const u8 = null,
25
+
};
26
+
27
+
pub const EmbeddableItem = struct {
28
+
kind: Kind,
29
+
uri: []const u8,
30
+
cid: []const u8,
31
+
collection: []const u8,
32
+
author_did: []const u8,
33
+
created_at: ?[]const u8 = null,
34
+
text: ?[]const u8 = null,
35
+
blob: ?BlobRef = null,
36
+
};
37
+
38
+
pub const ExtractOptions = struct {
39
+
include_media_refs: bool = false,
40
+
skip_collections: []const []const u8 = &.{ "app.bsky.feed.like", "app.bsky.graph.follow" },
41
+
min_rkey_tid_us: ?i64 = null,
42
+
};
43
+
44
+
pub fn extractRecords(
45
+
arena: std.mem.Allocator,
46
+
did: []const u8,
47
+
records: []const repo_walk.Record,
48
+
options: ExtractOptions,
49
+
) ![]EmbeddableItem {
50
+
var out: std.ArrayList(EmbeddableItem) = .empty;
51
+
for (records) |record| {
52
+
const text = try record_text.recordToText(arena, record.collection, record.value);
53
+
if (std.mem.trim(u8, text, " \t\r\n").len > 0) {
54
+
try out.append(arena, .{
55
+
.kind = .text,
56
+
.uri = record.uri,
57
+
.cid = record.cid,
58
+
.collection = record.collection,
59
+
.author_did = did,
60
+
.created_at = findString(record.value, "createdAt"),
61
+
.text = text,
62
+
});
63
+
}
64
+
65
+
if (options.include_media_refs) {
66
+
try collectMediaRefs(arena, &out, did, record);
67
+
}
68
+
}
69
+
return out.toOwnedSlice(arena);
70
+
}
71
+
72
+
pub fn filterFromOptions(options: ExtractOptions) repo_walk.WalkFilter {
73
+
return .{
74
+
.skip_collections = options.skip_collections,
75
+
.min_rkey_tid_us = options.min_rkey_tid_us,
76
+
};
77
+
}
78
+
79
+
fn collectMediaRefs(
80
+
arena: std.mem.Allocator,
81
+
out: *std.ArrayList(EmbeddableItem),
82
+
did: []const u8,
83
+
record: repo_walk.Record,
84
+
) !void {
85
+
const embed = getPath(record.value, &.{ "embed" }) orelse return;
86
+
const embed_type = findString(embed, "$type") orelse "";
87
+
88
+
if (std.mem.eql(u8, embed_type, "app.bsky.embed.images")) {
89
+
const images = getPath(embed, &.{ "images" }) orelse return;
90
+
if (images != .array) return;
91
+
for (images.array.items) |image| {
92
+
const ref = blobRefFromObject(image, "image") orelse continue;
93
+
try out.append(arena, mediaItem(.image_ref, did, record, ref));
94
+
}
95
+
} else if (std.mem.eql(u8, embed_type, "app.bsky.embed.video")) {
96
+
const ref = blobRefFromObject(embed, "video") orelse return;
97
+
try out.append(arena, mediaItem(.video_ref, did, record, ref));
98
+
} else if (std.mem.eql(u8, embed_type, "app.bsky.embed.recordWithMedia")) {
99
+
const media = getPath(embed, &.{ "media" }) orelse return;
100
+
const media_type = findString(media, "$type") orelse "";
101
+
if (std.mem.eql(u8, media_type, "app.bsky.embed.images")) {
102
+
const images = getPath(media, &.{ "images" }) orelse return;
103
+
if (images != .array) return;
104
+
for (images.array.items) |image| {
105
+
const ref = blobRefFromObject(image, "image") orelse continue;
106
+
try out.append(arena, mediaItem(.image_ref, did, record, ref));
107
+
}
108
+
} else if (std.mem.eql(u8, media_type, "app.bsky.embed.video")) {
109
+
const ref = blobRefFromObject(media, "video") orelse return;
110
+
try out.append(arena, mediaItem(.video_ref, did, record, ref));
111
+
}
112
+
}
113
+
}
114
+
115
+
fn mediaItem(kind: Kind, did: []const u8, record: repo_walk.Record, ref: BlobRef) EmbeddableItem {
116
+
return .{
117
+
.kind = kind,
118
+
.uri = record.uri,
119
+
.cid = record.cid,
120
+
.collection = record.collection,
121
+
.author_did = did,
122
+
.created_at = findString(record.value, "createdAt"),
123
+
.blob = ref,
124
+
};
125
+
}
126
+
127
+
fn blobRefFromObject(value: std.json.Value, field: []const u8) ?BlobRef {
128
+
if (value != .object) return null;
129
+
const blob = value.object.get(field) orelse return null;
130
+
if (blob != .object) return null;
131
+
const ref = blob.object.get("ref") orelse blob.object.get("cid") orelse return null;
132
+
const cid = switch (ref) {
133
+
.string => |s| s,
134
+
.object => |obj| blk: {
135
+
const link = obj.get("$link") orelse return null;
136
+
if (link != .string) return null;
137
+
break :blk link.string;
138
+
},
139
+
else => return null,
140
+
};
141
+
return .{
142
+
.cid = cid,
143
+
.mime_type = findString(blob, "mimeType"),
144
+
.alt = findString(value, "alt"),
145
+
};
146
+
}
147
+
148
+
fn getPath(value: std.json.Value, path: []const []const u8) ?std.json.Value {
149
+
var cur = value;
150
+
for (path) |part| {
151
+
if (cur != .object) return null;
152
+
cur = cur.object.get(part) orelse return null;
153
+
}
154
+
return cur;
155
+
}
156
+
157
+
fn findString(value: std.json.Value, key: []const u8) ?[]const u8 {
158
+
if (value != .object) return null;
159
+
const v = value.object.get(key) orelse return null;
160
+
if (v != .string) return null;
161
+
return v.string;
162
+
}
163
+
164
+
test "filter defaults skip low-signal social records" {
165
+
const filter = filterFromOptions(.{});
166
+
try std.testing.expectEqual(@as(usize, 2), filter.skip_collections.len);
167
+
}
+2263
src/worker.js
+2263
src/worker.js
···
1
+
import {
2
+
CompositeDidDocumentResolver,
3
+
LocalActorResolver,
4
+
PlcDidDocumentResolver,
5
+
WebDidDocumentResolver,
6
+
XrpcHandleResolver
7
+
} from "@atcute/identity-resolver";
8
+
import { fromStream as repoFromStream } from "@atcute/repo";
9
+
10
+
const VOYAGE_URL = "https://api.voyageai.com/v1/embeddings";
11
+
const TPUF_BASE = "https://api.turbopuffer.com/v2/namespaces";
12
+
const TYPEAHEAD_URL = "https://typeahead.waow.tech/xrpc/app.bsky.actor.searchActorsTypeahead";
13
+
const MODEL = "voyage-4-lite";
14
+
const DIMENSIONS = 1024;
15
+
const INDEX_LIMIT = 5000;
16
+
const INDEX_BATCH_SIZE = 96;
17
+
const MAX_TEXT_LEN = 4000;
18
+
const ARTIFACT_ATTRS = [
19
+
"artifactKind",
20
+
"artifactTitle",
21
+
"artifactBody",
22
+
"artifactUrl",
23
+
"artifactMedia",
24
+
"artifactRefs",
25
+
"artifactDate",
26
+
"artifactConfidence"
27
+
];
28
+
29
+
const actorResolver = new LocalActorResolver({
30
+
handleResolver: new XrpcHandleResolver({
31
+
serviceUrl: "https://public.api.bsky.app"
32
+
}),
33
+
didDocumentResolver: new CompositeDidDocumentResolver({
34
+
methods: {
35
+
plc: new PlcDidDocumentResolver(),
36
+
web: new WebDidDocumentResolver()
37
+
}
38
+
})
39
+
});
40
+
41
+
export default {
42
+
async fetch(request, env) {
43
+
const url = new URL(request.url);
44
+
45
+
if (url.pathname === "/api/search") return search(request, env, url);
46
+
if (url.pathname === "/api/update") return update(request, env, url);
47
+
if (url.pathname === "/api/actors") return actors(request, url);
48
+
if (url.pathname === "/health") return json({ ok: true, name: "Pensieve" });
49
+
50
+
return new Response(renderIndex(), {
51
+
headers: {
52
+
"content-type": "text/html; charset=utf-8",
53
+
"cache-control": "no-store"
54
+
}
55
+
});
56
+
}
57
+
};
58
+
59
+
async function actors(request, url) {
60
+
if (request.method !== "GET") return json({ error: "method_not_allowed" }, 405);
61
+
62
+
const q = (url.searchParams.get("q") || "").trim();
63
+
if (!q) return json({ actors: [] });
64
+
65
+
const response = await fetch(`${TYPEAHEAD_URL}?q=${encodeURIComponent(q)}&limit=6`, {
66
+
headers: { "x-client": "pensieve" }
67
+
});
68
+
const body = await response.json().catch(() => ({ actors: [] }));
69
+
return json({ actors: Array.isArray(body.actors) ? body.actors : [] });
70
+
}
71
+
72
+
async function search(request, env, url) {
73
+
if (request.method !== "GET") return json({ error: "method_not_allowed" }, 405);
74
+
75
+
const query = (url.searchParams.get("q") || "").trim();
76
+
const actor = (url.searchParams.get("actor") || "").replace(/^@+/, "").trim().toLowerCase();
77
+
const topK = clamp(Number(url.searchParams.get("top_k") || 8), 1, 20);
78
+
79
+
if (!query) return json({ error: "missing_query", message: "Tell Pensieve what you are trying to remember." }, 400);
80
+
if (!actor) return json({ error: "missing_actor", message: "Enter a name or handle first." }, 400);
81
+
if (!env.VOYAGE_API_KEY || !env.TURBOPUFFER_API_KEY) {
82
+
return json({ error: "missing_worker_secret", message: "Search is not configured yet." }, 500);
83
+
}
84
+
85
+
try {
86
+
const identity = await resolveIdentity(actor);
87
+
const namespace = actorNamespace(env, identity.did);
88
+
const vector = await embed(query, env.VOYAGE_API_KEY);
89
+
const body = await queryNamespace(namespace, query, vector, topK, env.TURBOPUFFER_API_KEY);
90
+
91
+
return json({
92
+
query,
93
+
actor: identity.handle || actor,
94
+
did: identity.did,
95
+
indexed: null,
96
+
needsIndex: body.missing,
97
+
results: readableResults(body.rows, topK)
98
+
});
99
+
} catch (error) {
100
+
console.error(JSON.stringify({
101
+
event: "search_failed",
102
+
actor,
103
+
message: error?.message || String(error),
104
+
stack: error?.stack || ""
105
+
}));
106
+
return json({ error: "search_unavailable", message: "Search is temporarily unavailable." }, 502);
107
+
}
108
+
}
109
+
110
+
async function update(request, env, url) {
111
+
if (request.method !== "GET") return json({ error: "method_not_allowed" }, 405);
112
+
113
+
const query = (url.searchParams.get("q") || "").trim();
114
+
const actor = (url.searchParams.get("actor") || "").replace(/^@+/, "").trim().toLowerCase();
115
+
const topK = clamp(Number(url.searchParams.get("top_k") || 8), 1, 20);
116
+
117
+
if (!query) return json({ error: "missing_query", message: "Write what you are trying to find." }, 400);
118
+
if (!actor) return json({ error: "missing_actor", message: "Enter a name or handle first." }, 400);
119
+
if (!env.VOYAGE_API_KEY || !env.TURBOPUFFER_API_KEY) {
120
+
return json({ error: "missing_worker_secret", message: "Search is not configured yet." }, 500);
121
+
}
122
+
123
+
const stream = new TransformStream();
124
+
const writer = stream.writable.getWriter();
125
+
const encoder = new TextEncoder();
126
+
127
+
const emit = async (event) => {
128
+
await writer.write(encoder.encode(`${JSON.stringify(event)}\n`));
129
+
};
130
+
131
+
(async () => {
132
+
try {
133
+
await emit({ type: "progress", phase: "resolve", label: "resolving handle", pct: 2 });
134
+
const identity = await resolveIdentity(actor);
135
+
const namespace = actorNamespace(env, identity.did);
136
+
137
+
const indexed = await indexActorFromCar(identity, namespace, env, {
138
+
replace: true,
139
+
onProgress: emit
140
+
});
141
+
142
+
await emit({ type: "progress", phase: "query", label: "searching rebuilt index", pct: 96 });
143
+
const vector = await embed(query, env.VOYAGE_API_KEY);
144
+
const body = await queryNamespace(namespace, query, vector, topK, env.TURBOPUFFER_API_KEY);
145
+
146
+
await emit({
147
+
type: "done",
148
+
data: {
149
+
query,
150
+
actor: identity.handle || actor,
151
+
did: identity.did,
152
+
indexed,
153
+
needsIndex: false,
154
+
results: readableResults(body.rows, topK)
155
+
}
156
+
});
157
+
} catch (error) {
158
+
console.error(JSON.stringify({
159
+
event: "update_failed",
160
+
actor,
161
+
message: error?.message || String(error),
162
+
stack: error?.stack || ""
163
+
}));
164
+
await emit({ type: "error", message: "Pensieve could not read that public history." });
165
+
} finally {
166
+
await writer.close();
167
+
}
168
+
})();
169
+
170
+
return new Response(stream.readable, {
171
+
headers: {
172
+
"content-type": "application/x-ndjson; charset=utf-8",
173
+
"cache-control": "no-store"
174
+
}
175
+
});
176
+
}
177
+
178
+
async function queryNamespace(namespace, query, vector, topK, apiKey) {
179
+
const [vectorResult, textResult] = await Promise.all([
180
+
queryVectorNamespace(namespace, vector, topK * 3, apiKey),
181
+
queryTextNamespace(namespace, query, topK * 3, apiKey)
182
+
]);
183
+
184
+
return {
185
+
missing: vectorResult.missing && textResult.missing,
186
+
rows: fuseRows(vectorResult.rows, textResult.rows, topK)
187
+
};
188
+
}
189
+
190
+
async function queryVectorNamespace(namespace, vector, topK, apiKey) {
191
+
const response = await fetch(`${TPUF_BASE}/${encodeURIComponent(namespace)}/query`, {
192
+
method: "POST",
193
+
headers: {
194
+
"authorization": `Bearer ${apiKey}`,
195
+
"content-type": "application/json"
196
+
},
197
+
body: JSON.stringify({
198
+
rank_by: ["vector", "ANN", vector],
199
+
top_k: topK,
200
+
include_attributes: true
201
+
})
202
+
});
203
+
204
+
const body = await response.json().catch(() => ({}));
205
+
if (response.status === 404) return { missing: true, rows: [] };
206
+
if (!response.ok) throw new Error("query_failed");
207
+
return { missing: false, rows: Array.isArray(body.rows) ? body.rows : [] };
208
+
}
209
+
210
+
async function queryTextNamespace(namespace, query, topK, apiKey) {
211
+
const response = await fetch(`${TPUF_BASE}/${encodeURIComponent(namespace)}/query`, {
212
+
method: "POST",
213
+
headers: {
214
+
"authorization": `Bearer ${apiKey}`,
215
+
"content-type": "application/json"
216
+
},
217
+
body: JSON.stringify({
218
+
rank_by: ["text", "BM25", query],
219
+
top_k: topK,
220
+
include_attributes: true
221
+
})
222
+
});
223
+
224
+
const body = await response.json().catch(() => ({}));
225
+
if (response.status === 404) return { missing: true, rows: [] };
226
+
if (!response.ok) {
227
+
console.warn(JSON.stringify({
228
+
event: "bm25_query_failed",
229
+
status: response.status,
230
+
detail: JSON.stringify(body).slice(0, 300)
231
+
}));
232
+
return { missing: false, rows: [] };
233
+
}
234
+
return { missing: false, rows: Array.isArray(body.rows) ? body.rows : [] };
235
+
}
236
+
237
+
function fuseRows(vectorRows, textRows, topK) {
238
+
const byId = new Map();
239
+
addRankedRows(byId, vectorRows, "vectorRank", 1.0);
240
+
addRankedRows(byId, textRows, "textRank", 1.35);
241
+
242
+
return Array.from(byId.values())
243
+
.sort((a, b) => b.score - a.score)
244
+
.slice(0, topK)
245
+
.map(({ row }) => row);
246
+
}
247
+
248
+
function addRankedRows(byId, rows, rankName, weight) {
249
+
rows.forEach((row, index) => {
250
+
const id = row.id || row.uri || String(index);
251
+
const current = byId.get(id) || { row, score: 0 };
252
+
current.row = { ...current.row, ...row };
253
+
current[rankName] = index + 1;
254
+
current.score += weight / (60 + index + 1);
255
+
byId.set(id, current);
256
+
});
257
+
}
258
+
259
+
async function embed(text, apiKey) {
260
+
return (await embedMany([text], "query", apiKey))[0];
261
+
}
262
+
263
+
async function embedMany(texts, inputType, apiKey) {
264
+
const response = await fetch(VOYAGE_URL, {
265
+
method: "POST",
266
+
headers: {
267
+
"authorization": `Bearer ${apiKey}`,
268
+
"content-type": "application/json"
269
+
},
270
+
body: JSON.stringify({
271
+
model: MODEL,
272
+
input_type: inputType,
273
+
output_dimension: DIMENSIONS,
274
+
input: texts
275
+
})
276
+
});
277
+
278
+
const body = await response.json().catch(() => ({}));
279
+
if (!response.ok || !Array.isArray(body.data)) throw new Error("embedding_failed");
280
+
return body.data.map((row) => row.embedding);
281
+
}
282
+
283
+
async function resolveIdentity(actor) {
284
+
return actorResolver.resolve(actor);
285
+
}
286
+
287
+
async function indexActorFromCar(identity, namespace, env, options = {}) {
288
+
const onProgress = options.onProgress || (async () => {});
289
+
if (options.replace) {
290
+
await onProgress({ type: "progress", phase: "prepare", label: "preparing index update", pct: 5 });
291
+
await deleteNamespace(namespace, env.TURBOPUFFER_API_KEY);
292
+
}
293
+
294
+
await onProgress({ type: "progress", phase: "fetch", label: "fetching repo CAR", pct: 8 });
295
+
const getRepoUrl = new URL("/xrpc/com.atproto.sync.getRepo", identity.pds);
296
+
getRepoUrl.searchParams.set("did", identity.did);
297
+
const response = await fetch(getRepoUrl);
298
+
if (!response.ok || !response.body) {
299
+
throw new Error(`get_repo_failed:${response.status}:${identity.pds}`);
300
+
}
301
+
302
+
const repo = repoFromStream(response.body);
303
+
const items = [];
304
+
const stats = {
305
+
scanned: 0,
306
+
indexed: 0,
307
+
skippedNoText: 0,
308
+
missingBlocks: 0
309
+
};
310
+
311
+
try {
312
+
for await (const entry of repo) {
313
+
stats.scanned += 1;
314
+
if (stats.scanned === 1 || stats.scanned % 250 === 0) {
315
+
await onProgress({
316
+
type: "progress",
317
+
phase: "walk",
318
+
label: `walking CAR · scanned ${stats.scanned.toLocaleString()} records`,
319
+
scanned: stats.scanned,
320
+
indexed: items.length,
321
+
pct: Math.min(38, 10 + Math.floor(stats.scanned / 250))
322
+
});
323
+
}
324
+
325
+
const artifact = inferArtifact(entry.collection, entry.record);
326
+
const text = artifactSearchText(entry.collection, artifact);
327
+
if (!text.trim()) {
328
+
stats.skippedNoText += 1;
329
+
continue;
330
+
}
331
+
332
+
items.push({
333
+
uri: `at://${identity.did}/${entry.collection}/${entry.rkey}`,
334
+
cid: cidString(entry.cid),
335
+
collection: entry.collection,
336
+
authorDid: identity.did,
337
+
createdAt: findIsoDate(entry.record),
338
+
text,
339
+
artifact
340
+
});
341
+
342
+
if (items.length >= INDEX_LIMIT) break;
343
+
}
344
+
} finally {
345
+
stats.missingBlocks = repo.missingBlocks.length;
346
+
await repo.dispose();
347
+
}
348
+
349
+
await onProgress({
350
+
type: "progress",
351
+
phase: "embed",
352
+
label: `embedding ${items.length.toLocaleString()} searchable records`,
353
+
scanned: stats.scanned,
354
+
indexed: 0,
355
+
total: items.length,
356
+
pct: 42
357
+
});
358
+
359
+
for (let i = 0; i < items.length; i += INDEX_BATCH_SIZE) {
360
+
const batch = items.slice(i, i + INDEX_BATCH_SIZE);
361
+
const vectors = await embedMany(batch.map((item) => item.text), "document", env.VOYAGE_API_KEY);
362
+
const rows = batch.map((item, index) => ({
363
+
id: rowId(item.uri),
364
+
vector: vectors[index],
365
+
uri: item.uri,
366
+
cid: item.cid,
367
+
collection: item.collection,
368
+
authorDid: item.authorDid,
369
+
createdAt: item.createdAt,
370
+
text: item.text,
371
+
artifactKind: item.artifact.kind,
372
+
artifactTitle: item.artifact.title,
373
+
artifactBody: item.artifact.body,
374
+
artifactUrl: item.artifact.url,
375
+
artifactMedia: JSON.stringify(item.artifact.media || []),
376
+
artifactRefs: JSON.stringify(item.artifact.refs || []),
377
+
artifactDate: item.artifact.date || item.createdAt,
378
+
artifactConfidence: String(item.artifact.confidence || "low")
379
+
}));
380
+
381
+
const writeBody = {
382
+
distance_metric: "cosine_distance",
383
+
schema: {
384
+
text: {
385
+
type: "string",
386
+
full_text_search: true
387
+
},
388
+
artifactKind: {
389
+
type: "string"
390
+
},
391
+
artifactTitle: {
392
+
type: "string",
393
+
full_text_search: true
394
+
},
395
+
artifactBody: {
396
+
type: "string",
397
+
full_text_search: true
398
+
},
399
+
artifactUrl: {
400
+
type: "string"
401
+
},
402
+
artifactMedia: {
403
+
type: "string"
404
+
},
405
+
artifactRefs: {
406
+
type: "string"
407
+
},
408
+
artifactDate: {
409
+
type: "string"
410
+
},
411
+
artifactConfidence: {
412
+
type: "string"
413
+
}
414
+
},
415
+
upsert_rows: rows
416
+
};
417
+
418
+
const upsert = await fetch(`${TPUF_BASE}/${encodeURIComponent(namespace)}`, {
419
+
method: "POST",
420
+
headers: {
421
+
"authorization": `Bearer ${env.TURBOPUFFER_API_KEY}`,
422
+
"content-type": "application/json"
423
+
},
424
+
body: JSON.stringify(writeBody)
425
+
});
426
+
if (!upsert.ok) {
427
+
const detail = await upsert.text().catch(() => "");
428
+
throw new Error(`index_failed:${upsert.status}:${detail.slice(0, 500)}`);
429
+
}
430
+
stats.indexed += rows.length;
431
+
await onProgress({
432
+
type: "progress",
433
+
phase: "embed",
434
+
label: `embedded ${stats.indexed.toLocaleString()} / ${items.length.toLocaleString()} records`,
435
+
scanned: stats.scanned,
436
+
indexed: stats.indexed,
437
+
total: items.length,
438
+
pct: 42 + Math.round((stats.indexed / Math.max(items.length, 1)) * 50)
439
+
});
440
+
}
441
+
442
+
return stats;
443
+
}
444
+
445
+
async function deleteNamespace(namespace, apiKey) {
446
+
const response = await fetch(`${TPUF_BASE}/${encodeURIComponent(namespace)}`, {
447
+
method: "DELETE",
448
+
headers: {
449
+
"authorization": `Bearer ${apiKey}`
450
+
}
451
+
});
452
+
if (response.ok || response.status === 404) return;
453
+
const detail = await response.text().catch(() => "");
454
+
throw new Error(`delete_namespace_failed:${response.status}:${detail.slice(0, 300)}`);
455
+
}
456
+
457
+
function actorNamespace(env, did) {
458
+
const prefix = env.TURBOPUFFER_NAMESPACE_PREFIX || "pensieve";
459
+
return `${prefix}-${did.replace(/[^a-zA-Z0-9_-]/g, "-")}`;
460
+
}
461
+
462
+
function cidString(cid) {
463
+
if (typeof cid === "string") return cid;
464
+
if (cid?.$link) return cid.$link;
465
+
return String(cid);
466
+
}
467
+
468
+
function rowId(uri) {
469
+
let hash = 2166136261;
470
+
for (let i = 0; i < uri.length; i += 1) {
471
+
hash ^= uri.charCodeAt(i);
472
+
hash = Math.imul(hash, 16777619);
473
+
}
474
+
return `${hash >>> 0}-${uri.slice(-48).replace(/[^a-zA-Z0-9_-]/g, "-")}`;
475
+
}
476
+
477
+
function inferArtifact(collection, value) {
478
+
const fields = [];
479
+
const urls = [];
480
+
const refs = [];
481
+
const media = [];
482
+
const dates = [];
483
+
walkArtifact(value, "", { fields, urls, refs, media, dates });
484
+
485
+
const title = pickField(fields, titleScore);
486
+
const body = pickField(fields, (field) => bodyScore(field, title?.value));
487
+
const url = pickUrl(urls);
488
+
const date = pickDate(dates) || findIsoDate(value);
489
+
const titleValue = title?.value || "";
490
+
const bodyValue = body?.value && body.value !== titleValue ? body.value : "";
491
+
const confidence = titleValue || bodyValue ? "medium" : "low";
492
+
493
+
return {
494
+
kind: collectionKind(collection),
495
+
title: titleValue,
496
+
body: bodyValue,
497
+
url: url?.value || "",
498
+
media: media.slice(0, 4),
499
+
refs: refs.slice(0, 6),
500
+
date,
501
+
confidence
502
+
};
503
+
}
504
+
505
+
function artifactSearchText(collection, artifact) {
506
+
const lines = [
507
+
`collection: ${collection}`,
508
+
`kind: ${artifact.kind || collectionKind(collection)}`,
509
+
artifact.title ? `title: ${artifact.title}` : "",
510
+
artifact.body ? `body: ${artifact.body}` : "",
511
+
artifact.url ? `url: ${artifact.url}` : "",
512
+
...(artifact.media || []).map((item) => item.alt ? `media: ${item.alt}` : ""),
513
+
...(artifact.refs || []).map((item) => item.uri ? `reference: ${item.uri}` : "")
514
+
].filter(Boolean);
515
+
return lines.join("\n").slice(0, MAX_TEXT_LEN);
516
+
}
517
+
518
+
function walkArtifact(value, path, out) {
519
+
if (isStrongRef(value)) {
520
+
out.refs.push({ uri: value.uri, cid: value.cid, path });
521
+
return;
522
+
}
523
+
524
+
if (Array.isArray(value)) {
525
+
if (value.every((item) => typeof item === "string" || typeof item === "number")) {
526
+
const kept = value.map(String).filter((item) => isSemanticString(path, item) || looksLikeUrl(item));
527
+
if (kept.length) addArtifactValue(path, kept.join(", "), out);
528
+
return;
529
+
}
530
+
for (const item of value) walkArtifact(item, path, out);
531
+
return;
532
+
}
533
+
534
+
if (value && typeof value === "object") {
535
+
const blobCid = blobCidFrom(value);
536
+
if (blobCid) {
537
+
const mimeType = typeof value.mimeType === "string" ? value.mimeType : "";
538
+
if (mimeType.startsWith("image/")) out.media.push({ cid: blobCid, path, alt: "" });
539
+
return;
540
+
}
541
+
for (const [key, child] of Object.entries(value)) {
542
+
if (isNoiseKey(key)) continue;
543
+
const nextPath = path ? `${path}.${key}` : key;
544
+
walkArtifact(child, nextPath, out);
545
+
}
546
+
return;
547
+
}
548
+
549
+
if (typeof value === "string" || typeof value === "number") {
550
+
addArtifactValue(path, value, out);
551
+
}
552
+
}
553
+
554
+
function addArtifactValue(path, raw, out) {
555
+
const value = String(raw).trim();
556
+
if (!value) return;
557
+
if (looksLikeIsoTimestamp(value)) {
558
+
out.dates.push({ path, value });
559
+
return;
560
+
}
561
+
if (looksLikeUrl(value)) {
562
+
out.urls.push({ path, value });
563
+
return;
564
+
}
565
+
if (typeof raw === "number" && (!isSemanticNumberPath(path) || raw === 0)) return;
566
+
if (isSemanticString(path, value) || typeof raw === "number") out.fields.push({ path, value });
567
+
}
568
+
569
+
function pickField(fields, scoreFn) {
570
+
return fields
571
+
.map((field, index) => ({ field, score: scoreFn(field) - index * 0.001 }))
572
+
.filter((item) => item.score > 0)
573
+
.sort((a, b) => b.score - a.score)[0]?.field || null;
574
+
}
575
+
576
+
function titleScore(field) {
577
+
const leaf = pathLeaf(field.path);
578
+
let score = 0;
579
+
if (["title", "name", "trackName", "eventName", "displayName", "label"].includes(leaf)) score += 10;
580
+
if (["artistName", "author", "creator", "service", "musicServiceBaseDomain"].includes(leaf)) score += 4;
581
+
if (field.value.length > 160) score -= 5;
582
+
if (/^[a-z0-9.-]+\.[a-z]{2,}$/i.test(field.value)) score -= 2;
583
+
return score;
584
+
}
585
+
586
+
function bodyScore(field, titleValue = "") {
587
+
const leaf = pathLeaf(field.path);
588
+
let score = 0;
589
+
if (["text", "body", "description", "content", "summary", "note", "comment", "message", "caption", "alt"].includes(leaf)) score += 10;
590
+
if (field.path.endsWith(".text")) score += 3;
591
+
if (["artistName", "name"].includes(leaf)) score += 4;
592
+
if (field.value === titleValue) score -= 20;
593
+
if (field.value.length > 1200) score -= 4;
594
+
return score;
595
+
}
596
+
597
+
function pickUrl(urls) {
598
+
return urls
599
+
.map((url, index) => ({ url, score: urlScore(url) - index * 0.001 }))
600
+
.sort((a, b) => b.score - a.score)[0]?.url || null;
601
+
}
602
+
603
+
function urlScore(url) {
604
+
const leaf = pathLeaf(url.path);
605
+
let score = 1;
606
+
if (["url", "uri", "href", "link", "originUrl", "externalUri"].includes(leaf)) score += 5;
607
+
if (/avatar|icon|image|thumb/i.test(url.path)) score -= 3;
608
+
return score;
609
+
}
610
+
611
+
function pickDate(dates) {
612
+
return dates
613
+
.map((date, index) => ({ date, score: dateScore(date) - index * 0.001 }))
614
+
.sort((a, b) => b.score - a.score)[0]?.date.value || "";
615
+
}
616
+
617
+
function dateScore(date) {
618
+
const leaf = pathLeaf(date.path);
619
+
let score = 1;
620
+
if (["createdAt", "playedTime", "startTime", "publishedAt", "updatedAt"].includes(leaf)) score += 8;
621
+
if (["indexedAt"].includes(leaf)) score -= 4;
622
+
return score;
623
+
}
624
+
625
+
function collectionKind(collection) {
626
+
return humanWords(String(collection || "").split(".").pop() || "item");
627
+
}
628
+
629
+
function appendTextLine(lines, path, raw) {
630
+
const value = String(raw).trim();
631
+
if (!value) return;
632
+
lines.push(path ? `${path}: ${value}` : value);
633
+
}
634
+
635
+
function isNoiseKey(key) {
636
+
return key.startsWith("$") || ["cid", "rev", "sig", "prev", "version", "ref"].includes(key);
637
+
}
638
+
639
+
function pathLeaf(path) {
640
+
return String(path || "").split(".").pop() || "";
641
+
}
642
+
643
+
function looksLikeUrl(value) {
644
+
return /^https?:\/\//i.test(String(value).trim());
645
+
}
646
+
647
+
function blobCidFrom(value) {
648
+
return value?.ref?.$link || value?.ref?.["$link"] || value?.["$link"] || "";
649
+
}
650
+
651
+
function isIdentifier(value) {
652
+
const s = String(value).trim();
653
+
if (!s) return true;
654
+
if (s.startsWith("did:plc:") || s.startsWith("did:web:") || s.startsWith("at://")) return true;
655
+
if (s.startsWith("bafy") || s.startsWith("bafk")) return true;
656
+
if (/^[a-z2-7]{13}$/.test(s)) return true;
657
+
if (/^[0-9a-f]{32,}$/i.test(s)) return true;
658
+
return false;
659
+
}
660
+
661
+
function isSemanticString(path, value) {
662
+
const s = String(value).trim();
663
+
if (s.length < 2) return false;
664
+
if (isIdentifier(s) || looksLikeIsoTimestamp(s)) return false;
665
+
if (/^https?:\/\//i.test(s)) return false;
666
+
if (/^[a-z0-9._:-]+$/i.test(s) && s.length > 24) return false;
667
+
668
+
const leaf = path.split(".").pop() || "";
669
+
if (["uri", "url", "did", "rkey", "subject", "createdAt", "updatedAt", "indexedAt"].includes(leaf)) return false;
670
+
671
+
return /[\p{L}\p{N}]/u.test(s);
672
+
}
673
+
674
+
function isSemanticNumberPath(path) {
675
+
const leaf = path.split(".").pop() || "";
676
+
return ["rating", "score", "count", "position", "duration", "year"].includes(leaf);
677
+
}
678
+
679
+
function looksLikeIsoTimestamp(value) {
680
+
return /^\d{4}-\d{2}-\d{2}[T ]/.test(value);
681
+
}
682
+
683
+
function isStrongRef(value) {
684
+
return !!(
685
+
value &&
686
+
typeof value === "object" &&
687
+
!Array.isArray(value) &&
688
+
typeof value.uri === "string" &&
689
+
typeof value.cid === "string" &&
690
+
value.uri.startsWith("at://")
691
+
);
692
+
}
693
+
694
+
function findIsoDate(value) {
695
+
if (!value || typeof value !== "object") return "";
696
+
if (typeof value.createdAt === "string") return value.createdAt;
697
+
for (const child of Object.values(value)) {
698
+
if (child && typeof child === "object") {
699
+
const found = findIsoDate(child);
700
+
if (found) return found;
701
+
}
702
+
}
703
+
return "";
704
+
}
705
+
706
+
function readableResult(row) {
707
+
const collection = row.collection || collectionFromUri(row.uri || "");
708
+
const artifact = rowToArtifact(collection, row);
709
+
return {
710
+
uri: row.uri || "",
711
+
collection,
712
+
date: humanDate(artifact.date || row.createdAt),
713
+
body: artifact.body || artifact.title || cleanRecordText(row.text || ""),
714
+
artifact,
715
+
distance: typeof row.$dist === "number" ? row.$dist : row.dist
716
+
};
717
+
}
718
+
719
+
function readableResults(rows, topK) {
720
+
const byArtifact = new Map();
721
+
for (const result of (Array.isArray(rows) ? rows : []).map(readableResult)) {
722
+
const key = artifactKey(result);
723
+
const current = byArtifact.get(key);
724
+
if (current) {
725
+
current.count += 1;
726
+
continue;
727
+
}
728
+
byArtifact.set(key, { ...result, count: 1 });
729
+
}
730
+
return Array.from(byArtifact.values()).slice(0, topK);
731
+
}
732
+
733
+
function artifactKey(result) {
734
+
const artifact = result.artifact || {};
735
+
return [
736
+
result.collection || "",
737
+
artifact.kind || "",
738
+
artifact.title || "",
739
+
artifact.body || result.body || "",
740
+
artifact.url || ""
741
+
].join("\u0000").toLowerCase();
742
+
}
743
+
744
+
function rowToArtifact(collection, row) {
745
+
const hasArtifact = row.artifactTitle || row.artifactBody || row.artifactUrl || row.artifactKind;
746
+
if (hasArtifact) {
747
+
return {
748
+
kind: row.artifactKind || collectionKind(collection),
749
+
title: row.artifactTitle || "",
750
+
body: row.artifactBody || "",
751
+
url: row.artifactUrl || "",
752
+
media: parseJsonArray(row.artifactMedia),
753
+
refs: parseJsonArray(row.artifactRefs),
754
+
date: row.artifactDate || row.createdAt || "",
755
+
confidence: row.artifactConfidence || "low"
756
+
};
757
+
}
758
+
759
+
const text = cleanRecordText(row.text || "");
760
+
const lines = text.split("\n").map((line) => line.trim()).filter(Boolean);
761
+
const title = lines.length > 1 && lines[0].length <= 90 ? lines[0] : "";
762
+
const body = title ? lines.slice(1).join("\n") : text;
763
+
return {
764
+
kind: collectionKind(collection),
765
+
title,
766
+
body,
767
+
url: firstUrl(text),
768
+
media: [],
769
+
refs: [],
770
+
date: row.createdAt || "",
771
+
confidence: "low"
772
+
};
773
+
}
774
+
775
+
function parseJsonArray(value) {
776
+
if (!value) return [];
777
+
if (Array.isArray(value)) return value;
778
+
try {
779
+
const parsed = JSON.parse(value);
780
+
return Array.isArray(parsed) ? parsed : [];
781
+
} catch {
782
+
return [];
783
+
}
784
+
}
785
+
786
+
function firstUrl(value) {
787
+
return String(value || "").match(/https?:\/\/[^\s)]+/)?.[0] || "";
788
+
}
789
+
790
+
function collectionFromUri(uri) {
791
+
const parts = String(uri || "").split("/");
792
+
return parts.length >= 4 ? parts[3] : "";
793
+
}
794
+
795
+
function cleanRecordText(text) {
796
+
const lines = text.split("\n");
797
+
const textLine = lines.find((line) => line.startsWith("text: "));
798
+
if (textLine) {
799
+
const start = lines.indexOf(textLine);
800
+
const body = [];
801
+
for (const line of lines.slice(start)) {
802
+
if (body.length > 0 && /^[a-zA-Z0-9_.]+: /.test(line)) break;
803
+
body.push(line.replace(/^text: /, ""));
804
+
}
805
+
return body.join("\n").trim();
806
+
}
807
+
808
+
return lines
809
+
.filter((line) => !line.startsWith("collection: "))
810
+
.map((line) => line.replace(/^[a-zA-Z0-9_.]+: /, ""))
811
+
.filter((line) => line.trim())
812
+
.join("\n")
813
+
.trim()
814
+
.slice(0, 900);
815
+
}
816
+
817
+
function humanWords(value) {
818
+
return String(value || "item")
819
+
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
820
+
.replace(/[-_]+/g, " ")
821
+
.toLowerCase();
822
+
}
823
+
824
+
function humanDate(value) {
825
+
if (!value) return "";
826
+
const date = new Date(value);
827
+
if (Number.isNaN(date.valueOf())) return "";
828
+
return date.toLocaleDateString("en-US", { month: "short", day: "numeric", year: "numeric" });
829
+
}
830
+
831
+
function json(body, status = 200) {
832
+
return new Response(JSON.stringify(body), {
833
+
status,
834
+
headers: {
835
+
"content-type": "application/json; charset=utf-8",
836
+
"cache-control": "no-store"
837
+
}
838
+
});
839
+
}
840
+
841
+
function clamp(value, min, max) {
842
+
if (!Number.isFinite(value)) return min;
843
+
return Math.min(max, Math.max(min, Math.round(value)));
844
+
}
845
+
846
+
function renderIndex() {
847
+
return `<!doctype html>
848
+
<html lang="en">
849
+
<head>
850
+
<meta charset="utf-8">
851
+
<meta name="viewport" content="width=device-width, initial-scale=1">
852
+
<title>Pensieve</title>
853
+
<style>
854
+
:root {
855
+
color-scheme: dark;
856
+
--bg: #120d08;
857
+
--paper: #d9c28f;
858
+
--paper-2: #c4a86e;
859
+
--ink: #2a1710;
860
+
--ink-soft: #5f4229;
861
+
--ink-faint: #876b45;
862
+
--line: #7c5f37;
863
+
--line-strong: #4c2d1c;
864
+
--text: #2a1710;
865
+
--muted: #5f4229;
866
+
--dim: #876b45;
867
+
--water: #dff8f5;
868
+
--water-2: #7aa0a2;
869
+
--brass: #9b6b22;
870
+
--candle: #ffd985;
871
+
--danger: #8d2e20;
872
+
--shadow: rgba(0, 0, 0, .38);
873
+
--sans: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
874
+
--serif: Iowan Old Style, Palatino, Palatino Linotype, Book Antiqua, Georgia, serif;
875
+
--mono: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", monospace;
876
+
font-family: var(--sans);
877
+
}
878
+
879
+
* { box-sizing: border-box; }
880
+
881
+
body {
882
+
margin: 0;
883
+
min-height: 100vh;
884
+
background:
885
+
radial-gradient(ellipse at 16% 0%, rgba(255, 217, 133, .26), transparent 34%),
886
+
radial-gradient(ellipse at 76% 6%, rgba(223, 248, 245, .12), transparent 42%),
887
+
linear-gradient(90deg, rgba(255,255,255,.025) 1px, transparent 1px),
888
+
linear-gradient(rgba(255,255,255,.02) 1px, transparent 1px),
889
+
linear-gradient(180deg, rgba(67, 38, 20, .64), transparent 300px),
890
+
var(--bg);
891
+
background-size: auto, auto, 96px 96px, 96px 48px, auto, auto;
892
+
color: var(--text);
893
+
}
894
+
895
+
.page {
896
+
width: min(930px, calc(100% - 28px));
897
+
margin: 0 auto;
898
+
padding: 24px 0 56px;
899
+
}
900
+
901
+
header {
902
+
display: flex;
903
+
justify-content: space-between;
904
+
align-items: center;
905
+
gap: 16px;
906
+
padding-bottom: 24px;
907
+
}
908
+
909
+
.wordmark {
910
+
display: inline-flex;
911
+
align-items: center;
912
+
gap: 12px;
913
+
font-family: var(--serif);
914
+
font-weight: 600;
915
+
font-size: 22px;
916
+
color: #f5ead2;
917
+
letter-spacing: 0;
918
+
text-shadow: 0 1px 0 rgba(0,0,0,.36);
919
+
}
920
+
921
+
.wordmark::before {
922
+
content: "";
923
+
width: 34px;
924
+
height: 34px;
925
+
border-radius: 50%;
926
+
border: 1px solid rgba(217, 194, 143, .86);
927
+
background:
928
+
repeating-radial-gradient(circle, transparent 0 5px, rgba(183, 239, 231, .42) 6px 7px),
929
+
radial-gradient(circle at 50% 54%, rgba(183, 239, 231, .18), rgba(8, 7, 6, .34) 62%);
930
+
box-shadow: inset 0 0 18px rgba(183, 239, 231, .16), 0 0 0 4px rgba(214, 181, 99, .08);
931
+
}
932
+
933
+
.note {
934
+
color: #d9caa9;
935
+
font-family: var(--mono);
936
+
font-size: 12px;
937
+
}
938
+
939
+
.memory-basin {
940
+
position: relative;
941
+
overflow: visible;
942
+
border: 1px solid rgba(76, 45, 28, .72);
943
+
border-radius: 14px 58px 22px 46px;
944
+
padding: 34px 26px 26px;
945
+
background:
946
+
radial-gradient(ellipse at 17% 13%, rgba(255, 246, 210, .6), transparent 30%),
947
+
radial-gradient(ellipse at 83% 23%, rgba(88, 48, 21, .22), transparent 28%),
948
+
linear-gradient(115deg, rgba(72, 38, 19, .1), transparent 42%),
949
+
repeating-linear-gradient(2deg, rgba(76, 45, 28, .065) 0 1px, transparent 1px 7px),
950
+
var(--paper);
951
+
box-shadow:
952
+
inset 0 0 0 1px rgba(255, 255, 255, .18),
953
+
inset 0 -24px 70px rgba(89, 47, 24, .16),
954
+
0 30px 90px rgba(0,0,0,.42);
955
+
}
956
+
957
+
.memory-basin::before {
958
+
content: "";
959
+
position: absolute;
960
+
inset: 16px 28px auto;
961
+
height: 120px;
962
+
border-radius: 52% 48% 46% 54%;
963
+
border: 1px solid rgba(76, 45, 28, .2);
964
+
background:
965
+
repeating-radial-gradient(ellipse at 50% 50%, transparent 0 17px, rgba(42, 23, 16, .13) 18px 19px),
966
+
radial-gradient(ellipse at 50% 48%, rgba(223, 248, 245, .3), rgba(124, 160, 162, .16) 44%, transparent 70%);
967
+
pointer-events: none;
968
+
opacity: .74;
969
+
mask-image: linear-gradient(180deg, black, transparent 72%);
970
+
}
971
+
972
+
.memory-basin::after {
973
+
content: "";
974
+
position: absolute;
975
+
right: 34px;
976
+
top: 28px;
977
+
width: 148px;
978
+
height: 86px;
979
+
border-radius: 50%;
980
+
border: 1px solid rgba(42, 23, 16, .52);
981
+
background:
982
+
repeating-radial-gradient(ellipse at 50% 50%, transparent 0 9px, rgba(42, 23, 16, .22) 10px 11px),
983
+
radial-gradient(ellipse at 50% 50%, rgba(237, 252, 249, .58), rgba(122, 160, 162, .24) 48%, rgba(42,23,16,.16) 70%);
984
+
box-shadow:
985
+
inset 0 0 26px rgba(42,23,16,.34),
986
+
0 0 0 5px rgba(245,234,210,.2),
987
+
0 18px 40px rgba(42,23,16,.22);
988
+
pointer-events: none;
989
+
}
990
+
991
+
.sconces {
992
+
position: absolute;
993
+
inset: 92px 54px auto;
994
+
display: flex;
995
+
justify-content: space-between;
996
+
pointer-events: none;
997
+
z-index: 0;
998
+
}
999
+
1000
+
.sconce {
1001
+
width: 18px;
1002
+
height: 9px;
1003
+
border-radius: 50%;
1004
+
background: rgba(42, 23, 16, .42);
1005
+
box-shadow: 0 0 0 1px rgba(42, 23, 16, .18);
1006
+
opacity: .6;
1007
+
}
1008
+
1009
+
.sconce::after {
1010
+
content: "";
1011
+
display: block;
1012
+
width: 8px;
1013
+
height: 8px;
1014
+
margin: -8px auto 0;
1015
+
border-radius: 999px 999px 999px 0;
1016
+
transform: rotate(-35deg);
1017
+
background: var(--candle);
1018
+
box-shadow: 0 0 18px rgba(255, 217, 133, .74), 0 0 44px rgba(255, 217, 133, .34);
1019
+
}
1020
+
1021
+
.hero {
1022
+
display: grid;
1023
+
grid-template-columns: minmax(0, 1fr) 190px;
1024
+
gap: 24px;
1025
+
align-items: center;
1026
+
margin-bottom: 24px;
1027
+
position: relative;
1028
+
z-index: 1;
1029
+
}
1030
+
1031
+
.kicker {
1032
+
margin: 0 0 8px;
1033
+
color: var(--brass);
1034
+
font-family: var(--mono);
1035
+
font-size: 12px;
1036
+
font-weight: 700;
1037
+
}
1038
+
1039
+
h1 {
1040
+
margin: 0;
1041
+
max-width: 520px;
1042
+
font-family: var(--serif);
1043
+
font-size: clamp(36px, 6vw, 58px);
1044
+
line-height: .98;
1045
+
font-weight: 560;
1046
+
color: var(--ink);
1047
+
letter-spacing: 0;
1048
+
}
1049
+
1050
+
.intro {
1051
+
max-width: 520px;
1052
+
margin: 12px 0 0;
1053
+
color: var(--muted);
1054
+
font-size: 16px;
1055
+
line-height: 1.5;
1056
+
}
1057
+
1058
+
.basin {
1059
+
display: none;
1060
+
}
1061
+
1062
+
.map-trail {
1063
+
position: absolute;
1064
+
left: 38px;
1065
+
right: 38px;
1066
+
top: 224px;
1067
+
height: 72px;
1068
+
pointer-events: none;
1069
+
opacity: .68;
1070
+
background:
1071
+
radial-gradient(ellipse at 8% 55%, rgba(42,23,16,.5) 0 4px, transparent 5px),
1072
+
radial-gradient(ellipse at 14% 44%, rgba(42,23,16,.5) 0 3px, transparent 4px),
1073
+
radial-gradient(ellipse at 28% 66%, rgba(42,23,16,.42) 0 4px, transparent 5px),
1074
+
radial-gradient(ellipse at 36% 48%, rgba(42,23,16,.42) 0 3px, transparent 4px),
1075
+
radial-gradient(ellipse at 58% 58%, rgba(42,23,16,.4) 0 4px, transparent 5px),
1076
+
radial-gradient(ellipse at 66% 40%, rgba(42,23,16,.4) 0 3px, transparent 4px),
1077
+
radial-gradient(ellipse at 84% 60%, rgba(42,23,16,.38) 0 4px, transparent 5px),
1078
+
radial-gradient(ellipse at 91% 44%, rgba(42,23,16,.38) 0 3px, transparent 4px);
1079
+
transform: rotate(-2deg);
1080
+
}
1081
+
1082
+
.map-trail::before {
1083
+
content: "";
1084
+
position: absolute;
1085
+
left: 5%;
1086
+
right: 6%;
1087
+
top: 32px;
1088
+
border-top: 1px dashed rgba(42, 23, 16, .24);
1089
+
transform: rotate(1deg);
1090
+
}
1091
+
1092
+
.search-panel {
1093
+
position: relative;
1094
+
z-index: 20;
1095
+
overflow: visible;
1096
+
background:
1097
+
linear-gradient(180deg, rgba(255,255,255,.22), transparent),
1098
+
repeating-linear-gradient(1deg, rgba(42,23,16,.045) 0 1px, transparent 1px 9px),
1099
+
rgba(235, 216, 163, .54);
1100
+
border: 1px solid rgba(76, 45, 28, .44);
1101
+
border-radius: 7px;
1102
+
padding: 14px;
1103
+
box-shadow:
1104
+
inset 0 1px 0 rgba(255,255,255,.22),
1105
+
0 10px 26px rgba(42,23,16,.14);
1106
+
}
1107
+
1108
+
.panel-title {
1109
+
display: flex;
1110
+
justify-content: space-between;
1111
+
gap: 14px;
1112
+
margin: 0 0 14px;
1113
+
color: var(--ink-soft);
1114
+
font-family: var(--mono);
1115
+
font-size: 12px;
1116
+
}
1117
+
1118
+
.panel-title span:last-child {
1119
+
color: var(--ink-faint);
1120
+
}
1121
+
1122
+
.steps {
1123
+
display: grid;
1124
+
grid-template-columns: 1fr;
1125
+
gap: 12px;
1126
+
align-items: end;
1127
+
}
1128
+
1129
+
.actions {
1130
+
display: block;
1131
+
}
1132
+
1133
+
.actions button {
1134
+
width: 100%;
1135
+
}
1136
+
1137
+
.field {
1138
+
min-width: 0;
1139
+
}
1140
+
1141
+
label {
1142
+
display: block;
1143
+
color: var(--ink-soft);
1144
+
font-family: var(--mono);
1145
+
font-size: 11px;
1146
+
margin-bottom: 7px;
1147
+
}
1148
+
1149
+
input {
1150
+
width: 100%;
1151
+
height: 48px;
1152
+
border: 1px solid var(--line);
1153
+
border-radius: 5px;
1154
+
background: rgba(255, 244, 205, .42);
1155
+
color: var(--ink);
1156
+
font-family: var(--serif);
1157
+
font-size: 18px;
1158
+
font-weight: 540;
1159
+
padding: 0 14px;
1160
+
outline: none;
1161
+
}
1162
+
1163
+
input::placeholder { color: rgba(95, 66, 41, .68); }
1164
+
1165
+
input:focus {
1166
+
border-color: rgba(42, 23, 16, .76);
1167
+
box-shadow: 0 0 0 3px rgba(42, 23, 16, .12);
1168
+
}
1169
+
1170
+
.person-wrap {
1171
+
position: relative;
1172
+
z-index: 30;
1173
+
}
1174
+
1175
+
.person {
1176
+
display: flex;
1177
+
align-items: center;
1178
+
gap: 10px;
1179
+
height: 48px;
1180
+
border: 1px solid var(--line);
1181
+
border-radius: 5px;
1182
+
padding: 0 12px;
1183
+
background: rgba(255, 244, 205, .42);
1184
+
}
1185
+
1186
+
.person:focus-within {
1187
+
border-color: rgba(42, 23, 16, .76);
1188
+
box-shadow: 0 0 0 3px rgba(42, 23, 16, .12);
1189
+
}
1190
+
1191
+
.selected-avatar, .avatar {
1192
+
width: 28px;
1193
+
height: 28px;
1194
+
border-radius: 50%;
1195
+
background: #24242a;
1196
+
flex: 0 0 auto;
1197
+
object-fit: cover;
1198
+
}
1199
+
1200
+
.selected-avatar {
1201
+
display: none;
1202
+
}
1203
+
1204
+
.person.has-avatar .selected-avatar {
1205
+
display: block;
1206
+
}
1207
+
1208
+
.person.has-avatar {
1209
+
border-color: rgba(42, 23, 16, .72);
1210
+
}
1211
+
1212
+
.person input {
1213
+
min-width: 0;
1214
+
height: 46px;
1215
+
padding: 0;
1216
+
border: 0;
1217
+
border-radius: 0;
1218
+
background: transparent;
1219
+
box-shadow: none;
1220
+
font-family: var(--serif);
1221
+
font-size: 18px;
1222
+
font-weight: 540;
1223
+
line-height: 46px;
1224
+
caret-color: var(--ink);
1225
+
-webkit-appearance: none;
1226
+
appearance: none;
1227
+
}
1228
+
1229
+
.person input:focus {
1230
+
border-color: transparent;
1231
+
box-shadow: none;
1232
+
}
1233
+
1234
+
.menu {
1235
+
position: absolute;
1236
+
z-index: 1000;
1237
+
display: none;
1238
+
left: 0;
1239
+
right: 0;
1240
+
top: calc(100% + 8px);
1241
+
background:
1242
+
linear-gradient(180deg, rgba(255,255,255,.16), transparent),
1243
+
#e0c993;
1244
+
border: 1px solid var(--line-strong);
1245
+
border-radius: 6px;
1246
+
overflow: hidden;
1247
+
box-shadow: 0 18px 48px rgba(0, 0, 0, .44);
1248
+
}
1249
+
1250
+
.menu.open { display: block; }
1251
+
1252
+
.actor {
1253
+
display: flex;
1254
+
gap: 10px;
1255
+
align-items: center;
1256
+
width: 100%;
1257
+
padding: 10px;
1258
+
border: 0;
1259
+
background: #e0c993;
1260
+
color: var(--ink);
1261
+
text-align: left;
1262
+
cursor: pointer;
1263
+
font-family: var(--serif);
1264
+
font-size: 16px;
1265
+
}
1266
+
1267
+
.actor:hover, .actor.active { background: #cfb373; }
1268
+
1269
+
.actor strong, .actor span {
1270
+
display: block;
1271
+
overflow: hidden;
1272
+
text-overflow: ellipsis;
1273
+
white-space: nowrap;
1274
+
}
1275
+
1276
+
.actor span {
1277
+
color: var(--ink-soft);
1278
+
font-size: 13px;
1279
+
}
1280
+
1281
+
button {
1282
+
height: 48px;
1283
+
border: 0;
1284
+
border-radius: 5px;
1285
+
background: var(--ink);
1286
+
color: #f5ead2;
1287
+
font-family: var(--serif);
1288
+
font-size: 18px;
1289
+
font-weight: 700;
1290
+
cursor: pointer;
1291
+
}
1292
+
1293
+
button:hover { background: #3a2116; }
1294
+
button:disabled { opacity: .64; cursor: wait; }
1295
+
1296
+
button.wordmark {
1297
+
height: auto;
1298
+
padding: 0;
1299
+
border: 0;
1300
+
background: transparent;
1301
+
color: #f5ead2;
1302
+
cursor: pointer;
1303
+
}
1304
+
1305
+
button.wordmark:hover,
1306
+
button.wordmark:focus {
1307
+
background: transparent;
1308
+
color: var(--water);
1309
+
outline: none;
1310
+
}
1311
+
1312
+
.help {
1313
+
margin: 13px 2px 0;
1314
+
color: var(--ink-faint);
1315
+
font-size: 13px;
1316
+
line-height: 1.45;
1317
+
}
1318
+
1319
+
.progress {
1320
+
display: none;
1321
+
margin-top: 12px;
1322
+
padding: 12px;
1323
+
border: 1px solid rgba(76, 45, 28, .44);
1324
+
border-radius: 7px;
1325
+
background: rgba(255, 244, 205, .28);
1326
+
}
1327
+
1328
+
.progress.active {
1329
+
display: block;
1330
+
}
1331
+
1332
+
.progress-bar {
1333
+
height: 4px;
1334
+
margin-bottom: 10px;
1335
+
overflow: hidden;
1336
+
border-radius: 999px;
1337
+
background: rgba(42, 23, 16, .18);
1338
+
}
1339
+
1340
+
.progress-fill {
1341
+
width: 0%;
1342
+
height: 100%;
1343
+
border-radius: inherit;
1344
+
background: linear-gradient(90deg, #466f70, var(--water));
1345
+
transition: width .25s ease;
1346
+
}
1347
+
1348
+
.progress-meta {
1349
+
display: flex;
1350
+
align-items: baseline;
1351
+
justify-content: space-between;
1352
+
gap: 12px;
1353
+
color: var(--ink-soft);
1354
+
font-family: var(--mono);
1355
+
font-size: 12px;
1356
+
font-variant-numeric: tabular-nums;
1357
+
}
1358
+
1359
+
.progress-detail {
1360
+
margin-top: 6px;
1361
+
color: var(--ink-faint);
1362
+
font-size: 12px;
1363
+
line-height: 1.45;
1364
+
}
1365
+
1366
+
.results {
1367
+
margin-top: 18px;
1368
+
border: 1px solid rgba(76, 45, 28, .46);
1369
+
border-radius: 7px;
1370
+
background:
1371
+
linear-gradient(180deg, rgba(255,255,255,.14), transparent),
1372
+
rgba(235, 216, 163, .42);
1373
+
font-family: var(--serif);
1374
+
overflow: hidden;
1375
+
}
1376
+
1377
+
.results:empty {
1378
+
display: none;
1379
+
}
1380
+
1381
+
.empty {
1382
+
color: var(--ink-soft);
1383
+
padding: 18px 20px;
1384
+
font-family: var(--serif);
1385
+
font-size: 18px;
1386
+
line-height: 1.5;
1387
+
}
1388
+
1389
+
.result {
1390
+
display: grid;
1391
+
grid-template-columns: 44px minmax(0, 1fr);
1392
+
gap: 14px;
1393
+
padding: 16px 18px;
1394
+
border-top: 1px solid rgba(76, 45, 28, .3);
1395
+
cursor: pointer;
1396
+
transition: background .16s ease, box-shadow .16s ease;
1397
+
}
1398
+
1399
+
.result:first-child {
1400
+
border-top: 0;
1401
+
}
1402
+
1403
+
.result:hover,
1404
+
.result:focus {
1405
+
background: rgba(255, 244, 205, .2);
1406
+
box-shadow: inset 3px 0 0 rgba(31, 95, 95, .52);
1407
+
outline: none;
1408
+
}
1409
+
1410
+
.result-head {
1411
+
display: flex;
1412
+
align-items: baseline;
1413
+
justify-content: space-between;
1414
+
gap: 14px;
1415
+
margin-bottom: 6px;
1416
+
font-family: var(--mono);
1417
+
font-size: 11px;
1418
+
line-height: 1.5;
1419
+
}
1420
+
1421
+
.result-stamp {
1422
+
width: 44px;
1423
+
height: 44px;
1424
+
display: grid;
1425
+
place-items: center;
1426
+
border: 1px solid rgba(76, 45, 28, .42);
1427
+
border-radius: 50%;
1428
+
background:
1429
+
radial-gradient(circle at 38% 32%, rgba(255,255,255,.3), transparent 30%),
1430
+
radial-gradient(circle at 50% 50%, rgba(95,66,41,.16), rgba(42,23,16,.08));
1431
+
color: var(--ink);
1432
+
font-family: var(--serif);
1433
+
font-size: 18px;
1434
+
font-weight: 700;
1435
+
overflow: hidden;
1436
+
box-shadow: inset 0 0 0 3px rgba(255,244,205,.28);
1437
+
}
1438
+
1439
+
a.result-stamp {
1440
+
text-decoration: none;
1441
+
flex: 0 0 auto;
1442
+
}
1443
+
1444
+
.result-stamp img {
1445
+
width: 100%;
1446
+
height: 100%;
1447
+
object-fit: cover;
1448
+
display: block;
1449
+
}
1450
+
1451
+
.result-main {
1452
+
min-width: 0;
1453
+
}
1454
+
1455
+
.result-app {
1456
+
display: flex;
1457
+
align-items: baseline;
1458
+
gap: 7px;
1459
+
min-width: 0;
1460
+
overflow: hidden;
1461
+
color: var(--ink);
1462
+
font-family: var(--serif);
1463
+
font-size: 16px;
1464
+
font-weight: 700;
1465
+
white-space: nowrap;
1466
+
text-overflow: ellipsis;
1467
+
}
1468
+
1469
+
.result-app-link {
1470
+
color: var(--ink);
1471
+
font-family: var(--serif);
1472
+
font-size: 16px;
1473
+
font-weight: 700;
1474
+
text-decoration: none;
1475
+
overflow: hidden;
1476
+
text-overflow: ellipsis;
1477
+
}
1478
+
1479
+
.result-app-link:hover {
1480
+
color: #1f5f5f;
1481
+
}
1482
+
1483
+
.result-type {
1484
+
color: var(--ink-faint);
1485
+
font-family: var(--mono);
1486
+
font-size: 11px;
1487
+
font-weight: 500;
1488
+
min-width: 0;
1489
+
overflow: hidden;
1490
+
text-overflow: ellipsis;
1491
+
}
1492
+
1493
+
.result-meta {
1494
+
color: var(--ink-faint);
1495
+
white-space: nowrap;
1496
+
flex: 0 0 auto;
1497
+
}
1498
+
1499
+
.result-count {
1500
+
color: var(--ink-soft);
1501
+
}
1502
+
1503
+
.result time {
1504
+
display: inline;
1505
+
color: var(--ink-soft);
1506
+
}
1507
+
1508
+
.result p {
1509
+
margin: 0;
1510
+
line-height: 1.52;
1511
+
white-space: pre-wrap;
1512
+
font-size: 16px;
1513
+
}
1514
+
1515
+
.artifact-title {
1516
+
margin: 0 0 4px;
1517
+
color: var(--ink);
1518
+
font-family: var(--serif);
1519
+
font-size: 18px;
1520
+
font-weight: 700;
1521
+
line-height: 1.25;
1522
+
overflow-wrap: anywhere;
1523
+
}
1524
+
1525
+
.result-content-link {
1526
+
display: block;
1527
+
color: inherit;
1528
+
text-decoration: none;
1529
+
}
1530
+
1531
+
.result-content-link:hover .artifact-title,
1532
+
.result-content-link:hover .artifact-body {
1533
+
text-decoration: underline;
1534
+
text-decoration-thickness: 1px;
1535
+
text-underline-offset: 3px;
1536
+
}
1537
+
1538
+
.artifact-body {
1539
+
margin: 0;
1540
+
line-height: 1.52;
1541
+
white-space: pre-wrap;
1542
+
overflow-wrap: anywhere;
1543
+
font-size: 16px;
1544
+
}
1545
+
1546
+
.artifact-link {
1547
+
display: inline-block;
1548
+
margin-top: 8px;
1549
+
color: #1f5f5f;
1550
+
font-family: var(--mono);
1551
+
font-size: 12px;
1552
+
text-decoration: none;
1553
+
overflow-wrap: anywhere;
1554
+
}
1555
+
1556
+
.artifact-link:hover { text-decoration: underline; }
1557
+
1558
+
.result a {
1559
+
color: #1f5f5f;
1560
+
font-family: var(--mono);
1561
+
font-size: 12px;
1562
+
font-weight: 600;
1563
+
text-decoration: none;
1564
+
flex: 0 0 auto;
1565
+
}
1566
+
1567
+
.result a:hover { text-decoration: underline; }
1568
+
1569
+
.result .result-content-link {
1570
+
display: block;
1571
+
color: inherit;
1572
+
font-family: var(--serif);
1573
+
font-size: inherit;
1574
+
font-weight: inherit;
1575
+
text-decoration: none;
1576
+
flex: 1 1 auto;
1577
+
}
1578
+
1579
+
.result .result-content-link .artifact-title {
1580
+
color: var(--ink);
1581
+
font-family: var(--serif);
1582
+
font-size: 18px;
1583
+
font-weight: 700;
1584
+
}
1585
+
1586
+
.result .result-content-link .artifact-body {
1587
+
color: var(--ink);
1588
+
font-family: var(--serif);
1589
+
font-size: 16px;
1590
+
font-weight: 400;
1591
+
}
1592
+
1593
+
.result .result-app-link {
1594
+
color: var(--ink);
1595
+
font-family: var(--serif);
1596
+
font-size: 16px;
1597
+
font-weight: 700;
1598
+
}
1599
+
1600
+
.message {
1601
+
display: block;
1602
+
color: var(--muted);
1603
+
line-height: 1.5;
1604
+
}
1605
+
1606
+
.error {
1607
+
display: block;
1608
+
padding: 18px;
1609
+
color: var(--danger);
1610
+
}
1611
+
1612
+
@media (min-width: 720px) {
1613
+
.steps {
1614
+
grid-template-columns: minmax(180px, .72fr) minmax(0, 1.28fr);
1615
+
}
1616
+
1617
+
.actions {
1618
+
grid-column: 1 / -1;
1619
+
}
1620
+
}
1621
+
1622
+
@media (max-width: 700px) {
1623
+
.page { width: min(100% - 20px, 880px); padding-top: 18px; }
1624
+
header { padding-bottom: 22px; }
1625
+
.note { display: none; }
1626
+
.memory-basin { padding: 18px; border-radius: 22px 22px 16px 16px; }
1627
+
.memory-basin::before { inset: 12px 18px auto; height: 64px; }
1628
+
.memory-basin::after {
1629
+
top: 18px;
1630
+
right: 22px;
1631
+
width: 92px;
1632
+
height: 54px;
1633
+
opacity: .72;
1634
+
}
1635
+
.map-trail {
1636
+
left: 24px;
1637
+
right: 24px;
1638
+
top: 286px;
1639
+
opacity: .48;
1640
+
}
1641
+
.hero { display: block; margin-bottom: 18px; }
1642
+
.basin { display: none; }
1643
+
h1 { font-size: 40px; }
1644
+
.intro { font-size: 15px; }
1645
+
.result-head {
1646
+
display: block;
1647
+
}
1648
+
.result {
1649
+
grid-template-columns: 38px minmax(0, 1fr);
1650
+
gap: 11px;
1651
+
padding: 15px 14px;
1652
+
}
1653
+
.result-stamp {
1654
+
width: 38px;
1655
+
height: 38px;
1656
+
font-size: 16px;
1657
+
}
1658
+
.result a {
1659
+
display: inline-block;
1660
+
margin-top: 4px;
1661
+
}
1662
+
}
1663
+
</style>
1664
+
</head>
1665
+
<body>
1666
+
<div class="page">
1667
+
<header>
1668
+
<button id="reset" class="wordmark" type="button">Pensieve</button>
1669
+
<div class="note">public memory search</div>
1670
+
</header>
1671
+
1672
+
<main>
1673
+
<section class="memory-basin">
1674
+
<div class="sconces" aria-hidden="true"><span class="sconce"></span><span class="sconce"></span></div>
1675
+
<div class="hero">
1676
+
<div>
1677
+
<p class="kicker">public posts and notes</p>
1678
+
<h1>Find the thing you half remember.</h1>
1679
+
<p class="intro">Enter someone’s handle and describe the thing you are trying to find.</p>
1680
+
</div>
1681
+
<div class="basin" aria-hidden="true"></div>
1682
+
</div>
1683
+
<div class="map-trail" aria-hidden="true"></div>
1684
+
1685
+
<div class="search-panel" aria-label="Search memories">
1686
+
<div class="panel-title"><span>Pensieve</span><span>search from a memory</span></div>
1687
+
<form id="search">
1688
+
<div class="steps">
1689
+
<div class="field">
1690
+
<label for="actor-input">whose history?</label>
1691
+
<div class="person-wrap">
1692
+
<div id="actor-field" class="person">
1693
+
<img id="actor-avatar" class="selected-avatar" alt="">
1694
+
<input id="actor-input" placeholder="name or handle" autocomplete="off" autocapitalize="off" autocorrect="off" spellcheck="false">
1695
+
</div>
1696
+
<div id="actor-menu" class="menu" role="listbox"></div>
1697
+
</div>
1698
+
</div>
1699
+
1700
+
<div class="field">
1701
+
<label for="query">what are you looking for?</label>
1702
+
<input id="query" name="q" placeholder="the part you remember..." autocomplete="off" spellcheck="true">
1703
+
</div>
1704
+
1705
+
<div class="actions">
1706
+
<button id="submit" name="action" value="search" type="submit">Search</button>
1707
+
</div>
1708
+
</div>
1709
+
</form>
1710
+
1711
+
<p class="help">Press Enter to search. The first search for someone may take a moment while Pensieve reads what they have shared publicly.</p>
1712
+
<div id="progress" class="progress" aria-live="polite">
1713
+
<div class="progress-bar"><div id="progress-fill" class="progress-fill"></div></div>
1714
+
<div class="progress-meta">
1715
+
<span id="progress-label">starting</span>
1716
+
<span id="progress-count"></span>
1717
+
</div>
1718
+
<div id="progress-detail" class="progress-detail">Looking through public posts and notes. This can take a moment the first time.</div>
1719
+
</div>
1720
+
</div>
1721
+
1722
+
<section id="results" class="results" aria-live="polite">
1723
+
<div class="empty">Start with a name or handle, then write the part you remember.</div>
1724
+
</section>
1725
+
</section>
1726
+
</main>
1727
+
</div>
1728
+
1729
+
<script>
1730
+
let selectedActor = null;
1731
+
let actorOptions = [];
1732
+
let activeActor = -1;
1733
+
let actorTimer = null;
1734
+
let isWorking = false;
1735
+
1736
+
const form = document.querySelector("#search");
1737
+
const reset = document.querySelector("#reset");
1738
+
const query = document.querySelector("#query");
1739
+
const submit = document.querySelector("#submit");
1740
+
const results = document.querySelector("#results");
1741
+
const actorInput = document.querySelector("#actor-input");
1742
+
const actorAvatar = document.querySelector("#actor-avatar");
1743
+
const actorField = document.querySelector("#actor-field");
1744
+
const actorMenu = document.querySelector("#actor-menu");
1745
+
const progress = document.querySelector("#progress");
1746
+
const progressFill = document.querySelector("#progress-fill");
1747
+
const progressLabel = document.querySelector("#progress-label");
1748
+
const progressCount = document.querySelector("#progress-count");
1749
+
const progressDetail = document.querySelector("#progress-detail");
1750
+
1751
+
reset.addEventListener("click", () => {
1752
+
selectedActor = null;
1753
+
actorOptions = [];
1754
+
activeActor = -1;
1755
+
actorInput.value = "";
1756
+
query.value = "";
1757
+
clearSelectedActorImage();
1758
+
actorMenu.classList.remove("open");
1759
+
progress.classList.remove("active");
1760
+
results.innerHTML = '<div class="empty">Start with a name or handle, then write the part you remember.</div>';
1761
+
actorInput.focus();
1762
+
});
1763
+
1764
+
actorInput.addEventListener("input", () => {
1765
+
const normalized = actorInput.value.replace(/^@+/, "");
1766
+
if (actorInput.value !== normalized) actorInput.value = normalized;
1767
+
if (selectedActor?.handle !== normalized.trim()) {
1768
+
selectedActor = null;
1769
+
clearSelectedActorImage();
1770
+
}
1771
+
window.clearTimeout(actorTimer);
1772
+
actorTimer = window.setTimeout(searchActors, 140);
1773
+
});
1774
+
1775
+
actorInput.addEventListener("focus", () => {
1776
+
if (actorOptions.length) actorMenu.classList.add("open");
1777
+
});
1778
+
1779
+
actorInput.addEventListener("keydown", (event) => {
1780
+
if (!actorMenu.classList.contains("open")) return;
1781
+
if (event.key === "ArrowDown") {
1782
+
event.preventDefault();
1783
+
activeActor = Math.min(actorOptions.length - 1, activeActor + 1);
1784
+
renderActorMenu();
1785
+
}
1786
+
if (event.key === "ArrowUp") {
1787
+
event.preventDefault();
1788
+
activeActor = Math.max(0, activeActor - 1);
1789
+
renderActorMenu();
1790
+
}
1791
+
if (event.key === "Enter" && activeActor >= 0) {
1792
+
event.preventDefault();
1793
+
chooseActor(actorOptions[activeActor]);
1794
+
}
1795
+
if (event.key === "Escape") {
1796
+
actorMenu.classList.remove("open");
1797
+
}
1798
+
});
1799
+
1800
+
document.addEventListener("click", (event) => {
1801
+
if (!event.target.closest(".person-wrap")) actorMenu.classList.remove("open");
1802
+
});
1803
+
1804
+
form.addEventListener("submit", async (event) => {
1805
+
event.preventDefault();
1806
+
actorMenu.classList.remove("open");
1807
+
isWorking = true;
1808
+
const q = query.value.trim();
1809
+
if (!q) {
1810
+
results.innerHTML = '<div class="error">Write what you are trying to find.</div>';
1811
+
return;
1812
+
}
1813
+
1814
+
submit.disabled = true;
1815
+
document.querySelectorAll("#search button").forEach((button) => { button.disabled = true; });
1816
+
progress.classList.remove("active");
1817
+
results.innerHTML = '<div class="empty">Searching the public history...</div>';
1818
+
1819
+
try {
1820
+
const actor = selectedActor?.handle || actorInput.value.replace(/^@+/, "").trim();
1821
+
if (!actor) {
1822
+
results.innerHTML = '<div class="error">Enter a name or handle first.</div>';
1823
+
return;
1824
+
}
1825
+
const params = new URLSearchParams({ q, actor });
1826
+
const response = await fetch("/api/search?" + params.toString());
1827
+
const data = await response.json();
1828
+
if (!response.ok) throw new Error(data.message || "Search is unavailable.");
1829
+
if (data.needsIndex) {
1830
+
await updateIndex(q, actor);
1831
+
return;
1832
+
}
1833
+
renderResults(data);
1834
+
} catch (error) {
1835
+
results.innerHTML = '<div class="error">' + escapeHtml(error.message) + '</div>';
1836
+
} finally {
1837
+
isWorking = false;
1838
+
document.querySelectorAll("#search button").forEach((button) => { button.disabled = false; });
1839
+
}
1840
+
});
1841
+
1842
+
async function updateIndex(q, actor) {
1843
+
startProgress();
1844
+
results.innerHTML = "";
1845
+
const params = new URLSearchParams({ q, actor });
1846
+
const response = await fetch("/api/update?" + params.toString());
1847
+
if (!response.ok || !response.body) {
1848
+
const data = await response.json().catch(() => ({}));
1849
+
throw new Error(data.message || "Pensieve could not read that public history.");
1850
+
}
1851
+
1852
+
const reader = response.body.getReader();
1853
+
const decoder = new TextDecoder();
1854
+
let buffer = "";
1855
+
1856
+
while (true) {
1857
+
const { value, done } = await reader.read();
1858
+
if (done) break;
1859
+
buffer += decoder.decode(value, { stream: true });
1860
+
const lines = buffer.split("\\n");
1861
+
buffer = lines.pop() || "";
1862
+
for (const line of lines) {
1863
+
if (!line.trim()) continue;
1864
+
handleUpdateEvent(JSON.parse(line));
1865
+
}
1866
+
}
1867
+
if (buffer.trim()) handleUpdateEvent(JSON.parse(buffer));
1868
+
}
1869
+
1870
+
function handleUpdateEvent(event) {
1871
+
if (event.type === "progress") {
1872
+
updateProgress(event);
1873
+
return;
1874
+
}
1875
+
if (event.type === "done") {
1876
+
updateProgress({ label: "done", pct: 100, indexed: event.data?.indexed?.indexed });
1877
+
renderResults(event.data || {});
1878
+
return;
1879
+
}
1880
+
if (event.type === "error") {
1881
+
throw new Error(event.message || "Pensieve could not read that public history.");
1882
+
}
1883
+
}
1884
+
1885
+
function startProgress() {
1886
+
progress.classList.add("active");
1887
+
progressFill.style.width = "0%";
1888
+
progressLabel.textContent = "starting";
1889
+
progressCount.textContent = "";
1890
+
progressDetail.textContent = "Looking through public posts and notes. This can take a moment the first time.";
1891
+
}
1892
+
1893
+
function updateProgress(event) {
1894
+
const pct = Math.max(0, Math.min(100, Number(event.pct || 0)));
1895
+
progressFill.style.width = pct + "%";
1896
+
progressLabel.textContent = progressLabelFor(event);
1897
+
if (event.total) {
1898
+
progressCount.textContent = Number(event.indexed || 0).toLocaleString() + " / " + Number(event.total).toLocaleString();
1899
+
} else if (event.scanned) {
1900
+
progressCount.textContent = Number(event.scanned).toLocaleString() + " items";
1901
+
} else if (event.indexed) {
1902
+
progressCount.textContent = Number(event.indexed).toLocaleString() + " ready";
1903
+
} else {
1904
+
progressCount.textContent = "";
1905
+
}
1906
+
if (event.phase === "walk") {
1907
+
progressDetail.textContent = "Gathering the public things this person has shared.";
1908
+
} else if (event.phase === "embed") {
1909
+
progressDetail.textContent = "Preparing those items so they can be searched by meaning.";
1910
+
} else if (event.phase === "query") {
1911
+
progressDetail.textContent = "Searching the updated history.";
1912
+
}
1913
+
}
1914
+
1915
+
function progressLabelFor(event) {
1916
+
if (event.phase === "resolve") return "finding that person";
1917
+
if (event.phase === "prepare") return "getting ready";
1918
+
if (event.phase === "fetch") return "opening public history";
1919
+
if (event.phase === "walk") return "looking through public history";
1920
+
if (event.phase === "embed") return "preparing search";
1921
+
if (event.phase === "query") return "searching";
1922
+
return event.label || event.phase || "working";
1923
+
}
1924
+
1925
+
async function searchActors() {
1926
+
const q = actorInput.value.replace(/^@+/, "").trim();
1927
+
if (isWorking || q.length < 2) {
1928
+
actorMenu.classList.remove("open");
1929
+
return;
1930
+
}
1931
+
1932
+
const response = await fetch("/api/actors?q=" + encodeURIComponent(q));
1933
+
const data = await response.json();
1934
+
if (isWorking) {
1935
+
actorMenu.classList.remove("open");
1936
+
return;
1937
+
}
1938
+
actorOptions = data.actors || [];
1939
+
activeActor = actorOptions.length ? 0 : -1;
1940
+
renderActorMenu();
1941
+
}
1942
+
1943
+
function renderActorMenu() {
1944
+
if (!actorOptions.length) {
1945
+
actorMenu.classList.remove("open");
1946
+
actorMenu.innerHTML = "";
1947
+
return;
1948
+
}
1949
+
1950
+
actorMenu.innerHTML = actorOptions.map((actor, index) => {
1951
+
const avatar = actor.avatar
1952
+
? '<img class="avatar" src="' + escapeAttr(actor.avatar) + '" alt="">'
1953
+
: '<span class="avatar"></span>';
1954
+
return '<button type="button" class="actor' + (index === activeActor ? ' active' : '') + '" data-index="' + index + '">' +
1955
+
avatar +
1956
+
'<span><strong>' + escapeHtml(actor.displayName || actor.handle) + '</strong><span>@' + escapeHtml(actor.handle) + '</span></span>' +
1957
+
'</button>';
1958
+
}).join("");
1959
+
1960
+
actorMenu.querySelectorAll("[data-index]").forEach((button) => {
1961
+
button.addEventListener("click", () => chooseActor(actorOptions[Number(button.dataset.index)]));
1962
+
});
1963
+
actorMenu.classList.add("open");
1964
+
}
1965
+
1966
+
function chooseActor(actor) {
1967
+
selectedActor = actor;
1968
+
actorInput.value = actor.handle;
1969
+
results.innerHTML = '<div class="empty">' + escapeHtml(actor.handle) + ' is selected. Write the part you remember.</div>';
1970
+
if (actor.avatar) {
1971
+
actorAvatar.src = actor.avatar;
1972
+
actorField.classList.add("has-avatar");
1973
+
} else {
1974
+
clearSelectedActorImage();
1975
+
}
1976
+
actorMenu.classList.remove("open");
1977
+
}
1978
+
1979
+
function clearSelectedActorImage() {
1980
+
actorAvatar.removeAttribute("src");
1981
+
actorField.classList.remove("has-avatar");
1982
+
}
1983
+
1984
+
function renderResults(data) {
1985
+
const rows = data.results || [];
1986
+
const prefix = renderSearchNote(data);
1987
+
1988
+
if (!rows.length) {
1989
+
results.innerHTML = prefix + '<div class="empty">Nothing matched. Try describing it another way.</div>';
1990
+
return;
1991
+
}
1992
+
1993
+
results.innerHTML = prefix + rows.map((row, index) => {
1994
+
const collection = row.collection || collectionFromUri(row.uri);
1995
+
const artifact = normalizeArtifact(row, collection);
1996
+
const date = row.date ? '<time>' + escapeHtml(row.date) + '</time>' : '<span>undated</span>';
1997
+
const app = appFallback(collection);
1998
+
const rawRecord = row.uri ? recordUrl(row.uri) : "";
1999
+
const primaryUrl = resultPrimaryUrl(row.uri, collection, artifact);
2000
+
const appUrl = app.url || appUrlForCollection(collection);
2001
+
const link = primaryUrl ? '<a class="record-link" href="' + escapeAttr(primaryUrl) + '" target="_blank" rel="noopener">open</a>' : "";
2002
+
const rawLink = rawRecord && rawRecord !== primaryUrl ? '<a class="raw-link" href="' + escapeAttr(rawRecord) + '" target="_blank" rel="noopener">details</a>' : "";
2003
+
const count = row.count > 1 ? '<span class="result-count">' + Number(row.count).toLocaleString() + ' times</span>' : "";
2004
+
const title = artifact.title ? '<h2 class="artifact-title">' + escapeHtml(artifact.title) + '</h2>' : "";
2005
+
const body = artifact.body ? '<p class="artifact-body">' + escapeHtml(artifact.body) + '</p>' : "";
2006
+
const url = artifact.url ? '<a class="artifact-link" href="' + escapeAttr(artifact.url) + '" target="_blank" rel="noopener">' + escapeHtml(readableUrl(artifact.url)) + '</a>' : "";
2007
+
const stamp = appUrl
2008
+
? '<a class="result-stamp app-link" href="' + escapeAttr(appUrl) + '" target="_blank" rel="noopener" aria-label="open ' + escapeAttr(app.name) + '">' + escapeHtml(app.initial) + '</a>'
2009
+
: '<div class="result-stamp" aria-hidden="true">' + escapeHtml(app.initial) + '</div>';
2010
+
const appName = appUrl
2011
+
? '<a class="result-app-name result-app-link app-link" href="' + escapeAttr(appUrl) + '" target="_blank" rel="noopener">' + escapeHtml(app.name) + '</a>'
2012
+
: '<span class="result-app-name">' + escapeHtml(app.name) + '</span>';
2013
+
const mainContent = title || body ? title + body : '<p class="artifact-body">' + escapeHtml(row.body || "Untitled item") + '</p>';
2014
+
const contentLink = primaryUrl ? '<a class="result-content-link" href="' + escapeAttr(primaryUrl) + '" target="_blank" rel="noopener">' + mainContent + '</a>' : mainContent;
2015
+
return '<article class="result" tabindex="0" role="link" data-result="' + index + '" data-collection="' + escapeAttr(collection) + '" data-primary-url="' + escapeAttr(primaryUrl) + '">' +
2016
+
stamp +
2017
+
'<div class="result-main">' +
2018
+
'<div class="result-head">' +
2019
+
'<div class="result-app">' + appName + '<span class="result-type">' + escapeHtml(artifact.kind || recordKind(collection)) + '</span></div>' +
2020
+
'<div class="result-meta">' + date + ' ' + count + ' ' + link + ' ' + rawLink + '</div>' +
2021
+
'</div>' +
2022
+
contentLink + url +
2023
+
'</div>' +
2024
+
'</article>';
2025
+
}).join("");
2026
+
2027
+
bindResultCards();
2028
+
paintAppBadges(rows);
2029
+
}
2030
+
2031
+
function bindResultCards() {
2032
+
results.querySelectorAll(".result[data-primary-url]").forEach((card) => {
2033
+
const open = () => {
2034
+
const url = card.dataset.primaryUrl;
2035
+
if (url) window.open(url, "_blank", "noopener");
2036
+
};
2037
+
card.addEventListener("click", (event) => {
2038
+
if (event.target.closest("a, button")) return;
2039
+
open();
2040
+
});
2041
+
card.addEventListener("keydown", (event) => {
2042
+
if (event.key !== "Enter" && event.key !== " ") return;
2043
+
if (event.target.closest("a, button")) return;
2044
+
event.preventDefault();
2045
+
open();
2046
+
});
2047
+
});
2048
+
}
2049
+
2050
+
function renderSearchNote(data) {
2051
+
if (data.indexed) {
2052
+
const indexed = Number(data.indexed.indexed || 0).toLocaleString();
2053
+
const scanned = Number(data.indexed.scanned || 0).toLocaleString();
2054
+
return '<div class="empty message">Pensieve read ' + scanned + ' public items and kept ' + indexed + ' that can be searched.</div>';
2055
+
}
2056
+
if (data.needsIndex) {
2057
+
return '<div class="empty message">Pensieve has not read this public history yet.</div>';
2058
+
}
2059
+
return "";
2060
+
}
2061
+
2062
+
function recordUrl(uri) {
2063
+
return String(uri).startsWith("at://") ? "https://pds.ls/" + String(uri) : uri;
2064
+
}
2065
+
2066
+
function resultPrimaryUrl(uri, collection, artifact) {
2067
+
if (artifact?.url) return artifact.url;
2068
+
const native = nativeRecordUrl(uri, collection);
2069
+
if (native) return native;
2070
+
return uri ? recordUrl(uri) : "";
2071
+
}
2072
+
2073
+
function nativeRecordUrl(uri, collection) {
2074
+
const parsed = parseAtUri(uri);
2075
+
if (!parsed) return "";
2076
+
if (collection === "app.bsky.feed.post") {
2077
+
return "https://bsky.app/profile/" + encodeURIComponent(parsed.repo) + "/post/" + encodeURIComponent(parsed.rkey);
2078
+
}
2079
+
if (collection.startsWith("sh.tangled.")) {
2080
+
return "https://tangled.org/" + encodeURIComponent(parsed.repo) + "/" + encodeURIComponent(collection) + "/" + encodeURIComponent(parsed.rkey);
2081
+
}
2082
+
return "";
2083
+
}
2084
+
2085
+
function parseAtUri(uri) {
2086
+
const match = String(uri || "").match(/^at:\\/\\/([^/]+)\\/([^/]+)\\/([^/]+)$/);
2087
+
return match ? { repo: match[1], collection: match[2], rkey: match[3] } : null;
2088
+
}
2089
+
2090
+
function collectionFromUri(uri) {
2091
+
const parts = String(uri || "").split("/");
2092
+
return parts.length >= 4 ? parts[3] : "";
2093
+
}
2094
+
2095
+
function recordKind(collection) {
2096
+
const leaf = String(collection || "").split(".").pop() || "item";
2097
+
return humanWords(leaf);
2098
+
}
2099
+
2100
+
function normalizeArtifact(row, collection) {
2101
+
const artifact = row.artifact && typeof row.artifact === "object" ? row.artifact : {};
2102
+
const fallback = row.body || "";
2103
+
return {
2104
+
kind: artifact.kind || recordKind(collection),
2105
+
title: artifact.title || "",
2106
+
body: artifact.body || (!artifact.title ? fallback : ""),
2107
+
url: artifact.url || firstUrl(fallback),
2108
+
media: Array.isArray(artifact.media) ? artifact.media : [],
2109
+
refs: Array.isArray(artifact.refs) ? artifact.refs : [],
2110
+
confidence: artifact.confidence || "low"
2111
+
};
2112
+
}
2113
+
2114
+
function firstUrl(value) {
2115
+
return String(value || "").match(/https?:\\/\\/[^\\s)]+/)?.[0] || "";
2116
+
}
2117
+
2118
+
function readableUrl(value) {
2119
+
try {
2120
+
const url = new URL(value);
2121
+
return url.hostname.replace(/^www\\./, "") + url.pathname.replace(/\\/$/, "");
2122
+
} catch {
2123
+
return value;
2124
+
}
2125
+
}
2126
+
2127
+
const appBadgeCache = new Map();
2128
+
const domainRedirects = { "tangled.sh": "tangled.org" };
2129
+
2130
+
function appFallback(collection) {
2131
+
const domain = collectionDomain(collection);
2132
+
const name = domain ? domain.replace(/^www\\./, "") : "public record";
2133
+
return { name, initial: name.slice(0, 1).toUpperCase() || "P", icon: "", url: domain ? "https://" + domain : "" };
2134
+
}
2135
+
2136
+
function collectionDomain(collection) {
2137
+
const parts = String(collection || "").split(".");
2138
+
if (parts.length < 2) return "";
2139
+
const domain = parts[1] + "." + parts[0];
2140
+
return domainRedirects[domain] || domain;
2141
+
}
2142
+
2143
+
function appUrlForCollection(collection) {
2144
+
const domain = collectionDomain(collection);
2145
+
return domain ? "https://" + domain : "";
2146
+
}
2147
+
2148
+
async function paintAppBadges(rows) {
2149
+
const collections = [...new Set(rows.map((row) => row.collection || collectionFromUri(row.uri)).filter(Boolean))];
2150
+
if (!collections.length) return;
2151
+
await Promise.all(collections.map(resolveAppBadge));
2152
+
document.querySelectorAll(".result[data-collection]").forEach((card) => {
2153
+
const meta = appBadgeCache.get(card.dataset.collection);
2154
+
if (!meta) return;
2155
+
const stamp = card.querySelector(".result-stamp");
2156
+
const name = card.querySelector(".result-app-name");
2157
+
if (name) name.textContent = meta.name;
2158
+
if (name?.tagName === "A" && meta.url) name.href = meta.url;
2159
+
if (stamp?.tagName === "A" && meta.url) stamp.href = meta.url;
2160
+
if (stamp && meta.icon) {
2161
+
stamp.innerHTML = '<img src="' + escapeAttr(meta.icon) + '" alt="">';
2162
+
} else if (stamp) {
2163
+
stamp.textContent = meta.initial;
2164
+
}
2165
+
});
2166
+
}
2167
+
2168
+
async function resolveAppBadge(collection) {
2169
+
if (appBadgeCache.has(collection)) return appBadgeCache.get(collection);
2170
+
const fallback = appFallback(collection);
2171
+
appBadgeCache.set(collection, fallback);
2172
+
2173
+
const domain = collectionDomain(collection);
2174
+
if (!domain) return fallback;
2175
+
2176
+
try {
2177
+
const profileResponse = await fetch("https://public.api.bsky.app/xrpc/app.bsky.actor.getProfile?actor=" + encodeURIComponent(domain), {
2178
+
signal: AbortSignal.timeout(3500)
2179
+
});
2180
+
if (!profileResponse.ok) return fallback;
2181
+
const profile = await profileResponse.json();
2182
+
let meta = {
2183
+
name: profile.displayName || fallback.name,
2184
+
initial: (profile.displayName || fallback.name).slice(0, 1).toUpperCase() || fallback.initial,
2185
+
icon: profile.avatar || "",
2186
+
url: "https://bsky.app/profile/" + encodeURIComponent(profile.handle || domain)
2187
+
};
2188
+
2189
+
const appResponse = await fetch("https://public.api.bsky.app/xrpc/com.atproto.repo.getRecord?repo=" + encodeURIComponent(profile.did) + "&collection=community.lexicon.app.profile&rkey=self", {
2190
+
signal: AbortSignal.timeout(3500)
2191
+
});
2192
+
if (appResponse.ok) {
2193
+
const appRecord = await appResponse.json();
2194
+
meta = mergeAppProfile(meta, appRecord.value, profile.did);
2195
+
}
2196
+
appBadgeCache.set(collection, meta);
2197
+
return meta;
2198
+
} catch (error) {
2199
+
return fallback;
2200
+
}
2201
+
}
2202
+
2203
+
function mergeAppProfile(meta, value, did) {
2204
+
if (!value || typeof value !== "object") return meta;
2205
+
const name = typeof value.name === "string" && value.name.trim() ? value.name.trim() : meta.name;
2206
+
const icon = appProfileIcon(value, did) || meta.icon;
2207
+
return {
2208
+
name,
2209
+
initial: name.slice(0, 1).toUpperCase() || meta.initial,
2210
+
icon,
2211
+
url: meta.url
2212
+
};
2213
+
}
2214
+
2215
+
function appProfileIcon(value, did) {
2216
+
const images = Array.isArray(value.images) ? value.images : [];
2217
+
const preferred = images.find((image) => image?.purpose === "community.lexicon.app.defs#purposeIcon") ||
2218
+
images.find((image) => image?.purpose === "community.lexicon.app.defs#purposeLogo") ||
2219
+
images.find((image) => image?.uri);
2220
+
if (!preferred) return "";
2221
+
if (typeof preferred.uri === "string") return preferred.uri;
2222
+
const cid = preferred.image?.ref?.$link || preferred.image?.ref?.["$link"] || preferred.image?.["$link"];
2223
+
return cid && did ? "https://cdn.bsky.app/img/avatar/plain/" + encodeURIComponent(did) + "/" + encodeURIComponent(cid) : "";
2224
+
}
2225
+
2226
+
function humanWords(value) {
2227
+
return String(value || "item")
2228
+
.replace(/([a-z0-9])([A-Z])/g, "$1 $2")
2229
+
.replace(/[-_]+/g, " ")
2230
+
.toLowerCase();
2231
+
}
2232
+
2233
+
function escapeHtml(value) {
2234
+
return String(value).replace(/[&<>"']/g, (char) => ({
2235
+
"&": "&",
2236
+
"<": "<",
2237
+
">": ">",
2238
+
'"': """,
2239
+
"'": "'"
2240
+
})[char]);
2241
+
}
2242
+
2243
+
function escapeAttr(value) {
2244
+
return escapeHtml(value);
2245
+
}
2246
+
</script>
2247
+
</body>
2248
+
</html>`;
2249
+
}
2250
+
2251
+
function escapeHtml(value) {
2252
+
return String(value).replace(/[&<>"']/g, (char) => ({
2253
+
"&": "&",
2254
+
"<": "<",
2255
+
">": ">",
2256
+
'"': """,
2257
+
"'": "'"
2258
+
})[char]);
2259
+
}
2260
+
2261
+
function escapeAttr(value) {
2262
+
return escapeHtml(value);
2263
+
}
+14
wrangler.jsonc
+14
wrangler.jsonc
···
1
+
{
2
+
"$schema": "node_modules/wrangler/config-schema.json",
3
+
"name": "pensieve",
4
+
"account_id": "3e9ba01cd687b3c4d29033908177072e",
5
+
"main": "src/worker.js",
6
+
"compatibility_date": "2026-06-29",
7
+
"workers_dev": true,
8
+
"observability": {
9
+
"enabled": true
10
+
},
11
+
"vars": {
12
+
"TURBOPUFFER_NAMESPACE_PREFIX": "pensieve"
13
+
}
14
+
}