# Storage

This document describes the layout of a **journal** directory where all captures, extracts, and insights are stored. Each dated `YYYYMMDD` folder is referred to as a **day**, and within each day captured content is organized into **segments** (timestamped duration folders). Each segment folder uses the format `HHMMSS_LEN/` where `HHMMSS` is the start time and `LEN` is the duration in seconds. This folder name serves as the **segment key**, uniquely identifying the segment within a given day.

## Top-Level Directory Structure

| Directory/File | Purpose |
|----------------|---------|
| `chronicle/` | Container for daily capture folders (`YYYYMMDD/`) containing segments, extracts, and agent outputs |
| `entities/` | Journal-level entity identity records (`<id>/entity.json`) |
| `facets/` | Facet-specific data: activities, entity relationships, historical events, news, action logs |
| `talents/` | Talent run logs in per-talent subdirectories (`<name>/<id>.jsonl`), day indexes (`<day>.jsonl`), and latest-run symlinks (`<name>.log`) |
| `solstone/apps/` | App-specific storage (distinct from codebase `solstone/apps/`) |
| `streams/` | Per-stream state files (`<name>.json`) tracking segment chains and sequence numbers |
| `imports/` | Imported audio files and processing artifacts |
| `tokens/` | Token usage logs from AI model calls, organized by day |
| `indexer/` | Search index (`journal.sqlite` FTS5 database) |
| `health/` | Service health logs (`<service>.log` files) |
| `config/` | Configuration files and journal-level action logs |
| `task_log.txt` | Optional log of utility runs in `[epoch]\tmessage` format |
| `summary.md` | Journal-wide statistics summary (generated by `journal journal-stats`) |
| `stats.json` | Detailed journal statistics in JSON format (generated by `journal journal-stats`) |

### Config directory

- `config/journal.json` – owner configuration for the journal (optional, see [configuration](config.md)).
- `config/convey.json` – Convey UI preferences (facet/app ordering, selected facet).
- `config/actions/` – journal-level action logs (see [action logs](logs.md#action-logs)).

## App Storage

The `solstone/apps/` directory provides storage space for Convey apps to persist configuration, data, and artifacts specific to this journal. Each app has its own directory at `solstone/apps/<app_name>/` where it can maintain app-specific state independent of the application codebase.

Apps typically use `config.json` for journal-specific settings and create subdirectories for data storage (e.g., `cache/`, `data/`, `logs/`). This is distinct from the app metadata file (`solstone/apps/<app>/app.json` in the codebase) which defines icon, label, and facet support across all journals. See [APPS.md](../../../docs/APPS.md) for storage utilities (`get_app_storage_path`, `load_app_config`, `save_app_config`).

## Search Index

The `indexer/` directory contains the full-text search index built from journal content.

**Files:**
- `indexer/journal.sqlite` – FTS5 SQLite database containing indexed chunks from agent outputs, activities, historical events, entities, and action logs

The indexer converts content to markdown chunks via the formatters framework, then indexes with metadata fields (day, facet, agent) for filtering. Raw audio/screen transcripts are formattable but not indexed — agent outputs provide more useful search results. Use `get_journal_index()` from `solstone/think/indexer/journal.py` to access the database programmatically.

Which content gets indexed is controlled by the `FORMATTERS` registry in `solstone/think/formatters.py`. Each entry maps a glob pattern to a formatter function and an `indexed` flag. The registry patterns must be specific enough to use as `Path.glob()` arguments from the journal root — adding a new content location requires a new entry.

Run `journal indexer` to rebuild the index from current journal content.
