# Memory System

The memory layer in Agenvoy has three tiers for conversation memory plus a cross-session error memory tier.

| Tier | Backed by | Scope |
|---|---|---|
| 1. Context window (16 messages + summary) | `history.json` + `summary.json` | session |
| 2. Semantic search (recent) | ToriiDB `DBSessionHist` (vector) | session |
| 3. Full-text archive (all history) | SQLite FTS5 via go-sqlkit | session |
| Error memory | ToriiDB `error_memory` (90d TTL) | cross-session |

Operator notes (SQLite `note` + `note_fts5`, the `find_note` tool, `/v1/note*`) were removed in v1.0.23; there is no global note tier any more.

## Three-tier conversation memory

### 1. Context window (`limits.max_history_messages`, default 16)

Each session keeps the most recent N messages in full and feeds them into the LLM context window. Anything older remains in `history.json` but is not sent to the LLM.

A rolling summary (`summary.json`) condenses older conversations so older context survives beyond the N-message window. Since v1.3.7 it rides inside the turn context — the one `user` message placed right after the recent history and ahead of the user input — as a `Prior summary` JSON block; before v1.3.7 it was its own `user` message between the history and the turn context, carrying a long fixed instruction block (Removed in v1.3.7). With a decision model set, each turn keeps only the `key_decisions`, `past_discussions` and `current_discussion` entries relevant to the request (a general rule about language, format or workflow counts as relevant); with no decision model, or when the check fails, every entry is kept, and when none is relevant the summary is left out. How to use it — follow `key_decisions`, treat the rest as background, answer "what did we discuss" from it in prose — lives in the system prompt's `Prior Conversation Context` section. Tool-history compaction no longer drops the summary.

**Incremental cursor:** `.summary_cursor` (per-session; the older `summary.meta.json` was migrated on startup up to v1.1.1 and is ignored since v1.1.2) holds the time of the last summarized message (format `YYYY-MM-DD HH:MM:SS`, taken from the record's `SendAt`). A system cron (`*/15 * * * *`) picks every non-`temp-` session whose `history.json` is newer than its `summary.json` (mtime gate) and runs `Generate` on it:

1. A session that already has a summary but no cursor only gets its cursor set to the latest message time — no LLM call
2. Since v1.3.1, when a decision model is set, the messages newer than the cursor (or without a timestamp) are grouped into exchanges — a user message and the replies after it — and the model drops those it judges noise with a confidence of at least 0.7 (greetings, thanks, retries of the same request, failed or canceled turns with no outcome), then removes `key_decisions` and `past_discussions` entries of the previous summary that those exchanges reverse or supersede. A failed check keeps everything; when no exchange is left, the pruned summary and the cursor are saved with no LLM call
3. Messages newer than the cursor (or without a timestamp) that are still left are split into chunks of up to `SUMMARY_RUNES` (32,000) characters; a chunk never ends on a user message
4. Each chunk runs **one** LLM call whose system prompt already contains the previous summary (`{{.Summary}}`), so merging happens during generation — no separate merge pass
5. After each successful chunk, `summary.json` and the cursor are saved; a failed or unparseable chunk stops the run so later chunks are not billed, and the next cron tick retries

Since v1.3.1 the summary prompt records only standing rules in `key_decisions` (at most 8), each starting with `Do: ` (agreed) or `Don't: ` (rejected) in English whatever the output language — since v1.3.7; the language-dependent markers such as `要：` / `不要：` and `する：` / `しない：` were Removed in v1.3.7, and an existing entry carrying one is rewritten at the next merge — and the text of every field is written in the configured `reply_lang`; with `auto` the summary follows the conversation's language, Chinese as Traditional Chinese.

### 2. Semantic search — ToriiDB (recent conversations)

The `chat_history` tool with `mode=search` and `match=semantic` runs vector similarity search via ToriiDB `db.VSearch`. Each hit triggers a context window expansion: 2 entries before + 1 entry after.

ToriiDB entries are cleaned during `history.json` compaction — entries older than the compact cutoff are removed, keeping ToriiDB focused on recent conversations. Older data lives in SQLite (tier 3).

### 3. Full-text archive — SQLite FTS5 (all history)

Every message written to `history.json` is **dual-written** to SQLite (`~/.config/agenvoy/.store/history.db`) via go-sqlkit. SQLite always holds the complete conversation history, even after `history.json` is compacted.

The `chat_history` tool with `mode=search` and `match=keyword` runs FTS5 full-text search on the SQLite archive + ToriiDB substring match on recent entries, combining results.

**Compaction:** when `history.json` exceeds `max_history_bytes` (default 4 MiB since v1.0.12, when it became four times the 1 MiB document ceiling), the oldest messages are trimmed to 80% on a complete user+assistant pair boundary. The cutoff timestamp is recorded in SQLite `message_meta.start_at` so that keyword search excludes entries already present in `history.json` (avoiding duplicates). ToriiDB entries older than the cutoff are also removed.

**Backfill:** on first encounter (SQLite has no data for a session but `history.json` has content), the entire existing history is backfilled into SQLite.

**Timestamps:** stored as UTC unix nanoseconds. Timestamps in message content are parsed via `time.ParseInLocation` (local timezone) and converted to UTC for storage. Search queries use `time.Now().UnixNano()` (already UTC).

## Storage layout

| Store | Content | Lifecycle |
|---|---|---|
| `history.json` | Recent messages (hot, LLM reads every turn) | Auto-compacted at 4 MiB |
| ToriiDB `DBSessionHist` | Recent messages with embeddings | Cleaned on compact (entries < cutoff removed) |
| SQLite `messages` | All messages ever written (dual-write) | Cleared on reset / remove-session |
| SQLite `message_meta` | `start_at` — compact cutoff timestamp | Cleared on reset / remove-session |
| `summary.json` | Rolling summary blob | Survives reset |
| ToriiDB `error_memory` | Resolved tool error records with their fix | 90d TTL (refresh on hit) |

## Migration note

Sessions and error memory used to live under per-session JSON files. They now live in the embedded stores under `~/.config/agenvoy/.store/` — ToriiDB (`db_0` tool cache, `db_1` chat vectors, `db_2` error memory, `db_3` online markers) and `history.db` for session config, the message archive, action history, file history and usage. Notes moved from ToriiDB into the SQLite `note` table in v0.35.3 and were removed altogether in v1.0.23, including the notes migration at daemon start; the `state` table was dropped in v0.35.3. `bot.json` / `bot.md` / `config.json` / `status.json` / `usage.log` / `summary.meta.json` were migrated in at daemon startup up to v1.1.1; v1.1.2 removed those migrations along with the SQLite column upgrades for databases older than v1.0, so an install that old should pass through v1.1.1 first. Do not reintroduce JSON paths.