# Deep Think: Memory Upgrade — Cross-Session Memory Transfer Protocols

**Topic**: `cross_session_transfer`
**Cycle**: 1
**Model**: gemini-2.5-pro
**Generated**: 2026-02-26T10:42:04.421852+00:00
**Elapsed**: 64.2s
**Thinking tokens**: 16384

---

## Generated Prompt (Phase 1 — Gemini Flash)

You are a world-class AI memory systems architect with deep expertise in vector database design, cognitive science, and production AI system optimization. Your task is to analyze AIVA's current memory system's cross-session capabilities and propose a highly specific, actionable strategy for improving memory transfer, consolidation, and contextualization across distinct user sessions. AIVA is a long-running personal AI assistant, meaning its memory profile will evolve significantly over time, encompassing diverse conversation types, memory age distributions, and importance score profiles.

**Current System State:**
The LanceDB plugin in `enhanced_index.ts` inherently persists `MemoryEntry` data (`id`, `text`, `vector`, `importance`, `category`, `createdAt`) to disk at `dbPath` (defaulting to `~/.openclaw/memory/lancedb` via `resolveDefaultDbPath()` in `enhanced_config.ts`). However, there are no explicit mechanisms or protocols for managing memory at session boundaries. The `MemoryDB` class is initialized when the plugin registers, and its `ensureInitialized()` method handles connecting to or creating the LanceDB table `TABLE_NAME = "memories"`. Memory operations (`store`, `update`, `delete`, `hybridSearch`) are performed directly on this persistent store. The `api.on("agent_end", ...)` hook triggers `autoCapture` for user messages, and `api.on("before_agent_start", ...)` triggers `autoRecall`. The `memory_consolidate` tool and background consolidation logic are designed for general maintenance, not specifically for session-end events. There is no concept of a "session ID" or similar identifier attached to memories.

**Problem Statement & Optimization Opportunities:**

1.  **Lack of Session Boundary Detection & Consolidation Triggering**: The current `consolidateMemories` function is a manual tool (`memory_consolidate`) or could be triggered periodically. This is inefficient for a long-running agent where memory churn and staleness might be highly correlated with active usage periods. We need a robust mechanism to detect explicit session end (e.g., user disconnect, inactivity timeout) and implicitly trigger a lightweight consolidation focused on recent session memories, or prioritize specific types of memories. The current `consolidateMemories` involves `db.listAllWithVectors()` which can be expensive.
    *   **Goal**: Reduce memory footprint and improve retrieval latency by ensuring memory freshness and reducing redundancy at the start of new sessions, targeting a 15% reduction in `hybridSearch` latency for cold starts (first 5 queries) after a consolidation.
    *   **Weakness**: No automatic, targeted session-end consolidation.

2.  **Absence of Session-Context Injection**: When a new session starts, `autoRecall` injects relevant memories based on the `event.prompt` of the *first* user message. This is reactive and can miss crucial context that defines the ongoing conversation, leading to suboptimal initial responses. There's no mechanism to load a "session summary" or "key takeaways" from the previous session proactively.
    *   **Goal**: Improve initial response quality and contextual relevance by providing a proactive session context, aiming for a 10% increase in `RECALL_INJECT_MAX` memories being genuinely relevant to the *overall* new session topic (not just the first query).
    *   **Weakness**: Reactive, query-based recall for new sessions.

3.  **No Protocol for "New Topic" vs. "Same Thread" Detection**: The current system treats all interactions as part of a continuous stream, making it hard to discern if a user's prompt signifies a wholly new conversation topic (requiring fresh recall) or a continuation of a prior, potentially paused, discussion. This can lead to irrelevant memories being recalled or important past context being missed.
    *   **Goal**: Enhance the relevance of `autoRecall` by differentiating between new topics and continuing threads, aiming to decrease the injection of "off-topic" memories (MMR similarity to current prompt < `RECALL_MIN_SIMILARITY`) by 20%.
    *   **Weakness**: Undifferentiated memory retrieval for varied session intents.

4.  **Static Memory Importance & Recency Decay**: The `IMPORTANCE_DECAY_PER_DAY` and `HALF_LIFE_DAYS` are global constants. There's no dynamic adjustment based on how frequently a memory is recalled or explicitly referenced across sessions. Memories that are consistently relevant but not recent might decay too quickly, while truly "dead" memories persist longer than necessary.
    *   **Goal**: Implement a feedback loop for importance and recency, aiming to boost the `finalScore` of frequently recalled memories by 5-10% without manual intervention, and accelerate decay for never-recalled memories.
    *   **Weakness**: Fixed decay rates; no usage-based importance boosting.

**Concrete Proposals:**

1.  **Session Boundary Detection & Triggered Consolidation:**
    *   Propose a new `SessionManager` class (or extend `MemoryDB`) that tracks session start/end times.
    *   Define a `SESSION_INACTIVITY_THRESHOLD_MS` constant.
    *   Implement an `api.on("session_end", ...)` hook (or equivalent, triggered by `agent_end` after `SESSION_INACTIVITY_THRESHOLD_MS` or explicit user action).
    *   Within this hook, trigger a *lightweight, targeted* `consolidateMemories` operation:
        *   Only process memories created/updated within the last `SESSION_CONSOLIDATION_WINDOW_MS` (e.g., 24 hours).
        *   Prioritize merging `DUPLICATE_THRESHOLD` memories over pruning `STALENESS_DAYS` ones for recent memories.
    *   **Pseudocode Sketch for `SessionManager` and `session_end` hook:**
        ```typescript
        // enhanced_config.ts
        export const SESSION_INACTIVITY_THRESHOLD_MS = 30 * 60 * 1000; // 30 minutes
        export const SESSION_CONSOLIDATION_WINDOW_MS = 24 * 60 * 60 * 1000; // 24 hours

        // enhanced_index.ts
        class SessionManager {
            private lastActivityTimestamp: number;
            private currentSessionId: string | null;

            constructor() {
                this.lastActivityTimestamp = Date.now();
                this.currentSessionId = randomUUID(); // New session on startup
            }

            public resetSession(): void {
                // Trigger lightweight consolidation for previous session
                this.triggerConsolidationForRecentMemories(this.currentSessionId);
                this.currentSessionId = randomUUID();
                this.lastActivityTimestamp = Date.now();
                api.logger.info(`memory-lancedb: New session started: ${this.currentSessionId}`);
            }

            public recordActivity(): void {
                this.lastActivityTimestamp = Date.now();
            }

            private async triggerConsolidationForRecentMemories(sessionId: string): Promise<void> {
                // Modified consolidateMemories to target recent memories
                // Filter allMemories by createdAt within SESSION_CONSOLIDATION_WINDOW_MS
                // Prioritize merges (DUPLICATE_THRESHOLD) over pruning for this window
                // ... implementation details ...
            }
        }
        // ... in register(api) ...
        const sessionManager = new SessionManager();
        api.on("agent_start", () => sessionManager.recordActivity());
        api.on("agent_end", async (event) => {
            sessionManager.recordActivity();
            // Check for inactivity to reset session
            if (Date.now() - sessionManager.lastActivityTimestamp > SESSION_INACTIVITY_THRESHOLD_MS) {
                sessionManager.resetSession();
            }
            // Existing auto-capture logic...
        });
        ```
    *   **Evaluation**: Measure `hybridSearch` P90 latency for the first 5 queries after a `session_end` event followed by a `session_start` event, comparing with current baseline.

2.  **Proactive Session Context Injection:**
    *   Introduce a new `SESSION_CONTEXT_MEMORY_CATEGORY = "session_context"` (or similar).
    *   At `session_end`, generate a concise summary of the session's most important memories (e.g., top 3-5 `importance`-weighted memories from the session's lifetime). Store this summary as a new `MemoryEntry` with `category: SESSION_CONTEXT_MEMORY_CATEGORY` and a reference to the `sessionId`.
    *   At `before_agent_start`, in addition to `autoRecall`, search for `SESSION_CONTEXT_MEMORY_CATEGORY` entries from the *previous* session (if `sessionId` is available or inferred) and prepend them to the context.
    *   **TypeScript Sketch for storing session context:**
        ```typescript
        // enhanced_config.ts
        export type MemoryCategory = (typeof MEMORY_CATEGORIES)[number] | "session_context";
        export const MEMORY_CATEGORIES = [ /* existing */ "session_context" ] as const;

        // enhanced_index.ts (within SessionManager.triggerConsolidationForRecentMemories or new method)
        async function captureSessionSummary(sessionId: string): Promise<void> {
            // Retrieve top memories from the just-ended session (e.g

---

## Thinking Process (Phase 2 — Gemini Pro)

**My Proposed Evolution of AIVA's Cross-Session Memory Architecture**

Alright, let's break this down. As a world-class AI memory systems architect, I see a fantastic opportunity to evolve AIVA's memory, not replace it. We're building a *cognitive* system, not just a database. The goal is a memory that's dynamic, session-aware, and deeply relevant to the user's ongoing interaction. This means integrating concepts from cognitive science—like episodic vs. semantic memory and Hebbian learning—into a practical, high-performance architecture. We will improve upon the existing foundation (LanceDB, `enhanced_index.ts`, hooks, `autoCapture`, `autoRecall`, `memory_consolidate`), addressing the identified weaknesses with targeted, actionable enhancements, with each proposal optimized for specific metrics.

**Proposal 1: Proactive Session Management & Targeted Consolidation**

*   **Architectural Philosophy:** The current reactive approach to memory management lacks session boundaries, leading to information overload and inefficient retrieval. We need proactive session management, mirroring how humans naturally segment their memories into episodes.
*   **Detailed Implementation Strategy:**
    *   **Component Changes:** Introduce a `SessionManager` class to track the current session, manage session boundaries (e.g., inactivity, explicit commands like `/new_session`), and handle session-related metadata. A new `sessionId` field will be added to the `MemoryEntry` schema.
    *   **Process Flow:**
        1.  At `agent_start`: The `SessionManager` is initialized, or retrieved from persistent storage.
        2.  At `agent_end`: The `SessionManager` updates the current session's end time.
        3.  The `SessionManager` monitors for inactivity periods (configurable via `SESSION_INACTIVITY_TIMEOUT_MS`). Upon exceeding this timeout, or upon explicit user commands, a new session starts.
        4.  During the nightly `memory_consolidate` process, consolidate memories within *each* session: Instead of `listAllWithVectors()`, perform a more efficient query such as `db.search().where("createdAt > ?").limit(...)`, where the timestamp acts as a cutoff. This cutoff can be configurable, such as `MEMORY_CONSOLIDATION_RETENTION_DAYS`.
    *   **Code Sketch:**

```typescript
// enhanced_config.ts
const SESSION_INACTIVITY_TIMEOUT_MS = 60 * 60 * 1000; // 1 hour
const MEMORY_CONSOLIDATION_RETENTION_DAYS = 30; // 30 days of memory kept

// SessionManager.ts
class SessionManager {
    currentSessionId: string;
    lastActiveAt: number;

    constructor() {
        this.currentSessionId = this.generateSessionId();
        this.lastActiveAt = Date.now();
    }

    startNewSession() {
        this.currentSessionId = this.generateSessionId();
        this.lastActiveAt = Date.now();
    }

    updateActivity() {
        this.lastActiveAt = Date.now();
    }

    isInactive(): boolean {
        return Date.now() - this.lastActiveAt > SESSION_INACTIVITY_TIMEOUT_MS;
    }

    generateSessionId(): string {
       return uuidv4();
    }

    getCurrentSessionId(): string {
        return this.currentSessionId;
    }
}

// enhanced_index.ts (in `autoCapture` and `autoRecall`)
// Add the sessionId to the MemoryEntry
const memoryEntry = new MemoryEntry({
    // ... other fields
    sessionId: sessionManager.getCurrentSessionId(), // Get the session ID from the manager
});

// enhanced_index.ts (in `memory_consolidate`)
// Consolidate memories within each session.
async function memory_consolidate() {
    const sessionManager = new SessionManager();
    const cutoffDate = new Date(Date.now() - MEMORY_CONSOLIDATION_RETENTION_DAYS * 24 * 60 * 60 * 1000); // 30 days ago
    const db = await LanceDB.connect(LANCEDB_PATH);

    const oldSessionId = sessionManager.getCurrentSessionId()
    const result = await db.search(query_vector)
            .where(`"createdAt" < "${cutoffDate.toISOString()}"`)
            .limit(1000000)

    if (result.length > 0) {
      await db.delete(`"id" IN (${result.map((r) => r.id).join(",")})`)
      console.

---

## Deep Think Response

Excellent. As a world-class AI memory systems architect, I will provide a detailed, multi-faceted strategy to evolve AIVA's memory system from a simple persistent store into a dynamic, cognitively-inspired architecture that excels at cross-session memory management.

This proposal is structured around four pillars, directly addressing the identified weaknesses and aiming to meet the specified performance goals. The core philosophy is to transition from a static, archival memory to a **dynamic, working memory system** that actively manages context, relevance, and importance based on user interaction patterns.

---

### **Executive Summary of the Proposed Architecture**

We will introduce a `SessionManager` to explicitly model user interaction sessions. This enables four key improvements:

1.  **Proactive Session Management & Targeted Consolidation:** We will detect session boundaries based on inactivity and trigger lightweight, targeted consolidation on recent memories, improving cold-start latency.
2.  **Generative Session Summarization & Context Priming:** At the end of a session, we will use an LLM to generate a concise summary, which is then used to "prime" the context of the next session, dramatically improving initial response relevance.
3.  **Semantic Coherence Analysis for Topic Detection:** Before recalling memories, we will analyze the semantic similarity between the user's new prompt and the previous conversation turn to differentiate between a "topic shift" and a "thread continuation," allowing for more precise memory retrieval.
4.  **Usage-Based Dynamic Importance Scaling:** We will modify the memory schema and scoring algorithm to incorporate a feedback loop, where memories that are frequently recalled have their importance boosted and their decay slowed, mimicking Hebbian learning principles ("what fires together, wires together").

This integrated strategy will transform AIVA's memory into a living system that intelligently curates, consolidates, and contextualizes information across time.

---

### **Proposal 1: Proactive Session Management & Targeted Consolidation**

**Architectural Philosophy:** Cognitive science shows that memory consolidation (akin to sleep) is not a brute-force process but a targeted one, strengthening important new memories and pruning weak ones. We will mimic this by triggering a focused consolidation process at the natural boundary of a user's interaction: the end of a session.

**Detailed Implementation Strategy:**

1.  **Data Model & Configuration Changes:**
    *   **Modify `MemoryEntry`:** Add a `sessionId: string` field to each memory entry. This is a foundational change that enables all subsequent proposals.
    *   **New Constants in `enhanced_config.ts`:**
        ```typescript
        // The time of user inactivity that defines a session break.
        export const SESSION_INACTIVITY_THRESHOLD_MS = 30 * 60 * 1000; // 30 minutes
        // The time window for targeted consolidation at session end.
        export const SESSION_CONSOLIDATION_WINDOW_MS = 24 * 60 * 60 * 1000; // Last 24 hours
        ```

2.  **Introduce the `SessionManager` Class:** This class will be instantiated within the plugin's `register` function and will manage the session lifecycle.

    ```typescript
    // enhanced_index.ts
    import { randomUUID } from 'crypto';

    class SessionManager {
        private lastActivityTimestamp: number;
        public currentSessionId: string;
        private inactivityCheckInterval: NodeJS.Timeout | null = null;
        private memoryDB: MemoryDB; // Reference to the MemoryDB instance

        constructor(memoryDB: MemoryDB) {
            this.memoryDB = memoryDB;
            this.lastActivityTimestamp = Date.now();
            this.currentSessionId = randomUUID();
            this.startInactivityMonitor();
            api.logger.info(`[Memory] New session started: ${this.currentSessionId}`);
        }

        public recordActivity(): void {
            this.lastActivityTimestamp = Date.now();
        }

        public endCurrentSession(reason: 'inactivity' | 'manual'): void {
            api.logger.info(`[Memory] Ending session ${this.currentSessionId} due to ${reason}.`);
            // This is the trigger for our new functionality
            this.triggerSessionEndProtocols(this.currentSessionId);

            // Start a new session
            this.currentSessionId = randomUUID();
            this.lastActivityTimestamp = Date.now();
            api.logger.info(`[Memory] New session started: ${this.currentSessionId}`);
        }

        private startInactivityMonitor(): void {
            if (this.inactivityCheckInterval) {
                clearInterval(this.inactivityCheckInterval);
            }
            this.inactivityCheckInterval = setInterval(() => {
                if (Date.now() - this.lastActivityTimestamp > SESSION_INACTIVITY_THRESHOLD_MS) {
                    this.endCurrentSession('inactivity');
                }
            }, 60 * 1000); // Check every minute
        }

        private async triggerSessionEndProtocols(endedSessionId: string): Promise<void> {
            // Placeholder for new session-end logic (Proposals 1 & 2)
            await this.memoryDB.consolidateRecentMemories(endedSessionId);
            // In Proposal 2, we'll add session summarization here.
        }
    }
    ```

3.  **Integrate `SessionManager` and Modify `MemoryDB`:**
    *   In `register(api)`, instantiate `const sessionManager = new SessionManager(memoryDB);`.
    *   Pass `sessionManager.currentSessionId` to `memoryDB.store()` so it can be attached to new `MemoryEntry` objects.
    *   Use API hooks to track activity: `api.on("agent_start", () => sessionManager.recordActivity());` and `api.on("agent_end", () => sessionManager.recordActivity());`.
    *   Create a new, targeted consolidation method in `MemoryDB`. This method avoids the expensive `listAllWithVectors()` for session-end tasks.

    ```typescript
    // In MemoryDB class within enhanced_index.ts
    public async consolidateRecentMemories(sessionId: string): Promise<void> {
        api.logger.info(`[Memory] Starting lightweight consolidation for recent memories...`);
        const consolidationWindowStart = new Date(Date.now() - SESSION_CONSOLIDATION_WINDOW_MS);

        // Fetch only recent memories, this is much cheaper than listAllWithVectors()
        const recentMemories = await this.db.search()
            .where(`createdAt > '${consolidationWindowStart.toISOString()}'`)
            .limit(1000) // Safety limit
            .execute<MemoryEntry>();

        if (recentMemories.length < 2) return;

        // The consolidation logic (merging duplicates) remains the same,
        // but it now operates on a much smaller, more relevant subset of data.
        // We prioritize merging over staleness pruning for this operation.
        const { toMerge, toKeep } = await this.identifyDuplicateMemories(recentMemories);

        if (toMerge.length > 0) {
            await this.mergeMemories(toMerge, toKeep);
            api.logger.info(`[Memory] Lightweight consolidation complete. Merged ${toMerge.length} memories.`);
        }
    }
    ```

**Evaluation & Metrics:**
*   **Goal:** 15% reduction in `hybridSearch` latency for cold starts.
*   **Method:** Establish a benchmark script that simulates a user session (10-20 interactions), followed by a 35-minute pause (to trigger the `SESSION_INACTIVITY_THRESHOLD_MS`), then a new session with 5 queries. Measure and compare the P90 latency of `hybridSearch` for those first 5 queries with and without this new consolidation mechanism enabled. The reduction in redundant recent memories should directly impact search performance.

---

### **Proposal 2: Generative Session Summarization & Context Priming**

**Architectural Philosophy:** When humans resume a conversation, they don't recall every detail; they recall the gist or key takeaways. We will create an "episodic summary" of the last session and use it to prime AIVA's context, ensuring a smooth and contextually-aware continuation.

**Detailed Implementation Strategy:**

1.  **Configuration and Schema:**
    *   **New Category in `enhanced_config.ts`:**
        ```typescript
        export const MEMORY_CATEGORIES = [ /* existing categories */, "session_summary" ] as const;
        export type MemoryCategory = (typeof MEMORY_CATEGORIES)[number];
        ```
    *   **Modify `MemoryEntry`:** Add an optional `metadata: { summarizedSessionId?: string }` field to store which session a summary belongs to.

2.  **Implement Session Summarization Logic:**
    *   This logic will be called from `SessionManager.triggerSessionEndProtocols`.

    ```typescript
    // In MemoryDB class
    public async generateAndStoreSessionSummary(sessionId: string): Promise<void> {
        api.logger.info(`[Memory] Generating summary for session ${sessionId}...`);
        // 1. Retrieve the most important memories from the ended session.
        const sessionMemories = await this.db.search()
            .where(`sessionId = '${sessionId}'`)
            .orderBy('importance DESC')
            .limit(10) // Top 10 most important memories
            .execute<MemoryEntry>();

        if (sessionMemories.length < 3) {
            api.logger.info(`[Memory] Not enough memories in session to generate a summary.`);
            return;
        }

        // 2. Format memories for an LLM prompt.
        const memoriesText = sessionMemories.map((m, i) => `${i+1}. (Importance: ${m.importance.toFixed(2)}) ${m.text}`).join('\n');
        const prompt = `Based on the following key memories from a conversation, provide a concise, one-paragraph summary of the main topics, conclusions, and action items. This summary will be used to remind the AI of the context in the next conversation.

Key Memories:
${memoriesText}

Summary:`;

        // 3. Call the LLM to generate the summary.
        const summaryText = await api.llm.call({ prompt }); // Assuming a simple LLM call interface

        // 4. Store the summary as a new, high-importance memory.
        await this.store({
            text: `Summary of previous session: ${summaryText}`,
            importance: 9.0, // High importance so it's easily recalled
            category: "session_summary",
            metadata: { summarizedSessionId: sessionId }
        });
        api.logger.info(`[Memory] Stored summary for session ${sessionId}.`);
    }
    ```

3.  **Modify `autoRecall` for Context Priming:**
    *   At the start of a new session, proactively inject the last session's summary.

    ```typescript
    // In MemoryDB.autoRecall, at the beginning
    private async autoRecall(event: LlmEvent): Promise<MemoryEntry[]> {
        let memoriesToInject: MemoryEntry[] = [];
        const isNewSessionStart = await this.sessionManager.isFirstInteractionOfSession(); // Add this check to SessionManager

        if (isNewSessionStart) {
            // Find the most recent session summary that isn't from the current session
            const lastSummary = await this.db.search()
                .where(`category = 'session_summary' AND sessionId != '${this.sessionManager.currentSessionId}'`)
                .orderBy('createdAt DESC')
                .limit(1)
                .execute<MemoryEntry>();

            if (lastSummary.length > 0) {
                memoriesToInject.push(...lastSummary);
                api.logger.info(`[Memory] Proactively injecting previous session summary.`);
            }
        }

        // ... existing hybridSearch logic follows ...
        // Ensure the final injected list is deduplicated and respects RECALL_INJECT_MAX
        const regularRecall = await this.hybridSearch(event.prompt, ...);
        memoriesToInject.push(...regularRecall);
        // Deduplicate and slice to RECALL_INJECT_MAX
        // ...
        return finalInjectedMemories;
    }
    ```

**Evaluation & Metrics:**
*   **Goal:** 10% increase in genuinely relevant memories for the *overall* new session.
*   **Method:** Use an "LLM-as-judge" evaluation framework. For a set of test conversations that span two sessions, compare the memories injected in the second session's first turn. The judge LLM will rate the relevance of each injected memory to the *full transcript* of the second session. We aim for a higher average relevance score for the system with session summaries enabled.

---

### **Proposal 3: Semantic Coherence Analysis for Topic Detection**

**Architectural Philosophy:** A user's intent can vary. A new prompt might be a continuation or a complete context switch. Our memory system should adapt its retrieval strategy accordingly, narrowing its focus for continuations and broadening it for new topics.

**Detailed Implementation Strategy:**

1.  **Configuration:**
    *   **New Constants in `enhanced_config.ts`:**
        ```typescript
        // Cosine similarity threshold to determine if a new prompt is a topic shift.
        export const TOPIC_SHIFT_THRESHOLD = 0.65;
        // Number of recent turns to compare against.
        export const TOPIC_COHERENCE_WINDOW = 3;
        ```

2.  **Implement `analyzeTopicCoherence` method:** This method will be called within `autoRecall` *before* the main `hybridSearch`.

    ```typescript
    // In MemoryDB class
    private async analyzeTopicCoherence(promptVector: number[]): Promise<'continuation' | 'shift'> {
        // Get the last N memories from the current session
        const recentMemories = await this.db.search()
            .where(`sessionId = '${this.sessionManager.currentSessionId}'`)
            .orderBy('createdAt DESC')
            .limit(TOPIC_COHERENCE_WINDOW)
            .execute<MemoryEntry>();

        if (recentMemories.length === 0) {
            return 'shift'; // No recent history, must be a new topic
        }

        // Calculate cosine similarity between the new prompt and recent memories
        const similarities = await Promise.all(
            recentMemories.map(mem => this.db.vectorSimilarity(promptVector, mem.vector))
        );
        const maxSimilarity = Math.max(...similarities);

        api.logger.debug(`[Memory] Topic coherence check, max similarity: ${maxSimilarity.toFixed(3)}`);

        return maxSimilarity > TOPIC_SHIFT_THRESHOLD ? 'continuation' : 'shift';
    }
    ```

3.  **Adapt `hybridSearch` based on Coherence:**
    *   Modify `hybridSearch` to accept an optional parameter indicating the topic context.

    ```typescript
    // In MemoryDB class, modify hybridSearch signature
    public async hybridSearch(
        query: string,
        limit: number = RECALL_INJECT_MAX,
        minSimilarity: number = RECALL_MIN_SIMILARITY,
        topicContext: 'continuation' | 'shift' = 'shift'
    ): Promise<MemoryEntry[]> {
        // ... (embedding logic) ...
        const queryVector = await this.embed(query);

        const search = this.db.search(queryVector, {
            k: limit,
            metric: 'cosine',
            nprobes: 20 // Adjust for performance/accuracy
        });

        // The core logic change:
        if (topicContext === 'continuation') {
            // For continuations, heavily prioritize memories from the CURRENT session.
            // LanceDB doesn't have direct boosting, so we can emulate it with a filtered search
            // or by post-processing scores. A simple pre-filter is effective.
            search.where(`sessionId = '${this.sessionManager.currentSessionId}'`);
        } else {
            // For topic shifts, perform a broad, global search.
            // We might even explicitly DOWN-WEIGHT recent memories if they are irrelevant.
            // For now, a global search is sufficient.
        }

        search.where(`_distance > ${minSimilarity}`);
        const vectorResults = await search.execute<MemoryEntry>();

        // ... (MMR and final score calculation remains the same) ...
    }

    // In autoRecall, update the call
    const topicContext = await this.analyzeTopicCoherence(await this.embed(event.prompt));
    const recalledMemories = await this.hybridSearch(event.prompt, RECALL_INJECT_MAX, RECALL_MIN_SIMILARITY, topicContext);
    ```

**Evaluation & Metrics:**
*   **Goal:** Decrease injection of "off-topic" memories by 20%.
*   **Method:** Create a dataset of conversation snippets, some being clear continuations and others clear topic shifts. Run `autoRecall` on the target prompt for each snippet. Measure the average cosine similarity of the injected memories to the target prompt's vector. For "shift" cases, we expect this new logic to yield a higher average similarity (i.e., more relevant recalls) compared to the baseline, which might inject context from the previous, now irrelevant, topic.

---

### **Proposal 4: Usage-Based Dynamic Importance Scaling**

**Architectural Philosophy:** A memory's importance is not static. A memory that is frequently recalled is, by definition, more important. We will implement a feedback loop that strengthens memories upon recall and accelerates the decay of unused memories, creating a more adaptive and self-organizing system.

**Detailed Implementation Strategy:**

1.  **Data Model Changes:**
    *   **Modify `MemoryEntry`:** Add two new fields.
        *   `recallCount: number` (default: 0)
        *   `lastRecalledAt: string` (ISO date, optional)

2.  **Update `MemoryDB` operations:**
    *   **Feedback on Recall:** When memories are injected into the context (i.e., returned by `autoRecall`), we must update their metadata. This requires a new method.

    ```typescript
    // In MemoryDB class
    public async recordMemoryRecall(memoryIds: string[]): Promise<void> {
        if (memoryIds.length === 0) return;
        const now = new Date().toISOString();
        // This is a critical operation. A production-grade LanceDB setup might
        // support partial updates. For now, we assume a read-modify-write cycle,
        // which should be batched for efficiency.
        for (const id of memoryIds) {
            // This is a simplification. A real implementation would need an efficient
            // way to "update" a record, potentially using LanceDB's upcoming features
            // or a more complex update-by-delete-and-add transaction.
            const oldEntry = await this.db.get(id);
            if (oldEntry) {
                const newEntry = {
                    ...oldEntry,
                    recallCount: (oldEntry.recallCount || 0) + 1,
                    lastRecalledAt: now
                };
                await this.db.update(id, newEntry);
            }
        }
    }

    // In enhanced_index.ts, after a successful llm.call that used memories
    api.on("agent_end", async (event) => {
        // Assuming `event.context.recalledMemories` contains the list of memories used
        if (event.context.recalledMemories) {
            const memoryIds = event.context.recalledMemories.map(m => m.id);
            await memoryDB.recordMemoryRecall(memoryIds);
        }
        // ... existing autoCapture logic ...
    });
    ```

3.  **Modify the Scoring Algorithm in `hybridSearch`:**
    *   The `finalScore` calculation should now factor in `recallCount` and `lastRecalledAt`.

    ```typescript
    // Inside the final scoring loop of hybridSearch
    // ... after calculating recencyScore and vectorScore ...

    const recallBoost = Math.log1p(entry.recallCount || 0) * 0.1; // Logarithmic boost to prevent runaway scores
    
    // Adjust recency: if recalled recently, it's as if it was created recently
    const effectiveCreatedAt = entry.lastRecalledAt ?
        Math.max(new Date(entry.createdAt).getTime(), new Date(entry.lastRecalledAt).getTime())
        : new Date(entry.createdAt).getTime();
    
    // Recalculate recencyScore with the effective date
    const ageInDays = (now - effectiveCreatedAt) / (1000 * 60 * 60 * 24);
    const recencyScore = Math.pow(0.5, ageInDays / HALF_LIFE_DAYS);

    // Combine scores
    const finalScore = (entry.importance / 10) * IMPORTANCE_WEIGHT +
                     recencyScore * RECENCY_WEIGHT +
                     vectorScore * SIMILARITY_WEIGHT +
                     recallBoost; // Add the recall boost
    ```

4.  **Enhance Consolidation Pruning:**
    *   In the *full* `consolidateMemories` tool (not the lightweight session-end one), use the new fields to prune more intelligently.

    ```typescript
    // In the full consolidateMemories
    // ... inside the loop to identify memories to prune ...
    const ageInDays = (Date.now() - new Date(mem.createdAt).getTime()) / (1000 * 60 * 60 * 24);
    const hasBeenRecalled = (mem.recallCount || 0) > 0;

    // Prune if it's old AND has never been recalled OR has very low importance.
    if (ageInDays > STALENESS_DAYS && !hasBeenRecalled && mem.importance < 3.0) {
        memoriesToPrune.push(mem);
    }
    ```

**Evaluation & Metrics:**
*   **Goal:** Boost `finalScore` of frequently recalled memories by 5-10% and accelerate decay for unused memories.
*   **Method:** Log the `finalScore` components (`importance`, `recencyScore`, `recallBoost`) for injected memories over a multi-week period. We should observe that for a recurring topic, the `recallBoost` and `recencyScore` (due to `lastRecalledAt`) for relevant memories consistently keep them at the top of the retrieval list, even as their `createdAt` date gets older. We can also track the average age of pruned memories, which should decrease as we more effectively identify and remove "dead" memories early.
