## MODULE 5: CQRS + Event Sourcing + Postgres Cold Ledger
**Q5/Q6 Implementation — Immutable Event Ledger**
**Stories**: 9 | **Estimate**: 5-6 days | **Wave**: 2 (after Module 2 Redis L1 schema)

**Module Vision**: The Postgres Cold Ledger (L4) is the immutable source of truth for all Genesis state mutations. Every saga, every state transition, every swarm execution is recorded as an append-only event. This enables Shadow Replay evaluation, Nightly Epoch aggregation, and full system reconstruction from scratch. The CQRS pattern separates reads (any model reads the snapshot) from writes (only the committed merge patch writes to DB).

**Module File Map**:
```
core/storage/postgres_schema.py             ← Story 5.01
core/storage/cold_ledger.py                 ← Story 5.02
core/storage/event_sourcing.py              ← Story 5.03
core/storage/saga_writer.py                 ← Story 5.04
core/storage/session_store.py               ← Story 5.05
core/storage/cold_ledger_interceptor.py     ← Story 5.06
core/storage/shadow_router.py               ← Story 5.07
tests/storage/test_cold_ledger.py           ← Story 5.08
tests/storage/test_event_sourcing.py        ← Story 5.09
```

---

### Story 5.01: Postgres Schema — sessions, events, swarm_sagas
**File**: `core/storage/postgres_schema.py`

- **As** the Postgres Cold Ledger, **I need** the canonical Postgres schema for sessions, events, and swarm_sagas tables, **so that** all agents write to the same immutable ledger structure.

**Acceptance Criteria**:
- [ ] `SESSIONS_DDL` string: CREATE TABLE sessions with `id UUID PK`, `started_at TIMESTAMP`, `ended_at TIMESTAMP`, `agent_id VARCHAR(100)`, `metadata JSONB`
- [ ] `EVENTS_DDL` string: CREATE TABLE events with `id UUID PK`, `session_id UUID FK→sessions`, `event_type VARCHAR(50)`, `payload JSONB`, `created_at TIMESTAMP DEFAULT NOW()`
- [ ] `SWARM_SAGAS_DDL` string: matches architecture spec exactly (`saga_id UUID PK`, `session_id UUID FK`, `orchestrator_dag JSONB`, `proposed_deltas JSONB`, `resolved_state JSONB`, `status VARCHAR(20)`, `created_at TIMESTAMP DEFAULT NOW()`)
- [ ] `create_all_tables(conn)` — executes all 3 DDL statements in a transaction
- [ ] `drop_all_tables(conn)` — for test teardown only
- [ ] Schema uses `IF NOT EXISTS` (idempotent — safe to run multiple times)
- [ ] All JSONB fields allow NULL (optional content)

**Implementation Skeleton**:
```python
SWARM_SAGAS_DDL = """
CREATE TABLE IF NOT EXISTS swarm_sagas (
    saga_id          UUID PRIMARY KEY,
    session_id       UUID REFERENCES sessions(id),
    orchestrator_dag JSONB,
    proposed_deltas  JSONB,
    resolved_state   JSONB,
    status           VARCHAR(20),
    created_at       TIMESTAMP DEFAULT NOW()
);
"""

def create_all_tables(conn) -> None:
    with conn.cursor() as cur:
        cur.execute(SESSIONS_DDL)
        cur.execute(EVENTS_DDL)
        cur.execute(SWARM_SAGAS_DDL)
    conn.commit()
```

**Test Plan**:
- BB: `create_all_tables` runs without error on clean DB
- BB: `create_all_tables` is idempotent (runs twice, no error)
- BB: `swarm_sagas.session_id` FK constraint enforced (insert with bad session_id → error)
- WB: All DDL strings contain `IF NOT EXISTS`
- WB: `drop_all_tables` drops in reverse FK order (sagas → events → sessions)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 5.02: ColdLedger — L4 Postgres Read/Write Client
**File**: `core/storage/cold_ledger.py`

- **As** the Genesis storage layer, **I need** a typed ColdLedger client for L4 Postgres operations, **so that** all agents use the same read/write interface without raw SQL.

**Acceptance Criteria**:
- [ ] `ColdLedger` class with Elestio PostgreSQL connection (uses `ElestioPostgresConfig`)
- [ ] `write_event(session_id, event_type, payload) -> str` — returns event UUID
- [ ] `get_events(session_id, event_type=None, limit=100) -> list[dict]` — reads event log
- [ ] `write_saga(saga: SwarmSaga) -> str` — returns saga UUID
- [ ] `get_saga(saga_id: str) -> Optional[SwarmSaga]` — returns full saga or None
- [ ] `get_sagas_by_session(session_id: str) -> list[SwarmSaga]` — all sagas for session
- [ ] All writes are append-only (no UPDATE on events table)
- [ ] Connection pooling via `psycopg2.pool.ThreadedConnectionPool(minconn=2, maxconn=10)`
- [ ] **NO SQLite** — uses Elestio Postgres exclusively

**Implementation Skeleton**:
```python
import psycopg2.pool, uuid
from data.genesis_memory.elestio_config import PostgresConfig

class ColdLedger:
    def __init__(self):
        self.pool = psycopg2.pool.ThreadedConnectionPool(
            2, 10, **PostgresConfig.get_connection_params()
        )

    def write_event(self, session_id: str, event_type: str, payload: dict) -> str:
        event_id = str(uuid.uuid4())
        conn = self.pool.getconn()
        try:
            with conn.cursor() as cur:
                cur.execute(
                    "INSERT INTO events (id, session_id, event_type, payload) VALUES (%s, %s, %s, %s)",
                    (event_id, session_id, event_type, json.dumps(payload))
                )
            conn.commit()
        finally:
            self.pool.putconn(conn)
        return event_id
```

**Test Plan**:
- BB: `write_event` then `get_events` → returns matching record
- BB: `get_events(event_type="dispatch_start")` → filters correctly
- BB: Two writes with same session_id → both returned by `get_events`
- WB: Connection pool uses `getconn`/`putconn` pattern (no connection leak)
- WB: No `import sqlite3` in file (NO SQLite rule)

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 5.03: EventSourcingStream — Immutable Event Append + Replay
**File**: `core/storage/event_sourcing.py`

- **As** the Genesis state reconstruction system, **I need** an event sourcing stream that can reconstruct any system state by replaying events, **so that** the system can be rebuilt from scratch from the L4 Cold Ledger.

**Acceptance Criteria**:
- [ ] `EventSourcingStream` class with `append(event: GenesisEvent) -> None` and `replay(session_id: str) -> list[GenesisEvent]`
- [ ] `GenesisEvent` dataclass: `id: str`, `session_id: str`, `event_type: str`, `payload: dict`, `version: int`, `created_at: datetime`
- [ ] `append` writes to ColdLedger (L4), then publishes to Redis Stream `genesis:events` via `XADD`
- [ ] `replay(session_id)` reads all events for session from L4, orders by `version`
- [ ] `replay_from_version(session_id, from_version: int) -> list[GenesisEvent]` — partial replay
- [ ] `get_current_state(session_id) -> dict` — fold all events into current state dict
- [ ] Event version auto-incremented: `SELECT MAX(version)+1 FROM events WHERE session_id=...`

**Test Plan**:
- BB: Append 3 events → replay returns all 3 in version order
- BB: `replay_from_version(v=2)` → returns only events with version >= 2
- BB: `get_current_state` folds events in order
- WB: `append` writes to BOTH L4 Postgres and Redis Stream (dual-write)
- WB: Version auto-increment uses DB-side MAX query (not application-side counter)

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 5.04: SwarmSagaWriter — Saga Lifecycle Recorder
**File**: `core/storage/saga_writer.py`

- **As** the multi-agent coherence system, **I need** a saga writer that records the full lifecycle of every multi-agent execution (DAG, proposed deltas, resolved state), **so that** every saga is auditable and replayable.

**Acceptance Criteria**:
- [ ] `SwarmSagaWriter` class with `open_saga(session_id, orchestrator_dag) -> str` (returns saga_id)
- [ ] `record_proposed_delta(saga_id, agent_id, delta: dict) -> None` — appends to `proposed_deltas` JSONB
- [ ] `close_saga(saga_id, resolved_state: dict, status: str) -> None` — writes final resolved state
- [ ] `SwarmSaga` dataclass matches Postgres schema fields exactly
- [ ] `open_saga` creates record with `status="RUNNING"`, `proposed_deltas=[]`
- [ ] `close_saga` sets `status="COMPLETED"|"PARTIAL_FAIL"|"FAILED"`
- [ ] Each delta stored as: `{"agent_id": ..., "delta": ..., "submitted_at": ...}`
- [ ] Concurrent `record_proposed_delta` calls are safe (uses DB-level array append)

**Test Plan**:
- BB: `open_saga` → status="RUNNING" in DB
- BB: `record_proposed_delta` → delta appears in `proposed_deltas` array
- BB: `close_saga` → status updated to "COMPLETED"
- WB: Multiple concurrent `record_proposed_delta` calls → all deltas persisted (no overwrite)
- WB: `open_saga` returns UUID not None

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 5.05: SessionStore — Session Lifecycle Manager
**File**: `core/storage/session_store.py`

- **As** the Genesis session system, **I need** a session store that creates, tracks, and closes sessions in the L4 Cold Ledger, **so that** every agent execution is scoped to a session with a start/end record.

**Acceptance Criteria**:
- [ ] `SessionStore` class with `open_session(agent_id: str, metadata: dict) -> str` (returns session_id UUID)
- [ ] `close_session(session_id: str) -> None` — sets `ended_at = NOW()`
- [ ] `get_active_sessions() -> list[dict]` — returns sessions where `ended_at IS NULL`
- [ ] `get_session(session_id: str) -> Optional[dict]` — returns full session record
- [ ] Sessions auto-tagged with `started_at = NOW()` on create
- [ ] `open_session` returns UUID4 string (not int)
- [ ] Orphaned sessions (no close after 24h) handled by `cleanup_orphaned_sessions()` — sets `ended_at = started_at + INTERVAL '24 hours'`

**Test Plan**:
- BB: `open_session` → `get_active_sessions()` includes new session
- BB: `close_session` → `get_active_sessions()` no longer includes it
- BB: `get_session(unknown_id)` → returns None
- WB: `cleanup_orphaned_sessions` uses `WHERE ended_at IS NULL AND started_at < NOW() - INTERVAL '24 hours'`
- WB: `open_session` generates UUID4 (not sequential int)

**Budget**: 5 implement + 3 BB + 3 WB + 1 IT + 1 stamp = 13 iterations

---

### Story 5.06: ColdLedgerInterceptor — Wires L4 Into Interceptor Chain
**File**: `core/storage/cold_ledger_interceptor.py`

- **As** the InterceptorChain, **I need** a `ColdLedgerInterceptor` that automatically records every dispatch start/complete/error to the L4 Postgres Cold Ledger, **so that** the immutable event record is maintained without business logic changes.

**Acceptance Criteria**:
- [ ] `ColdLedgerInterceptor` extends `BaseInterceptor`
- [ ] `metadata = InterceptorMetadata(name="cold_ledger", priority=5)` — runs very early (after telemetry at 0, before hydration at 10)
- [ ] `pre_execute`: writes `event_type="dispatch_start"` to ColdLedger
- [ ] `post_execute`: writes `event_type="dispatch_complete"` with result summary
- [ ] `on_error`: writes `event_type="dispatch_error"` with error class + message
- [ ] `on_correction`: writes `event_type="dispatch_correction"` with attempt number
- [ ] All writes are fire-and-forget async (don't block the chain)
- [ ] On ColdLedger unavailable → interceptor logs warning and continues (never blocks execution)

**Test Plan**:
- BB: After dispatch, `cold_ledger.get_events(session_id)` has `dispatch_start` + `dispatch_complete`
- BB: On error, events has `dispatch_error` entry
- BB: ColdLedger unavailable → chain continues normally (no exception propagation)
- WB: Priority=5 places it after telemetry (0) and before hydration (10)
- WB: Writes are async — don't add measurable latency to chain execution

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 5.07: ShadowRouter — Side-Effect Gating
**File**: `core/storage/shadow_router.py`

- **As** the Genesis evaluation and testing system, **I need** a Shadow Router that intercepts all external side-effects and routes them to LIVE or SHADOW mode, **so that** testing and Nightly Epoch sandbox runs never affect real customers.

**Acceptance Criteria**:
- [ ] `ShadowRouter` class with `route_side_effect(effect_type: str, payload: dict) -> ShadowResult`
- [ ] `ShadowResult` dataclass: `executed: bool`, `mode: str` ("LIVE"|"SHADOW"), `log_entry: dict`
- [ ] **LIVE mode** (default): executes the real external call, returns result
- [ ] **SHADOW mode**: logs the intended call to `data/observability/shadow_log.jsonl`, returns simulated success without executing
- [ ] Mode controlled by `SHADOW_MODE` env variable (default: `"LIVE"`)
- [ ] Per-effect-type override: `SHADOW_OVERRIDES = {"email": "SHADOW", "sms": "SHADOW"}` from env JSON
- [ ] Effect types: `"email"`, `"sms"`, `"crm_write"`, `"telnyx_call"`, `"ghl_webhook"`, `"external_api"`
- [ ] Shadow log entry includes: `effect_type`, `payload_hash`, `timestamp`, `mode`

**Test Plan**:
- BB: `SHADOW_MODE="SHADOW"` → all effects logged, none executed
- BB: `SHADOW_MODE="LIVE"` with `SHADOW_OVERRIDES={"email":"SHADOW"}` → email shadowed, others live
- BB: Shadow log file updated after each shadow execution
- WB: `ShadowResult.executed=False` in SHADOW mode, `True` in LIVE mode
- WB: `payload_hash` is sha256 of payload (not raw payload — privacy)

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 5.08 + 5.09: Test Suites — Module 5 Storage
**Files**: `tests/storage/test_cold_ledger.py` + `tests/storage/test_event_sourcing.py`

**Acceptance Criteria (5.08 — BB)**:
- [ ] Schema idempotency, event write/read, saga open/close/delta
- [ ] Session lifecycle: open → active → close → not-active
- [ ] ShadowRouter LIVE vs SHADOW modes
- [ ] 14+ BB test cases, uses test Postgres (not production)

**Acceptance Criteria (5.09 — WB)**:
- [ ] Event sourcing version ordering verified
- [ ] `get_current_state` fold correctness
- [ ] ColdLedger connection pool usage verified
- [ ] No SQLite usage in any storage module
- [ ] 10+ WB test cases

**Budget (each)**: 6 implement + 1 run + 1 fix + 1 re-run + 1 stamp = 10 iterations each

---


Successfully appended to E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md (270 lines) ✅ File written successfully! (270 lines)

💡 Performance tip: For optimal speed, consider chunking files into ≤30 line pieces in future operations.


## MODULE 6: Multi-Agent Coherence — Redis Streams + OCC
**Q6 Implementation — CQRS Execution Layer**
**Stories**: 9 | **Estimate**: 6-7 days | **Wave**: 3 (after Modules 4 + 5)

**Module Vision**: When 1000 Gemini agents run in parallel, they NEVER write directly to shared state. Each agent proposes a `StateDelta` (RFC 6902 JSON Patch). All deltas land in a Redis staging hash. Opus reviews all deltas as the Semantic Reducer and commits a single unified patch using Optimistic Concurrency Control. Redis Consumer Groups guarantee exactly-one task delivery. Cascade failures are isolated via bulkheads. The system scales to 1000 agents without state corruption.

**Module File Map**:
```
core/coherence/state_delta.py               ← Story 6.01
core/coherence/redis_master_state.py        ← Story 6.02
core/coherence/task_dag_pusher.py           ← Story 6.03
core/coherence/swarm_worker_base.py         ← Story 6.04
core/coherence/staging_area.py              ← Story 6.05
core/coherence/occ_commit.py                ← Story 6.06
core/coherence/bulkhead.py                  ← Story 6.07
core/coherence/coherence_orchestrator.py    ← Story 6.08
tests/coherence/test_coherence_pipeline.py  ← Story 6.09
```

---

### Story 6.01: StateDelta — RFC 6902 JSON Patch Proposal
**File**: `core/coherence/state_delta.py`

- **As** any Gemini swarm worker, **I need** a `StateDelta` class that represents my proposed state change as an RFC 6902 JSON Patch, **so that** I can propose changes without directly mutating shared state.

**Acceptance Criteria**:
- [ ] `StateDelta` dataclass: `agent_id: str`, `session_id: str`, `version_at_read: int`, `patch: list[dict]`, `submitted_at: datetime`
- [ ] `patch` is a list of RFC 6902 operations: `[{"op": "replace", "path": "/field", "value": ...}]`
- [ ] Valid ops: `"add"`, `"remove"`, `"replace"`, `"move"`, `"copy"`, `"test"`
- [ ] `validate_patch(patch: list) -> bool` — validates RFC 6902 structure
- [ ] `apply_patch(state: dict, patch: list) -> dict` — applies patch to state dict, returns new state
- [ ] `apply_patch` raises `PatchConflictError` if `op: "test"` assertion fails
- [ ] `PatchConflictError` custom exception with `path` and `expected` attributes
- [ ] Patches are immutable once created (dataclass with `frozen=True`)

**Implementation Skeleton**:
```python
from dataclasses import dataclass, field
from typing import Optional
import jsonpatch  # RFC 6902 library

class PatchConflictError(Exception):
    def __init__(self, path: str, expected, actual):
        self.path = path
        self.expected = expected
        self.actual = actual
        super().__init__(f"Patch conflict at {path}: expected {expected}, got {actual}")

@dataclass(frozen=True)
class StateDelta:
    agent_id: str
    session_id: str
    version_at_read: int
    patch: tuple  # tuple for hashability (frozen dataclass)
    submitted_at: datetime

    def apply_to(self, state: dict) -> dict:
        return jsonpatch.apply_patch(state, list(self.patch))
```

**Test Plan**:
- BB: Valid patch applied to state → correct new state
- BB: `op: "test"` with wrong value → `PatchConflictError` raised
- BB: `frozen=True` → attempt to mutate `agent_id` raises `FrozenInstanceError`
- WB: `validate_patch` rejects unknown op types
- WB: `apply_patch` returns new dict (does not mutate original)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 6.02: RedisMasterState — Versioned OCC State Store
**File**: `core/coherence/redis_master_state.py`

- **As** the CQRS coherence system, **I need** a versioned Redis master state that enforces Optimistic Concurrency Control on all writes, **so that** concurrent agents cannot corrupt shared state.

**Acceptance Criteria**:
- [ ] `RedisMasterState` class with key `genesis:state:master:{session_id}`
- [ ] State structure: `{"version": 42, "data": {...}}`
- [ ] `get_snapshot(session_id) -> tuple[int, dict]` — returns `(version, data)` READ-ONLY
- [ ] `commit_patch(session_id, version: int, patch: list) -> CommitResult` — OCC conditional write
- [ ] `CommitResult` dataclass: `success: bool`, `new_version: int`, `conflict: bool`
- [ ] OCC logic: `GET state → check version == expected → apply patch → SET state with new version`
- [ ] Uses Redis `WATCH` + `MULTI`/`EXEC` transaction for atomicity
- [ ] On version conflict (concurrent write changed it): returns `CommitResult(success=False, conflict=True)`
- [ ] `new_version = version + 1` on success

**Implementation Skeleton**:
```python
class RedisMasterState:
    KEY_PREFIX = "genesis:state:master:"

    async def get_snapshot(self, session_id: str) -> tuple[int, dict]:
        raw = await self.redis.get(f"{self.KEY_PREFIX}{session_id}")
        if not raw:
            return (0, {})
        state = json.loads(raw)
        return (state["version"], state["data"])

    async def commit_patch(self, session_id: str, version: int, patch: list) -> CommitResult:
        key = f"{self.KEY_PREFIX}{session_id}"
        async with self.redis.pipeline(transaction=True) as pipe:
            try:
                await pipe.watch(key)
                current = json.loads(await pipe.get(key) or '{"version":0,"data":{}}')
                if current["version"] != version:
                    return CommitResult(False, current["version"], conflict=True)
                new_data = jsonpatch.apply_patch(current["data"], patch)
                pipe.multi()
                await pipe.set(key, json.dumps({"version": version+1, "data": new_data}))
                await pipe.execute()
                return CommitResult(True, version+1, conflict=False)
            except redis.WatchError:
                return CommitResult(False, -1, conflict=True)
```

**Test Plan**:
- BB: Fresh session → `get_snapshot` returns `(0, {})`
- BB: `commit_patch` with correct version → `CommitResult(success=True, new_version=1)`
- BB: Two concurrent commits with same version → one succeeds, one gets `conflict=True`
- WB: Uses Redis `WATCH`/`MULTI`/`EXEC` (not optimistic lock via Lua script)
- WB: `new_version = version + 1` exactly

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 6.03: TaskDAGPusher — Redis Streams Task Queue
**File**: `core/coherence/task_dag_pusher.py`

- **As** the Opus DAG planner, **I need** a DAG pusher that adds tasks to the Redis Stream `genesis:swarm:tasks`, **so that** Gemini workers can claim tasks via Consumer Groups with exactly-once delivery.

**Acceptance Criteria**:
- [ ] `TaskDAGPusher` class with `push_dag(session_id, tasks: list[dict]) -> list[str]` — returns list of stream entry IDs
- [ ] Uses Redis `XADD genesis:swarm:tasks` for each task
- [ ] Each stream entry: `{"session_id": ..., "task_id": ..., "task_type": ..., "payload": json_str, "tier": ..., "priority": ...}`
- [ ] `create_consumer_group(group_name: str = "genesis_workers") -> bool` — creates `XGROUP CREATE` if not exists
- [ ] `get_stream_length() -> int` — `XLEN genesis:swarm:tasks`
- [ ] Stream key: `genesis:swarm:tasks` (matches architecture spec)
- [ ] Tasks pushed in DAG dependency order (topological sort not required — just push in provided order)

**Test Plan**:
- BB: `push_dag([task1, task2])` → `XLEN` = 2
- BB: Stream entry contains all required fields
- BB: `create_consumer_group` succeeds on empty stream and on existing group (idempotent)
- WB: `XADD` called once per task (not batch)
- WB: Returns list of stream IDs (not task UUIDs)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 6.04: SwarmWorkerBase — XREADGROUP Consumer
**File**: `core/coherence/swarm_worker_base.py`

- **As** any Gemini swarm worker, **I need** a base worker class that claims tasks from the Redis Stream via `XREADGROUP` and sends `XACK` after completion, **so that** exactly-one delivery is guaranteed and crashed workers don't lose tasks.

**Acceptance Criteria**:
- [ ] `SwarmWorkerBase` ABC with `process(task: dict) -> StateDelta` abstract method
- [ ] `run_worker_loop(group: str, consumer_id: str) -> None` — claims and processes tasks
- [ ] Uses `XREADGROUP GROUP genesis_workers {consumer_id} COUNT 1 BLOCK 5000 STREAMS genesis:swarm:tasks >`
- [ ] After `process()` succeeds: sends `XACK genesis:swarm:tasks genesis_workers {entry_id}`
- [ ] After `process()` fails: does NOT send XACK (task re-queues after PEL 60s timeout)
- [ ] `PEL_TIMEOUT_MS = 60000` — 60 second visibility timeout
- [ ] Worker claims pending entries on startup: `XAUTOCLAIM` for entries older than `PEL_TIMEOUT_MS`
- [ ] `stop()` — sets `_running = False`, exits loop gracefully

**Implementation Skeleton**:
```python
class SwarmWorkerBase(ABC):
    PEL_TIMEOUT_MS = 60000

    @abstractmethod
    async def process(self, task: dict) -> StateDelta:
        ...

    async def run_worker_loop(self, group: str, consumer_id: str) -> None:
        self._running = True
        while self._running:
            entries = await self.redis.xreadgroup(
                group, consumer_id, {"genesis:swarm:tasks": ">"},
                count=1, block=5000
            )
            if entries:
                stream, messages = entries[0]
                for entry_id, fields in messages:
                    task = {k.decode(): v.decode() for k, v in fields.items()}
                    try:
                        delta = await self.process(task)
                        await self.staging.submit_delta(delta)
                        await self.redis.xack("genesis:swarm:tasks", group, entry_id)
                    except Exception:
                        pass  # No XACK → re-queues after PEL timeout
```

**Test Plan**:
- BB: Task processed successfully → `XACK` sent → task not redelivered
- BB: Task throws exception → no `XACK` → task visible again after 60s
- BB: `stop()` exits loop cleanly
- WB: `XREADGROUP` uses `>` (new messages only, not pending)
- WB: `XAUTOCLAIM` called on startup for orphaned pending entries

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 6.05: StagingArea — ProposedDelta Collection Hub
**File**: `core/coherence/staging_area.py`

- **As** the CQRS coherence system, **I need** a staging area where all worker `StateDelta` proposals are collected before Opus reviews them, **so that** no agent writes directly to the master state.

**Acceptance Criteria**:
- [ ] `StagingArea` class with `submit_delta(delta: StateDelta) -> None` and `collect_all(session_id) -> list[StateDelta]`
- [ ] Deltas stored in Redis Hash: `genesis:staging:{session_id}` → `{agent_id: json(delta)}`
- [ ] `HSET genesis:staging:{session_id} {agent_id} {json(delta)}`
- [ ] `collect_all(session_id)` → `HGETALL genesis:staging:{session_id}` → deserialize all deltas
- [ ] `clear(session_id)` → `DEL genesis:staging:{session_id}` — cleanup after commit
- [ ] `wait_for_all(session_id, expected_count: int, timeout_ms: int = 60000) -> list[StateDelta]`
  - Polls `HLEN` until count matches or timeout
  - Returns whatever is available at timeout (partial result is valid)
- [ ] Staging hash TTL: 600s (auto-cleanup if no one collects)

**Test Plan**:
- BB: `submit_delta` then `collect_all` → returns matching delta
- BB: Multiple agents submit → all appear in `collect_all`
- BB: `clear` → `collect_all` returns empty list
- WB: `wait_for_all` returns partial results on timeout (not error)
- WB: Staging hash has 600s TTL applied on creation

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 6.06: OCCCommitEngine — Barrier Sync + OCC Write
**File**: `core/coherence/occ_commit.py`

- **As** the Opus merge commit engine, **I need** an OCC commit engine that waits for all agents to submit deltas, passes them to the SemanticMergeInterceptor, and commits the resolved state atomically, **so that** the master state is always a coherent, conflict-free merge.

**Acceptance Criteria**:
- [ ] `OCCCommitEngine` class with `execute_commit(session_id, expected_workers: int) -> CommitResult`
- [ ] **Step 1**: `StagingArea.wait_for_all(session_id, expected_workers, timeout_ms=60000)` — collect all deltas
- [ ] **Step 2**: Call `SemanticMergeInterceptor.resolve(deltas) -> merged_patch` — Opus resolves contradictions
- [ ] **Step 3**: `RedisMasterState.commit_patch(session_id, version, merged_patch)` — OCC conditional write
- [ ] **Step 4**: On version conflict → rehydrate and retry (max 3 retries)
- [ ] **Step 5**: `SwarmSagaWriter.close_saga(saga_id, resolved_state, status)` — record outcome
- [ ] Returns final `CommitResult` from `RedisMasterState`
- [ ] Emits `event_type="saga_committed"` to ColdLedger on success

**Test Plan**:
- BB: 3 workers submit → all collected → merged → committed
- BB: Version conflict → retry succeeds on attempt 2
- BB: 3 retries all fail → returns `CommitResult(success=False)`
- WB: `SemanticMergeInterceptor.resolve` called exactly once per commit attempt
- WB: `close_saga` called regardless of success/failure (audit trail always written)

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 6.07: BulkheadGuard — asyncio.gather Exception Isolation
**File**: `core/coherence/bulkhead.py`

- **As** the swarm execution layer, **I need** a bulkhead guard that runs multiple agent tasks concurrently while isolating failures, **so that** one crashed agent doesn't bring down the entire swarm.

**Acceptance Criteria**:
- [ ] `BulkheadGuard` class with `run_with_bulkhead(tasks: list[Coroutine]) -> list[BulkheadResult]`
- [ ] `BulkheadResult` dataclass: `agent_id: str`, `success: bool`, `result: Optional[dict]`, `error: Optional[str]`
- [ ] Uses `asyncio.gather(*tasks, return_exceptions=True)` — never raises
- [ ] Maps exceptions to `BulkheadResult(success=False, error=str(exception))`
- [ ] Maps successes to `BulkheadResult(success=True, result=...)`
- [ ] `get_success_rate(results) -> float` — `sum(r.success for r in results) / len(results)`
- [ ] On `success_rate < CRITICAL_THRESHOLD` (default 0.5): emits `event_type="swarm_critical_failure"` to ColdLedger
- [ ] Never raises exceptions — always returns list of results

**Test Plan**:
- BB: 4 tasks (3 success, 1 exception) → 3 `success=True`, 1 `success=False`
- BB: All tasks fail → `get_success_rate()` returns 0.0
- BB: `run_with_bulkhead` never raises (exceptions captured in results)
- WB: Uses `asyncio.gather(return_exceptions=True)` (not try/except per task)
- WB: `success_rate < 0.5` triggers ColdLedger event write

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 6.08: CoherenceOrchestrator — 8-Step Flow Coordinator
**File**: `core/coherence/coherence_orchestrator.py`

- **As** the Genesis multi-agent system, **I need** a `CoherenceOrchestrator` that runs the complete 8-step coherence execution flow (MAP → CLAIM → PROPOSE → BARRIER → REDUCE → COMMIT → RELEASE → SCAR), **so that** all multi-agent executions follow the same coherent pipeline.

**Acceptance Criteria**:
- [ ] `CoherenceOrchestrator` class with `execute(session_id, tasks: list[dict]) -> CoherenceResult`
- [ ] `CoherenceResult` dataclass: `success: bool`, `committed_state: dict`, `saga_id: str`, `workers_succeeded: int`, `workers_failed: int`
- [ ] Step 1 (MAP): call `TaskDAGPusher.push_dag(session_id, tasks)`
- [ ] Step 2 (CLAIM): workers auto-claim via `XREADGROUP` (async, running in background)
- [ ] Step 3 (PROPOSE): workers run, submit `StateDelta` to `StagingArea`
- [ ] Step 4 (BARRIER): `StagingArea.wait_for_all(expected_count=len(tasks), timeout_ms=60000)`
- [ ] Step 5 (REDUCE): `OCCCommitEngine.execute_commit()` — Opus merge + OCC write
- [ ] Step 6 (COMMIT): OCC conditional write (inside OCCCommitEngine)
- [ ] Step 7 (RELEASE): `ShadowRouter` executes merged API side-effects
- [ ] Step 8 (SCAR): On any worker failure → `Qdrant.upsert(genesis_scars, failure_vector)`
- [ ] Total orchestration timeout: 120 seconds

**Test Plan**:
- BB: 3-task DAG → all succeed → `CoherenceResult(success=True)`
- BB: 1 of 3 workers fail → `workers_failed=1`, partial commit attempted
- BB: Scar written to Qdrant on any failure
- WB: 8 steps fire in correct order
- WB: Total timeout enforced at 120s

**Budget**: 8 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 17 iterations

---

### Story 6.09: Test Suite — Module 6 Multi-Agent Coherence
**File**: `tests/coherence/test_coherence_pipeline.py`

**Acceptance Criteria**:
- [ ] BB: Full 8-step pipeline with 3 workers (all mock)
- [ ] BB: Version conflict + retry path
- [ ] BB: Worker crash → PEL re-queue verified
- [ ] WB: OCC `WATCH/MULTI/EXEC` used (not optimistic Lua)
- [ ] WB: `asyncio.gather(return_exceptions=True)` in bulkhead verified
- [ ] 16+ total test cases

**Budget**: 7 implement + 1 run + 1 fix + 1 re-run + 1 stamp = 11 iterations

---


Successfully appended to E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md (331 lines) ✅ File written successfully! (331 lines)

💡 Performance tip: For optimal speed, consider chunking files into ≤30 line pieces in future operations.


## MODULE 7: SemanticMergeInterceptor + Bulkheads
**Q6 Implementation — Opus as Semantic Reducer**
**Stories**: 7 | **Estimate**: 4-5 days | **Wave**: 3 (parallel with Module 6)

**Module Vision**: The SemanticMergeInterceptor is where Opus earns its keep. Gemini agents propose `StateDelta` patches — Opus decides which ones are right, which are contradictory, and how to merge them into a single coherent state. This is not "summarization" — this is logical contradiction resolution: the hardest cognitive task in multi-agent systems.

**Module File Map**:
```
core/merge/conflict_detector.py             ← Story 7.01
core/merge/semantic_merge_interceptor.py    ← Story 7.02
core/merge/merge_prompt_builder.py          ← Story 7.03
core/merge/patch_reconciler.py              ← Story 7.04
core/merge/merge_telemetry.py               ← Story 7.05
core/merge/compensating_transaction.py      ← Story 7.06
tests/merge/test_semantic_merge.py          ← Story 7.07
```

---

### Story 7.01: ConflictDetector — Patch Contradiction Analysis
**File**: `core/merge/conflict_detector.py`

- **As** the SemanticMergeInterceptor, **I need** a conflict detector that analyzes multiple `StateDelta` patches and identifies logical contradictions, **so that** Opus only gets invoked when there's a genuine conflict (saving expensive API calls for the 20% case).

**Acceptance Criteria**:
- [ ] `ConflictDetector` class with `detect(deltas: list[StateDelta]) -> ConflictReport`
- [ ] `ConflictReport` dataclass: `has_conflicts: bool`, `conflicting_paths: list[str]`, `conflict_types: list[str]`, `non_conflicting_deltas: list[StateDelta]`
- [ ] **Conflict type 1 — Path collision**: two deltas write to the same JSON path with different values
- [ ] **Conflict type 2 — Op contradiction**: one delta does `"add"` at path X, another does `"remove"` at path X
- [ ] **Conflict type 3 — Semantic contradiction**: two deltas set `"status"` to different values
- [ ] If no conflicts: returns `has_conflicts=False` and `non_conflicting_deltas=all_deltas` (merge without Opus)
- [ ] `fast_merge(non_conflicting_deltas: list[StateDelta]) -> list[dict]` — dict merge of non-conflicting patches, no Opus call

**Test Plan**:
- BB: Two deltas with same path + different values → `has_conflicts=True`
- BB: Two deltas touching different paths → `has_conflicts=False, fast_merge returns both`
- BB: Empty deltas list → `has_conflicts=False, non_conflicting_deltas=[]`
- WB: `conflicting_paths` contains the actual path string (e.g., "/status")
- WB: `fast_merge` returns list of all patches merged (no deduplication)

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 7.02: SemanticMergeInterceptor — Opus Contradiction Resolver
**File**: `core/merge/semantic_merge_interceptor.py`

- **As** the CQRS commit engine, **I need** a `SemanticMergeInterceptor` that passes conflicting `StateDelta` proposals to Opus and gets back a single unified JSON patch, **so that** multi-agent execution always produces coherent, conflict-free state.

**Acceptance Criteria**:
- [ ] `SemanticMergeInterceptor` class with `resolve(deltas: list[StateDelta], current_state: dict) -> list[dict]`
- [ ] Returns merged JSON patch (list of RFC 6902 operations)
- [ ] Step 1: `ConflictDetector.detect(deltas)` — fast path if no conflicts
- [ ] Step 2 (only if conflicts): calls Opus 4.6 with `MergePromptBuilder.build(deltas, conflicts, current_state)`
- [ ] Step 3: parses Opus response as JSON: `{"resolved_patch": [...], "resolution_rationale": "..."}`
- [ ] Step 4: validates resolved patch via `jsonpatch.apply_patch` test application
- [ ] On Opus parse failure → falls back to `fast_merge(non_conflicting_deltas)` (partial merge)
- [ ] **Opus is ONLY used for conflict resolution** — never for simple merges
- [ ] Records `{conflict_count, resolved, used_opus, latency_ms}` to events.jsonl

**Test Plan**:
- BB: Non-conflicting deltas → Opus NOT called (`used_opus=False`)
- BB: Conflicting deltas → Opus called, unified patch returned
- BB: Opus returns invalid JSON → fallback to fast_merge
- WB: Conflict detector called before any Opus invocation
- WB: `used_opus=False` in events.jsonl for fast-path resolves

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 7.03: MergePromptBuilder — Structured Conflict Prompt
**File**: `core/merge/merge_prompt_builder.py`

- **As** the SemanticMergeInterceptor, **I need** a prompt builder that constructs the exact Opus conflict-resolution prompt defined in the architecture spec, **so that** Opus receives maximally structured context for each merge operation.

**Acceptance Criteria**:
- [ ] `MergePromptBuilder` class with `build(deltas, conflict_report, current_state) -> str`
- [ ] Output prompt matches architecture spec template:
  ```
  You are the Genesis Semantic Reducer. Multiple swarm workers have produced conflicting outputs.
  CURRENT STATE (version {version}): {current_state}
  CONFLICT REPORT: Paths in conflict: {conflicting_paths}
  WORKER PROPOSALS:
  Worker {agent_id}: {delta.patch}
  ...
  Respond ONLY with valid JSON: {"resolved_patch": [...], "resolution_rationale": "..."}
  ```
- [ ] `current_state` truncated to 2000 chars if longer (prevents context overflow)
- [ ] Each worker's patch formatted as readable JSON with agent_id label
- [ ] Output is deterministic for same inputs (no randomness in builder)
- [ ] Prompt length logged to events.jsonl

**Test Plan**:
- BB: Prompt contains "You are the Genesis Semantic Reducer"
- BB: All conflicting agent IDs appear in prompt
- BB: Current state > 2000 chars → truncated in prompt
- WB: Output format is deterministic (same input → same output)
- WB: "Respond ONLY with valid JSON" instruction present

**Budget**: 4 implement + 3 BB + 3 WB + 1 IT + 1 stamp = 12 iterations

---

### Story 7.04: PatchReconciler — Merged Patch Validator + Applier
**File**: `core/merge/patch_reconciler.py`

- **As** the OCC commit engine, **I need** a patch reconciler that validates the Opus-resolved patch before it's committed, **so that** syntactically invalid or logically broken patches are rejected before touching master state.

**Acceptance Criteria**:
- [ ] `PatchReconciler` class with `validate_and_apply(state: dict, patch: list) -> ReconcileResult`
- [ ] `ReconcileResult` dataclass: `valid: bool`, `new_state: Optional[dict]`, `errors: list[str]`
- [ ] Validation step 1: schema check — `patch` is a list of dicts with required `op` and `path` keys
- [ ] Validation step 2: dry-run apply — `jsonpatch.apply_patch(state.copy(), patch)` in try/except
- [ ] Validation step 3: axiom checks — `assert "sqlite3" not in str(new_state)`, `assert "API_KEY" not in str(new_state)`
- [ ] On any validation failure: returns `ReconcileResult(valid=False, new_state=None, errors=[...])`
- [ ] On all passing: returns `ReconcileResult(valid=True, new_state=applied_state, errors=[])`

**Test Plan**:
- BB: Valid patch → `valid=True, new_state != None`
- BB: Patch with invalid `op` type → `valid=False, errors` contains description
- BB: New state containing "API_KEY" → axiom check fails, `valid=False`
- WB: Dry-run uses `state.copy()` (original state not mutated on failure)
- WB: All 3 validation steps run before returning (not fail-fast)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 7.05: MergeTelemetry — Merge Analytics
**File**: `core/merge/merge_telemetry.py`

- **As** the Genesis observability layer, **I need** merge telemetry that tracks conflict rates, Opus invocation rates, and merge latency, **so that** we can monitor if multi-agent coherence is degrading.

**Acceptance Criteria**:
- [ ] `MergeTelemetry` class with `record(merge_result: MergeRecord)` and `get_stats() -> dict`
- [ ] `MergeRecord` dataclass: `session_id`, `delta_count: int`, `conflict_count: int`, `used_opus: bool`, `merge_latency_ms: float`, `success: bool`
- [ ] Redis counters: `genesis:merge:stats:total`, `genesis:merge:stats:conflicts`, `genesis:merge:stats:opus_calls`
- [ ] `get_stats()` returns: `{"total_merges": n, "conflict_rate_pct": float, "opus_rate_pct": float, "avg_latency_ms": float}`
- [ ] `conflict_rate_pct = conflicts / total * 100`
- [ ] Writes full `MergeRecord` to events.jsonl for detailed analytics

**Test Plan**:
- BB: Record 10 merges (3 with conflicts, 2 using Opus) → `conflict_rate_pct=30.0, opus_rate_pct=20.0`
- BB: `get_stats()` reflects cumulative counters
- WB: Redis INCR used for counters (atomic)
- WB: `avg_latency_ms` calculated from list of recorded latencies

**Budget**: 4 implement + 3 BB + 2 WB + 1 IT + 1 stamp = 11 iterations

---

### Story 7.06: CompensatingTransaction — Partial-Fail Recovery
**File**: `core/merge/compensating_transaction.py`

- **As** the saga recovery system, **I need** a compensating transaction handler that generates and applies rollback operations for partially failed sagas, **so that** the system can recover from partial failures without manual intervention.

**Acceptance Criteria**:
- [ ] `CompensatingTransaction` class with `compensate(saga: SwarmSaga) -> CompensationResult`
- [ ] `CompensationResult` dataclass: `success: bool`, `compensating_ops: list[dict]`, `final_status: str`
- [ ] For each failed agent's proposed delta: generates inverse patch (`"add"` → `"remove"`, `"replace"` → restore original value)
- [ ] Applies compensating patches to master state via `OCCCommitEngine`
- [ ] Records compensation event to ColdLedger with `event_type="compensation_applied"`
- [ ] On compensation failure: escalates to `MemGPTEscalation`
- [ ] `invert_patch(patch: list) -> list` — generates RFC 6902 inverse operations

**Test Plan**:
- BB: Failed saga with `add` op → compensating `remove` op generated
- BB: Compensation applied successfully → master state rolled back
- BB: Compensation event written to ColdLedger
- WB: `invert_patch` correctly inverts all 4 reversible op types (add/remove/replace/move)
- WB: Copy op has no inverse → raises `IrreversibleOperationError`

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 7.07: Test Suite — Module 7 Semantic Merge
**File**: `tests/merge/test_semantic_merge.py`

**Acceptance Criteria**:
- [ ] BB: Non-conflicting merge → fast path (no Opus)
- [ ] BB: Conflicting merge → Opus called, resolved patch returned
- [ ] BB: Axiom violation in new state → rejected
- [ ] BB: Partial-fail saga → compensating transaction applied
- [ ] WB: ConflictDetector runs before Opus invocation (verified by call order)
- [ ] WB: `fast_merge` path has zero API calls
- [ ] 14+ total test cases, all Opus calls mocked

**Budget**: 6 implement + 1 run + 1 fix + 1 re-run + 1 stamp = 10 iterations

---


Successfully appended to E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md (194 lines)


## MODULE 8: Dual-Loop Architecture + Shadow Arena
**Q7 Implementation — Structural Self-Improvement Foundation**
**Stories**: 9 | **Estimate**: 6-7 days | **Wave**: 4 (after all prior modules)

**Module Vision**: The Dual-Loop is what makes Genesis not just a tool but an evolving system. The Actor Loop executes. The Meta Loop observes, proposes structural improvements, tests them in a Shadow Arena, and submits GitOps PRs. Kinan reviews and merges. Genesis is structurally reborn after each merge. This is not prompt tuning — this is code-level architectural self-improvement with a human merge gate.

**Module File Map**:
```
core/evolution/immutable_kernel.py          ← Story 8.01
core/evolution/axiomatic_tests.py           ← Story 8.02
core/evolution/meta_architect.py            ← Story 8.03
core/evolution/shadow_arena.py              ← Story 8.04
core/evolution/scar_aggregator.py           ← Story 8.05
core/evolution/code_proposer.py             ← Story 8.06
core/evolution/gitops_pr_creator.py         ← Story 8.07
core/evolution/tier1_autonomous_updater.py  ← Story 8.08
tests/evolution/test_dual_loop.py           ← Story 8.09
```

---

### Story 8.01: ImmutableKernel — File Permission Locker
**File**: `core/evolution/immutable_kernel.py`

- **As** the Genesis self-preservation system, **I need** an immutable kernel manager that identifies and locks the core files that Genesis cannot modify, **so that** the self-improvement system cannot wireheaded itself by modifying its own evaluators.

**Acceptance Criteria**:
- [ ] `ImmutableKernel` class with `lock_kernel() -> LockResult` and `verify_kernel() -> VerifyResult`
- [ ] `KERNEL_FILES` constant list — exact files from architecture spec:
  - `core/interceptors/base_interceptor.py`
  - `tests/axiomatic/test_axioms.py`
  - `core/storage/shadow_router.py` (Shadow Router isolation logic)
  - `credentials/` directory (no modifications allowed)
- [ ] `lock_kernel()` — sets files to read-only (`chmod 444` equivalent on Windows: `attrib +R`)
- [ ] `verify_kernel() -> VerifyResult` — checks all kernel files are still read-only
- [ ] `VerifyResult` dataclass: `intact: bool`, `modified_files: list[str]`
- [ ] `is_kernel_file(filepath: str) -> bool` — True if filepath in KERNEL_FILES
- [ ] On Windows: uses `os.chmod(path, stat.S_IREAD)` for read-only

**Implementation Skeleton**:
```python
import os, stat
from pathlib import Path

KERNEL_FILES = [
    "core/interceptors/base_interceptor.py",
    "tests/axiomatic/test_axioms.py",
    "core/storage/shadow_router.py",
]

class ImmutableKernel:
    def lock_kernel(self) -> LockResult:
        locked = []
        failed = []
        for filepath in KERNEL_FILES:
            try:
                os.chmod(filepath, stat.S_IREAD | stat.S_IRGRP | stat.S_IROTH)
                locked.append(filepath)
            except Exception as e:
                failed.append((filepath, str(e)))
        return LockResult(locked=locked, failed=failed)

    def is_kernel_file(self, filepath: str) -> bool:
        return any(k in filepath for k in KERNEL_FILES)
```

**Test Plan**:
- BB: `lock_kernel()` → files become read-only (write attempt raises PermissionError)
- BB: `verify_kernel()` on intact kernel → `intact=True, modified_files=[]`
- BB: `is_kernel_file("core/interceptors/base_interceptor.py")` → True
- WB: Lock uses `stat.S_IREAD` (not chmod 000 — readable but not writable)
- WB: `lock_kernel()` skips non-existent files with warning (not error)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 8.02: AxiomaticTests — Compiled GLOBAL_GENESIS_RULES Assertions
**File**: `core/evolution/axiomatic_tests.py`

- **As** the Genesis self-improvement guardrail, **I need** axiomatic test assertions compiled from `GLOBAL_GENESIS_RULES.md`, **so that** any proposed architectural change is instantly rejected if it violates a Genesis axiom.

**Acceptance Criteria**:
- [ ] `AxiomaticTests` class with `run_all(code_content: str, state_content: dict) -> AxiomResult`
- [ ] `AxiomResult` dataclass: `passed: bool`, `violations: list[AxiomViolation]`
- [ ] `AxiomViolation` dataclass: `axiom_id: str`, `description: str`, `evidence: str`
- [ ] **Axiom 1**: `"sqlite3"` not in any code file → `AXIOM_NO_SQLITE`
- [ ] **Axiom 2**: `"API_KEY"` or `"sk-"` not in any swarm output → `AXIOM_NO_API_KEY_LEAK`
- [ ] **Axiom 3**: `"WHERE version ="` present in all DB write operations in new code → `AXIOM_OCC_REQUIRED`
- [ ] **Axiom 4**: No writes to `C:` drive paths → `AXIOM_NO_C_DRIVE`
- [ ] **Axiom 5**: `"gemini.google.com"` accesses require auth check → `AXIOM_GEMINI_AUTH`
- [ ] `run_all` checks all 5 axioms, returns all violations (not fail-fast)

**Test Plan**:
- BB: Code with `import sqlite3` → `AXIOM_NO_SQLITE` violation
- BB: State output with `"sk-abc123"` → `AXIOM_NO_API_KEY_LEAK` violation
- BB: Clean code + clean state → `passed=True, violations=[]`
- WB: All 5 axioms checked before returning (not fail-fast)
- WB: `violations` list is empty list (not None) when all pass

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 8.03: MetaArchitect — Scar-Driven Structural Analysis
**File**: `core/evolution/meta_architect.py`

- **As** the Genesis Meta Loop, **I need** a Meta-Architect that reads Qdrant L3 scars and Postgres L4 sagas, identifies structural bottlenecks, and determines if a code-level fix is warranted, **so that** the Nightly Epoch has intelligent direction for what to improve.

**Acceptance Criteria**:
- [ ] `MetaArchitect` class with `analyze(lookback_days: int = 7) -> ArchitectureAnalysis`
- [ ] `ArchitectureAnalysis` dataclass: `bottlenecks: list[Bottleneck]`, `recommended_fixes: list[FixProposal]`, `scope: str` ("epistemic"|"ontological")
- [ ] `Bottleneck` dataclass: `description: str`, `frequency: int`, `affected_saga_ids: list[str]`, `scar_ids: list[str]`
- [ ] Queries Qdrant L3: cluster scars by semantic similarity → identify recurring failure patterns
- [ ] Queries Postgres L4: `SELECT * FROM swarm_sagas WHERE status='PARTIAL_FAIL' AND created_at > NOW() - INTERVAL '{lookback_days} days'`
- [ ] `scope="epistemic"` → prompt-level fix (no code PR needed)
- [ ] `scope="ontological"` → code-level fix required (Tier 2 — needs PR)
- [ ] Writes analysis to `data/observability/meta_architect_log.jsonl`

**Test Plan**:
- BB: 10 failed sagas in last 7 days → at least 1 bottleneck identified
- BB: `scope="epistemic"` returned for prompt-fixable issues
- BB: Analysis written to meta_architect_log.jsonl
- WB: Qdrant cluster query groups similar scars (not per-scar analysis)
- WB: `lookback_days` parameter drives Postgres interval correctly

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 8.04: ShadowArena — Containerized Test Sandbox
**File**: `core/evolution/shadow_arena.py`

- **As** the Meta-Loop evaluation system, **I need** a Shadow Arena that tests proposed architectural changes against historical failed sagas in isolation, **so that** proposed code changes are validated before a PR is opened.

**Acceptance Criteria**:
- [ ] `ShadowArena` class with `evaluate_proposal(proposal_branch: str, test_saga_ids: list[str]) -> ArenaResult`
- [ ] `ArenaResult` dataclass: `pass_rate: float`, `axiom_violations: list[str]`, `improved_metrics: dict`, `ready_for_pr: bool`
- [ ] Fetches historical failed sagas from Postgres by `test_saga_ids`
- [ ] Injects saga inputs into Redis L1 (Shadow Mode — `ShadowRouter` blocks all external calls)
- [ ] Re-runs failed sagas using the proposed code branch (via Python import of proposed module)
- [ ] Compares metrics: `old_success_rate` vs `new_success_rate` for tested sagas
- [ ] `ready_for_pr = pass_rate >= 0.8 AND axiom_violations == []`
- [ ] Writes full arena run to `data/observability/shadow_arena_runs.jsonl`

**Test Plan**:
- BB: Proposal that fixes 9/10 failed sagas → `pass_rate=0.9, ready_for_pr=True`
- BB: Proposal with axiom violation → `ready_for_pr=False` regardless of pass_rate
- BB: Arena run written to shadow_arena_runs.jsonl
- WB: `ShadowRouter` in SHADOW mode during all arena runs (no real external calls)
- WB: `ready_for_pr` requires BOTH pass_rate and axiom criteria

**Budget**: 8 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 17 iterations

---

### Story 8.05: ScarAggregator — L3 Failure Pattern Collector
**File**: `core/evolution/scar_aggregator.py`

- **As** the Nightly Epoch, **I need** a scar aggregator that collects, deduplicates, and clusters failure patterns from Qdrant L3, **so that** the Meta-Architect has structured failure intelligence rather than raw vectors.

**Acceptance Criteria**:
- [ ] `ScarAggregator` class with `aggregate(lookback_days: int = 7) -> ScarReport`
- [ ] `ScarReport` dataclass: `total_scars: int`, `clusters: list[ScarCluster]`, `new_since_last_epoch: int`
- [ ] `ScarCluster` dataclass: `cluster_id: str`, `representative_scar: str`, `member_count: int`, `avg_severity: float`
- [ ] Queries Qdrant `genesis_scars` for scars in last `lookback_days`
- [ ] Clusters scars by cosine similarity threshold (0.85 = same cluster)
- [ ] `new_since_last_epoch` counted by comparing scar timestamps vs last epoch timestamp
- [ ] Writes `ScarReport` to `data/observability/scar_aggregation_log.jsonl`
- [ ] `get_top_clusters(n=5) -> list[ScarCluster]` — most-frequent clusters by member_count

**Test Plan**:
- BB: 10 scars with similar vectors → 1-2 clusters returned
- BB: `new_since_last_epoch` > 0 when new scars added after last epoch
- BB: Aggregation report written to log
- WB: Clustering uses cosine similarity threshold (not k-means)
- WB: `get_top_clusters(5)` returns ≤5 clusters sorted by `member_count DESC`

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 8.06: CodeProposer — Structural Fix Generator
**File**: `core/evolution/code_proposer.py`

- **As** the Meta-Architect, **I need** a code proposer that uses Opus to generate the Python code for proposed architectural fixes, **so that** the Nightly Epoch can author interceptors and config changes programmatically.

**Acceptance Criteria**:
- [ ] `CodeProposer` class with `propose(bottleneck: Bottleneck, existing_code_context: str) -> CodeProposal`
- [ ] `CodeProposal` dataclass: `file_path: str`, `code_content: str`, `test_file_path: str`, `test_content: str`, `config_changes: dict`
- [ ] Opus prompt includes: bottleneck description, scar examples, existing interceptor code as reference
- [ ] Generated code MUST extend `BaseInterceptor` (validated post-generation)
- [ ] Generated tests MUST follow `tests/` structure convention
- [ ] `validate_proposal(proposal: CodeProposal) -> bool` — runs `AxiomaticTests` on generated code
- [ ] Writes proposal to `data/evolution/proposals/{epoch_id}/` directory
- [ ] Rejects proposals that fail axiomatic tests (no PR opened for violations)

**Test Plan**:
- BB: Generated interceptor extends `BaseInterceptor` (verified by AST inspection)
- BB: Proposal fails axiomatic test → `validate_proposal` returns False
- BB: Proposal written to `data/evolution/proposals/` directory
- WB: Opus prompt contains existing interceptor code as reference
- WB: `test_content` follows naming convention `test_{module_name}.py`

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 8.07: GitOpsPRCreator — GitHub PR Automation
**File**: `core/evolution/gitops_pr_creator.py`

- **As** the Meta-Loop, **I need** a GitOps PR creator that opens a GitHub PR with the proposed architectural changes + Shadow Arena results, **so that** Kinan has a single click-to-merge path for approving structural improvements.

**Acceptance Criteria**:
- [ ] `GitOpsPRCreator` class with `create_pr(proposal: CodeProposal, arena_result: ArenaResult) -> PRResult`
- [ ] `PRResult` dataclass: `pr_url: str`, `branch_name: str`, `pr_number: int`
- [ ] Creates branch: `genesis-auto-refactor-{epoch_id}` from `main`
- [ ] Writes proposed files to branch (uses `git` subprocess calls on E: drive repo)
- [ ] Opens PR via GitHub API (`POST /repos/{owner}/{repo}/pulls`)
- [ ] PR title: `[Genesis Auto] {bottleneck.description[:60]} — Epoch {epoch_id}`
- [ ] PR body includes: Shadow Arena pass_rate, improved metrics, scar IDs that motivated change
- [ ] GitHub token from `E:\genesis-system\credentials\GITHUB_TOKEN` file
- [ ] `GITHUB_REPO` from env variable (format: `owner/repo`)

**Test Plan**:
- BB: Valid proposal + arena result → PR URL returned
- BB: Branch name follows `genesis-auto-refactor-{epoch_id}` format
- BB: PR body contains Shadow Arena metrics
- WB: GitHub API call uses token from credentials file (not env variable)
- WB: Git operations run on E: drive repo (not C: drive)

**Budget**: 7 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 16 iterations

---

### Story 8.08: Tier1AutonomousUpdater — Epistemic Self-Updates
**File**: `core/evolution/tier1_autonomous_updater.py`

- **As** the Meta-Loop, **I need** a Tier 1 autonomous updater that applies epistemic changes (KG entities, Qdrant scars, prompt templates) without requiring a PR, **so that** the system continuously learns from every epoch without human bottlenecks.

**Acceptance Criteria**:
- [ ] `Tier1AutonomousUpdater` class with `apply_tier1(analysis: ArchitectureAnalysis) -> Tier1Result`
- [ ] `Tier1Result` dataclass: `kg_entities_added: int`, `scars_updated: int`, `prompts_updated: int`, `rules_updated: int`
- [ ] **KG entities**: `KNOWLEDGE_GRAPH/entities/epoch_{epoch_id}_learnings.jsonl` — write new axioms
- [ ] **Qdrant scars**: upsert new consolidated scars from epoch aggregation
- [ ] **Prompt templates**: update files in `config/prompts/` with improved template versions
- [ ] **GLOBAL_GENESIS_RULES.md**: appends new rule if analysis identifies missing guardrail (TEXT addition only — not code)
- [ ] All Tier 1 updates write audit entry to `data/observability/tier1_updates.jsonl`
- [ ] None of the updates modify `.py` files (Tier 1 = no code changes)

**Test Plan**:
- BB: Tier 1 update → KG entity file created in `KNOWLEDGE_GRAPH/entities/`
- BB: No `.py` files modified by Tier 1 update
- BB: Audit entry written to tier1_updates.jsonl
- WB: Qdrant upsert uses `genesis_scars` collection (not any new collection)
- WB: GLOBAL_GENESIS_RULES.md updates are append-only (existing rules not modified)

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 8.09: Test Suite — Module 8 Dual-Loop
**File**: `tests/evolution/test_dual_loop.py`

**Acceptance Criteria**:
- [ ] BB: Immutable kernel locked → write attempt raises PermissionError
- [ ] BB: Axiom violation → proposal rejected (no PR opened)
- [ ] BB: Shadow Arena pass_rate < 0.8 → `ready_for_pr=False`
- [ ] BB: Tier 1 update creates no `.py` files
- [ ] WB: `SHADOW_MODE="SHADOW"` during all arena runs
- [ ] WB: GitOps PR branch naming convention verified
- [ ] 16+ test cases

**Budget**: 7 implement + 1 run + 1 fix + 1 re-run + 1 stamp = 11 iterations

---


Successfully appended to E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md (278 lines) ✅ File written successfully! (278 lines)

💡 Performance tip: For optimal speed, consider chunking files into ≤30 line pieces in future operations.


## MODULE 9: Nightly Epoch — Component E (Meta-Architect Scheduler)
**Q7 Implementation — Weekly Structural Rebirth**
**Stories**: 10 | **Estimate**: 5-6 days | **Wave**: 4 (after Module 8)

**Module Vision**: Every Sunday at 2AM, Genesis wakes as Meta-Architect. It scans the week's failures, distills axioms, tests improvements in the Shadow Arena, and opens a GitOps PR. Kinan merges on Monday morning. Genesis has evolved. This is not a cron job — this is the weekly heartbeat of a self-improving system. The Nightly Epoch is what separates a static tool from a living AI OS.

**Module File Map**:
```
core/epoch/epoch_scheduler.py               ← Story 9.01
core/epoch/epoch_runner.py                  ← Story 9.02
core/epoch/redis_epoch_lock.py              ← Story 9.03
core/epoch/conversation_aggregator.py       ← Story 9.04
core/epoch/axiom_distiller.py               ← Story 9.05
core/epoch/epoch_knowledge_writer.py        ← Story 9.06
core/epoch/epoch_report_generator.py        ← Story 9.07
core/epoch/epoch_tier1_trigger.py           ← Story 9.08
tests/epoch/test_nightly_epoch.py           ← Story 9.09
tests/epoch/test_epoch_integration.py       ← Story 9.10
```

---

### Story 9.01: EpochScheduler — APScheduler Cron Trigger
**File**: `core/epoch/epoch_scheduler.py`

- **As** the Nightly Epoch system, **I need** an APScheduler-based cron scheduler that triggers the epoch runner every Sunday at 2AM AEST, **so that** structural self-improvement fires on a reliable, predictable schedule.

**Acceptance Criteria**:
- [ ] `EpochScheduler` class with `start() -> None` and `stop() -> None`
- [ ] Uses `apscheduler.schedulers.asyncio.AsyncIOScheduler`
- [ ] Cron trigger: `day_of_week='sun'`, `hour=16`, `minute=0` (UTC+0 for 2AM AEST/UTC+10)
- [ ] Calls `EpochRunner.run_epoch_safe()` (not raw `run_epoch` — uses lock)
- [ ] `get_next_run() -> datetime` — returns next scheduled epoch time
- [ ] `force_trigger() -> None` — for testing, immediately triggers epoch (bypasses cron)
- [ ] Scheduler auto-starts on `start()`, gracefully stops on `stop()`
- [ ] Logs scheduler start/stop to events.jsonl

**Implementation Skeleton**:
```python
from apscheduler.schedulers.asyncio import AsyncIOScheduler

class EpochScheduler:
    def __init__(self, runner: 'EpochRunner'):
        self.scheduler = AsyncIOScheduler()
        self.runner = runner

    def start(self) -> None:
        self.scheduler.add_job(
            self.runner.run_epoch_safe,
            'cron',
            day_of_week='sun',
            hour=16,   # UTC 16:00 = AEST 02:00
            minute=0,
            id='nightly_epoch'
        )
        self.scheduler.start()

    async def force_trigger(self) -> None:
        await self.runner.run_epoch_safe()
```

**Test Plan**:
- BB: `force_trigger()` calls `run_epoch_safe()`
- BB: `get_next_run()` returns future Sunday datetime
- BB: Start + stop + restart cycle — no duplicate jobs
- WB: Cron hour=16 (UTC) not hour=2 (AEST)
- WB: `run_epoch_safe` (lock version) called, not `run_epoch` directly

**Budget**: 5 implement + 3 BB + 3 WB + 1 IT + 1 stamp = 13 iterations

---

### Story 9.02: EpochRunner — Full Epoch Orchestrator
**File**: `core/epoch/epoch_runner.py`

- **As** the Nightly Epoch system, **I need** an epoch runner that coordinates all epoch phases (aggregate → distill → write → propose → arena → PR → tier1), **so that** the epoch is a reliable, end-to-end atomic process.

**Acceptance Criteria**:
- [ ] `EpochRunner` class with `run_epoch(epoch_id: str) -> EpochResult` and `run_epoch_safe() -> Optional[EpochResult]`
- [ ] `EpochResult` dataclass: `epoch_id: str`, `success: bool`, `phases_completed: list[str]`, `axioms_distilled: int`, `pr_opened: Optional[str]`, `tier1_updates: int`, `duration_seconds: float`
- [ ] `run_epoch_safe()` acquires `RedisEpochLock` before calling `run_epoch()` (prevents duplicate runs)
- [ ] 7-phase execution:
  1. `ConversationAggregator.aggregate(lookback_days=7)` → weekly conversation summary
  2. `ScarAggregator.aggregate(lookback_days=7)` → scar clusters
  3. `AxiomDistiller.distill(conversations, scars)` → new axioms
  4. `EpochKnowledgeWriter.write(axioms)` → KG + Qdrant persistence
  5. `MetaArchitect.analyze()` → structural analysis
  6. `CodeProposer.propose()` + `ShadowArena.evaluate()` → IF ontological scope
  7. `GitOpsPRCreator.create_pr()` → IF arena passes, OR `Tier1AutonomousUpdater.apply_tier1()`
- [ ] Writes epoch log to `data/observability/epoch_log.jsonl`
- [ ] Total epoch max duration: 2 hours (7200s timeout)

**Test Plan**:
- BB: Successful epoch → `EpochResult.success=True, phases_completed` has all 7 phases
- BB: `run_epoch_safe()` with lock held → returns None immediately (no duplicate run)
- BB: Epoch log written to epoch_log.jsonl
- WB: Phase execution sequential (not parallel) — each phase result feeds into next
- WB: Total duration < 7200s enforced (asyncio.wait_for)

**Budget**: 8 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 17 iterations

---

### Story 9.03: RedisEpochLock — Distributed Epoch Lock
**File**: `core/epoch/redis_epoch_lock.py`

- **As** the epoch scheduler, **I need** a Redis distributed lock for the epoch process, **so that** only one epoch can run at a time even if multiple Genesis instances are running.

**Acceptance Criteria**:
- [ ] `RedisEpochLock` class with `acquire() -> bool` and `release() -> None`
- [ ] Redis key: `epoch:lock:nightly` (matches architecture spec)
- [ ] TTL: `7200` seconds (2 hours — max epoch runtime)
- [ ] `acquire()` uses Redis `SET NX EX` (atomic set-if-not-exists with expiry)
- [ ] `acquire()` returns `True` if lock acquired, `False` if already held
- [ ] `release()` deletes the key (but only if this instance holds it)
- [ ] Lock value: `{epoch_id}:{host}:{pid}` — allows identification of lock holder
- [ ] `get_lock_holder() -> Optional[str]` — returns current lock value or None
- [ ] Context manager support: `async with RedisEpochLock() as lock:`

**Implementation Skeleton**:
```python
EPOCH_LOCK_KEY = "epoch:lock:nightly"
EPOCH_LOCK_TTL = 7200

class RedisEpochLock:
    async def acquire(self, epoch_id: str) -> bool:
        value = f"{epoch_id}:{socket.gethostname()}:{os.getpid()}"
        result = await self.redis.set(
            EPOCH_LOCK_KEY, value,
            nx=True, ex=EPOCH_LOCK_TTL
        )
        return result is not None

    async def release(self, epoch_id: str) -> None:
        # Only release if we own the lock (prevent accidental release)
        current = await self.redis.get(EPOCH_LOCK_KEY)
        if current and current.decode().startswith(epoch_id):
            await self.redis.delete(EPOCH_LOCK_KEY)
```

**Test Plan**:
- BB: First `acquire()` → True; second `acquire()` → False
- BB: `release()` → subsequent `acquire()` returns True
- BB: `get_lock_holder()` returns lock value while held
- WB: `SET NX EX` used (atomic operation, not GET+SET)
- WB: Lock value contains `epoch_id` (verified by `startswith(epoch_id)` check)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 9.04: ConversationAggregator — Weekly Kinan<>Genesis Summary
**File**: `core/epoch/conversation_aggregator.py`

- **As** the Nightly Epoch, **I need** a conversation aggregator that pulls the week's Genesis execution history from Postgres L4, **so that** the AxiomDistiller has real conversation data to work with.

**Acceptance Criteria**:
- [ ] `ConversationAggregator` class with `aggregate(lookback_days: int = 7) -> WeeklyConversationSummary`
- [ ] `WeeklyConversationSummary` dataclass: `total_sessions: int`, `total_tasks: int`, `failed_tasks: int`, `conversation_snippets: list[str]`, `period_start: datetime`, `period_end: datetime`
- [ ] Queries Postgres: `SELECT * FROM swarm_sagas WHERE created_at > NOW() - INTERVAL '{lookback_days} days'`
- [ ] Queries Postgres: `SELECT * FROM events WHERE event_type IN ('dispatch_start', 'mvfl_escalation', 'saga_committed') AND created_at > NOW() - INTERVAL '{lookback_days} days'`
- [ ] `conversation_snippets`: extracts up to 20 most significant events (MVFL escalations, saga completions, corrections)
- [ ] Returns only sanitized data (no API keys, no personal data) — runs through axiom filter

**Test Plan**:
- BB: 10 sagas in last 7 days → `total_sessions > 0`
- BB: `lookback_days=1` → only today's data
- BB: Snippets sanitized (no "API_KEY" in any snippet)
- WB: Postgres query uses parameterized interval (not string concatenation)
- WB: Snippets limited to 20 entries (not all events)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 9.05: AxiomDistiller — Gemini Pro Weekly Distillation
**File**: `core/epoch/axiom_distiller.py`

- **As** the Nightly Epoch, **I need** an axiom distiller that uses Gemini Pro to extract 5-10 high-value axioms from the week's conversation data and scar clusters, **so that** Genesis's living memory grows richer with every epoch.

**Acceptance Criteria**:
- [ ] `AxiomDistiller` class with `distill(conversations: WeeklyConversationSummary, scars: ScarReport) -> list[Axiom]`
- [ ] `Axiom` dataclass: `id: str` (format: `epoch_{date}_{N:03d}`), `content: str`, `category: str`, `confidence: float` (0.0-1.0), `source_saga_ids: list[str]`
- [ ] Uses Gemini Pro (not Flash — distillation quality matters)
- [ ] Distillation prompt matches architecture spec: "Extract 5-10 high-value axioms..."
- [ ] Parses response as JSON: `{"axioms": [...], "week_summary": "..."}`
- [ ] Rejects axioms with `confidence < 0.6` (low-confidence learnings discarded)
- [ ] Returns 5-10 axioms (not fewer, not more — Gemini instructed to match count)
- [ ] `week_summary` string stored alongside axioms in EpochResult

**Implementation Skeleton**:
```python
DISTILLATION_PROMPT = """
You are the Genesis Memory Distiller. Review this week's Genesis execution data.
TOTAL SESSIONS: {total_sessions}
FAILED TASKS: {failed_tasks}
SIGNIFICANT EVENTS: {snippets}
RECURRING FAILURE CLUSTERS: {scar_clusters}

Extract 5-10 high-value axioms — insights that, if hardwired, would prevent future failures.
Return ONLY valid JSON:
{{"axioms": [{{"id": "epoch_{date}_001", "content": "...", "category": "...", "confidence": 0.9, "source_saga_ids": [...]}}],
 "week_summary": "2-sentence summary of this week"}}
"""
```

**Test Plan**:
- BB: Returns 5-10 axioms (mocked Gemini response)
- BB: Axioms with `confidence < 0.6` filtered out
- BB: Axiom ID format: `epoch_2026_02_25_001`
- WB: Gemini Pro model used (not Flash) — verified by model param in API call
- WB: JSON parse failure → empty list returned (not exception)

**Budget**: 6 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 15 iterations

---

### Story 9.06: EpochKnowledgeWriter — Axiom Persistence (KG + Qdrant)
**File**: `core/epoch/epoch_knowledge_writer.py`

- **As** the Nightly Epoch, **I need** a knowledge writer that persists distilled axioms to both the Knowledge Graph and Qdrant L3, **so that** every epoch's learnings become part of the permanent Genesis memory.

**Acceptance Criteria**:
- [ ] `EpochKnowledgeWriter` class with `write(axioms: list[Axiom], epoch_id: str) -> WriteResult`
- [ ] `WriteResult` dataclass: `kg_file_path: str`, `qdrant_upserts: int`, `jsonl_entries: int`
- [ ] **KG write**: appends to `KNOWLEDGE_GRAPH/axioms/genesis_evolution_learnings.jsonl`
- [ ] **Qdrant write**: upserts each axiom as vector in `genesis_scars` collection with `axiom_type="distilled"` tag
- [ ] KG entry format: `{"id": axiom.id, "content": axiom.content, "category": axiom.category, "epoch": epoch_id, "date": today}`
- [ ] Vector embedding: `fast_embed(axiom.content)` — same embedding function used throughout
- [ ] `WriteResult.kg_file_path` = absolute path to updated KG file
- [ ] All writes atomic — if Qdrant fails, KG write still proceeds (partial success is valid)

**Test Plan**:
- BB: After `write()`, KG jsonl file has new entries
- BB: After `write()`, Qdrant has new vectors with `axiom_type="distilled"` tag
- BB: Qdrant failure → KG still written (partial success)
- WB: KG write uses append mode (not overwrite)
- WB: Each axiom gets individual Qdrant upsert (not batch — for error isolation)

**Budget**: 5 implement + 4 BB + 3 WB + 1 IT + 1 stamp = 14 iterations

---

### Story 9.07: EpochReportGenerator — Epoch Summary Report
**File**: `core/epoch/epoch_report_generator.py`

- **As** the Nightly Epoch, **I need** an epoch report generator that produces a structured summary of the epoch run, **so that** Kinan can review what Genesis learned this week in a readable format.

**Acceptance Criteria**:
- [ ] `EpochReportGenerator` class with `generate(result: EpochResult) -> EpochReport`
- [ ] `EpochReport` dataclass: `markdown_content: str`, `file_path: str`, `epoch_id: str`
- [ ] Markdown report includes: epoch date, phases completed, axioms distilled (list), PR URL if opened, tier1 updates count, week summary
- [ ] Written to: `data/observability/epoch_reports/epoch_{epoch_id}.md`
- [ ] Report header: `# Genesis Nightly Epoch — {date}\n\n## Status: {COMPLETED|PARTIAL|FAILED}`
- [ ] If PR opened: includes link and Shadow Arena pass rate
- [ ] Report generation is pure Python (no LLM calls)

**Test Plan**:
- BB: Generated markdown contains "# Genesis Nightly Epoch"
- BB: File written to correct path
- BB: PR URL included when `result.pr_opened is not None`
- WB: No LLM calls made during report generation
- WB: File path follows `epoch_{epoch_id}.md` convention

**Budget**: 4 implement + 3 BB + 2 WB + 1 IT + 1 stamp = 11 iterations

---

### Story 9.08: EpochTier1Trigger — Autonomous Epistemic Updates Post-Epoch
**File**: `core/epoch/epoch_tier1_trigger.py`

- **As** the Nightly Epoch conclusion, **I need** a tier1 trigger that applies epistemic updates autonomously at the end of every epoch regardless of whether a PR was opened, **so that** Genesis always makes incremental progress even when no structural PR is warranted.

**Acceptance Criteria**:
- [ ] `EpochTier1Trigger` class with `apply(axioms: list[Axiom], analysis: ArchitectureAnalysis) -> Tier1EpochResult`
- [ ] `Tier1EpochResult` dataclass: `kg_axioms_written: int`, `qdrant_scars_updated: int`, `prompt_templates_updated: int`, `rules_appended: int`
- [ ] Always runs (regardless of PR outcome)
- [ ] Uses `EpochKnowledgeWriter` for KG + Qdrant
- [ ] Uses `Tier1AutonomousUpdater` for prompt templates + rules
- [ ] Trigger fires in EpochRunner phase 7 after PR creation (or instead of, if scope="epistemic")
- [ ] Appends to `data/observability/tier1_updates.jsonl` with `source="nightly_epoch"`

**Test Plan**:
- BB: Always runs after epoch (even with `pr_opened=None`)
- BB: KG axioms written count matches `len(axioms)`
- BB: Source="nightly_epoch" in tier1_updates.jsonl entries
- WB: Calls both `EpochKnowledgeWriter` and `Tier1AutonomousUpdater` (not one or other)
- WB: Does not modify any `.py` files (Tier 1 = epistemic only)

**Budget**: 4 implement + 3 BB + 2 WB + 1 IT + 1 stamp = 11 iterations

---

### Story 9.09: Test Suite — Module 9 Nightly Epoch (Unit)
**File**: `tests/epoch/test_nightly_epoch.py`

**Acceptance Criteria**:
- [ ] BB: EpochScheduler force_trigger fires runner
- [ ] BB: RedisEpochLock prevents duplicate epoch (second acquire returns False)
- [ ] BB: AxiomDistiller rejects low-confidence axioms
- [ ] BB: EpochKnowledgeWriter writes to KG file
- [ ] WB: Gemini Pro (not Flash) used for distillation
- [ ] WB: `epoch:lock:nightly` key has TTL=7200
- [ ] 14+ test cases, all Gemini calls mocked

**Budget**: 6 implement + 1 run + 1 fix + 1 re-run + 1 stamp = 10 iterations

---

### Story 9.10: Test Suite — Module 9 Integration (Full Epoch Run)
**File**: `tests/epoch/test_epoch_integration.py`

**Acceptance Criteria**:
- [ ] Full epoch run with mocked LLM + real Redis + real Postgres (test DB)
- [ ] Verifies all 7 phases execute in order
- [ ] Verifies KG file updated post-run
- [ ] Verifies epoch log entry in `epoch_log.jsonl`
- [ ] Verifies lock released after epoch completes
- [ ] 10+ test cases

**Budget**: 7 implement + 1 run + 2 fix + 1 re-run + 1 stamp = 12 iterations

---


Successfully appended to E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md (325 lines) ✅ File written successfully! (325 lines)

💡 Performance tip: For optimal speed, consider chunking files into ≤30 line pieces in future operations.


Conversation compacted


This session is being continued from a previous conversation that ran out of context. The summary below covers the earlier portion of the conversation.

Analysis:
Let me analyze this conversation thoroughly and chronologically.

**Session Context**: This is a continuation from prior sessions (Session 85 handoff). The conversation resumed with a compacted summary showing what was accomplished in the prior part of this session.

**Prior session accomplishments (from summary)**:
1. Answered Kinan's question: 7 DT architecture (GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_V2.md) is SEPARATE from AIVA RLM Nexus PRD — they are two distinct execution tracks
2. Completed AIVA RLM Nexus PRD v2 — expanded from 1276 to 3222 lines, writing full specs for Modules 4-9 (42 stories)
3. Was about to begin writing the Genesis Persistent Context Architecture v2 Epic PRD (Track B)

**Current Session (after compacted summary)**:

The session resumed and immediately:
1. Set up todos (3 tasks — first two completed, third in_progress)
2. Read the full GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_V2.md blueprint (651 lines)
3. Started writing GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md in chunks

**User message during this session**: Kinan sent a message: "can you put somehwere that this will be a 1000 agent swarm operation? I need this completed today ! non-negotiable 1000 agent swarm with all teams firing. hive delegation in action!"

This was delivered as a system-reminder mid-task and needed to be addressed by embedding the 1000-agent swarm mandate into the PRD.

**What was written in this session**:
The PRD was built in 8 chunks (appended to the same file):

1. **Chunk 1**: PRD header, vision, metrics, architecture overview, clarifying questions, risk assessment, PRD hierarchy, Wave Execution Plan — WITH the 1000-agent swarm mandate from Kinan
2. **Chunk 2**: Module 1 — BaseInterceptor Framework (7 stories: 1.01-1.07)
3. **Chunk 3**: Module 2 — JIT Hydration + Zero-Amnesia State Machine (8 stories: 2.01-2.08)
4. **Chunk 4**: Module 3 — MVFL Error-Correction Trap + Voyager Defense (7 stories: 3.01-3.07)
5. **Chunk 5**: Module 4 — 80/20 Tier Router + Strangler Fig Migration (7 stories: 4.01-4.07)
6. **Chunk 6**: Module 5 — CQRS + Event Sourcing + Postgres Cold Ledger (9 stories: 5.01-5.09)
7. **Chunk 7**: Module 6 — Multi-Agent Coherence, Redis Streams + OCC (9 stories: 6.01-6.09)
8. **Chunk 8**: Module 7 — SemanticMergeInterceptor + Bulkheads (7 stories: 7.01-7.07)
9. **Chunk 9**: Module 8 — Dual-Loop Architecture + Shadow Arena (9 stories: 8.01-8.09)
10. **Chunk 10**: Module 9 — Nightly Epoch, Component E (10 stories: 9.01-9.10)

**File status**: The PRD was being written in append mode. The last chunk written was Module 9 (Stories 9.01-9.10). The summary request appears to have been triggered AFTER Module 9 was fully written.

**Total stories written**: 7+8+7+7+9+9+7+9+10 = 73 atomic stories across 9 modules.

**Pending items**: The PRD was nearly complete — all 9 modules have been written. The remaining work would be a closing section (verification checklist, execution order, etc.) but it's possible the PRD is essentially complete at this point.

Let me verify the file state. The last write was Module 9 chunk (325 lines appended). Given the chunk writing pattern, the PRD should be substantially complete. There may be a closing section needed (verification checklist, execution manifest, etc.) to properly close out the Epic PRD.

**Key technical decisions documented in the PRD**:
- `BaseInterceptor` ABC with 4 integration contracts (pre/post/on_error/on_correction)
- `InterceptorChain` with priority-ordered execution
- `dispatch_to_swarm()` as universal entry point
- `JITHydrationInterceptor` (priority=10) — ZERO_AMNESIA_STATE XML envelope
- `TierRouterInterceptor` (priority=20) — T0/T1/T2 classification
- `MVFLInterceptor` (priority=90) — Voyager Defense + correction loop
- `ColdLedgerInterceptor` (priority=5) — L4 event recording
- `ExecutionTelemetryInterceptor` (priority=0) — events.jsonl logging
- `RedisMasterState` with OCC (WATCH/MULTI/EXEC)
- `SwarmWorkerBase` with XREADGROUP + XACK + PEL 60s timeout
- `StagingArea` — Redis Hash for ProposedDeltas
- `OCCCommitEngine` — 8-step coherence flow
- `SemanticMergeInterceptor` — ConflictDetector + Opus only for conflicts
- `ImmutableKernel` — file permission locking
- `AxiomaticTests` — compiled GLOBAL_GENESIS_RULES assertions
- `MetaArchitect` — scar-driven structural analysis
- `ShadowArena` — containerized test sandbox
- `EpochScheduler` — APScheduler Sunday 2AM AEST
- `RedisEpochLock` — SET NX EX 7200 distributed lock
- 4-Wave execution plan for 1000-agent swarm

**Todo list state**: All 3 todos - first two completed, third in_progress (Write Genesis Persistent Context Architecture v2 Epic PRD)

Summary:
1. Primary Request and Intent:

**Resumed from prior session** — Three tasks were tracked:

**Task 1 (completed prior session)**: Answer Kinan's concurrent question: "does this above encapsulate the full rlm 7 dt responses architecture resolution? or is that separate? because we also want to get that show on the road too!" — The answer: 7 DT architecture (Track B) is SEPARATE from AIVA PRD (Track A). They are two distinct execution tracks: Track A = AIVA Royal Chamber (59 stories, complete), Track B = Genesis Persistent Context Architecture (needs its own Epic PRD).

**Task 2 (completed prior session)**: Complete AIVA RLM Nexus PRD v2 — Modules 4-9 were stubbed. Full atomic story specs were written for all 42 remaining stories, expanding the file from 1276 → 3222 lines.

**Task 3 (in_progress — this session's focus)**: Write the Genesis Persistent Context Architecture v2 Epic PRD (Track B) — the 7 Deep Think answers (Q1-Q7) as executable atomic stories. This is the PRD currently being written to `E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md`.

**Mid-session user directive**: "can you put somehwere that this will be a 1000 agent swarm operation? I need this completed today ! non-negotiable 1000 agent swarm with all teams firing. hive delegation in action!" — This was immediately embedded into the PRD's opening section as a hardwired execution mandate.

---

2. Key Technical Concepts:

- **Two Execution Tracks**: Track A (AIVA Royal Chamber — AIVA_RLM_NEXUS_PRD_V2.md, 59 stories, COMPLETE) and Track B (Genesis Persistent Context — this PRD, 73 stories)
- **PRD Writing Doctrine**: 1 story = 1 file, 1 function/class/schema. Budget: 5-8 implement + 3-5 BB + 2-3 WB + 1 IT + 1 stamp = 12-19 iterations
- **BaseInterceptor ABC**: 4 integration contracts: `pre_execute`, `post_execute`, `on_error`, `on_correction` (Q1 — Inversion of Control)
- **InterceptorChain**: Priority-ordered execution (0=telemetry, 5=cold ledger, 10=JIT hydration, 20=tier router, 90=MVFL)
- **dispatch_to_swarm()**: Universal entry point — every Genesis task flows through this function
- **ZERO_AMNESIA_STATE XML Envelope**: JIT-assembled from L1 Redis + L2 KG + L3 Qdrant in <45ms concurrent scatter-gather (Q5)
- **4-Layer Memory Stack**: L1 Redis (2ms), L2 KG (5ms), L3 Qdrant/Scars (10ms), L4 Postgres/Cold Ledger (50ms)
- **MVFL Error-Correction Trap**: 3 trigger conditions (syntax, semantic, external rejection) → Voyager Defense scorer → CorrectionLoop (max 3 attempts) → MemGPT escalation to Opus (Q2)
- **80/20 Tier Router**: T0 (deterministic Python), T1 (Gemini Flash), T2 (Opus) — 75%+ target for T0/T1 (Q3)
- **Strangler Fig Migration**: `@strangler_fig("task_type")` decorator wraps legacy calls progressively (Q3)
- **CQRS + Event Sourcing**: Append-only Postgres Cold Ledger (`sessions`, `events`, `swarm_sagas` tables) — immutable source of truth (Q5/Q6)
- **Optimistic Concurrency Control (OCC)**: `RedisMasterState` using Redis `WATCH/MULTI/EXEC` — `version` integer gates all state writes (Q6)
- **Redis Consumer Groups**: `XREADGROUP`/`XACK`/PEL 60s timeout for exactly-once task delivery at 1000-agent scale (Q6)
- **StagingArea**: Redis Hash `genesis:staging:{session_id}` — agents submit `StateDelta` RFC 6902 JSON Patches; NO direct state writes (Q6)
- **SemanticMergeInterceptor**: Gemini = Stateless Proposers. Opus = Semantic Reducer. Conflict detection runs first — Opus only invoked on genuine conflicts (Q6)
- **Bulkhead Guard**: `asyncio.gather(return_exceptions=True)` — crashed agent becomes payload, no rollback needed (Q6)
- **Dual-Loop Architecture**: Actor Loop (execution, rigid) ↔ Meta Loop (structural observation, proposes PRs) — never run simultaneously (Q7)
- **Immutable Kernel**: Core files locked with `chmod 444` — Genesis cannot modify its own evaluators (wireheading prevention) (Q7)
- **Axiomatic Tests**: 5 compiled assertions from GLOBAL_GENESIS_RULES.md — any proposed code change failing axioms instantly rejected (Q7)
- **Shadow Arena**: Historical failed sagas replayed against proposed code in SHADOW mode (ShadowRouter blocks external calls) — `pass_rate >= 0.8 AND axiom_violations == []` required for PR (Q7)
- **Nightly Epoch (Component E)**: APScheduler Sunday 2AM AEST (UTC 16:00) — 7-phase process: aggregate → distill → write → analyze → propose → arena/PR → tier1 (Q7)
- **RedisEpochLock**: `SET NX EX 7200` distributed lock on `epoch:lock:nightly` (Q7)
- **Tier 1 Autonomous Updates**: KG entities, Qdrant scars, prompt templates, GLOBAL_GENESIS_RULES text — no code changes, no PR required (Q7)
- **GitOps PR Creator**: Branch `genesis-auto-refactor-{epoch_id}` → Shadow Arena results → GitHub API PR — Kinan is the merge gate (Q7)
- **4-Wave Execution Plan**: Wave 1 (Modules 1+2+3 parallel), Wave 2 (Modules 4+5 parallel), Wave 3 (Modules 6+7 parallel), Wave 4 (Modules 8 then 9)
- **1000-Agent Swarm Mandate**: Kinan directive — all teams firing, hive delegation, complete today
- **Elestio PostgreSQL** (not SQLite — HARDWIRED rule), Redis via `REDIS_URL` env variable
- **RFC 6902 JSON Patch**: `StateDelta` proposals using `jsonpatch` library

---

3. Files and Code Sections:

**`E:\genesis-system\docs\architecture\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_V2.md`** (READ ONLY — 651 lines)
- Blueprint document synthesizing all 7 Deep Think sessions
- Read in 3 chunks (lines 0-200, 200-400, 400-651) to fully absorb architecture before writing PRD
- Key content: 7-question table, 4 integration contracts, MVFL triggers, 80/20 pipeline, Component A/B/C/D/E, Zero-Amnesia memory hierarchy, JIT hydration function, Redis schema, CQRS 8-step flow, swarm_sagas DDL, Dual-Loop diagram, Immutable Kernel, Axiomatic Tests, Nightly Epoch phases, Tier 1/2 deployment decision
- Critical code patterns absorbed:
```python
def interceptor_jit_hydration(task_payload):
    target_entities, intent_string = fast_extract(task_payload)
    intent_vector = fast_embed(intent_string)
    with ThreadPoolExecutor() as executor:
        f_state  = executor.submit(redis.get, f"genesis:state:task:{task_payload.id}")
        f_kg     = executor.submit(kg.get_1_hop_neighborhood, target_entities)
        f_scars  = executor.submit(qdrant.search, intent_vector, top_k=2)
    system_injection = f"""<ZERO_AMNESIA_STATE>...</ZERO_AMNESIA_STATE>"""
    return dispatch_to_swarm(system_prompt=system_injection, task=task_payload)
```

**`E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md`** (CREATED — written in 10 chunks, ~1900+ lines total)
- New file, created from scratch in this session
- 73 atomic stories across 9 modules
- Key code skeletons written per story:

*Module 1 — BaseInterceptor:*
```python
@dataclass
class InterceptorMetadata:
    name: str
    priority: int  # 0 = first, 100 = last
    enabled: bool = True

class BaseInterceptor(ABC):
    metadata: InterceptorMetadata
    @abstractmethod
    async def pre_execute(self, task_payload: dict) -> dict: ...
    @abstractmethod
    async def post_execute(self, result: dict, task_payload: dict) -> None: ...
    @abstractmethod
    async def on_error(self, error: Exception, task_payload: dict) -> dict: ...
    @abstractmethod
    async def on_correction(self, correction_payload: dict) -> dict: ...
```

*Module 1 — dispatch_to_swarm():*
```python
GLOBAL_CHAIN = InterceptorChain()

async def dispatch_to_swarm(task_payload, tier=None, context=None) -> dict:
    if "task_id" not in task_payload:
        task_payload["task_id"] = str(uuid.uuid4())
    task_payload["dispatched_at"] = datetime.now(timezone.utc).isoformat()
    try:
        enriched = await GLOBAL_CHAIN.execute_pre(task_payload)
        result = await _execute_task(enriched)
        await GLOBAL_CHAIN.execute_post(result, enriched)
        return {**result, "task_id": task_payload["task_id"], "status": "completed"}
    except Exception as e:
        correction = await GLOBAL_CHAIN.execute_error(e, task_payload)
        return {"task_id": task_payload["task_id"], "status": "error", "correction": correction}
```

*Module 2 — RedisL1State:*
```python
@dataclass
class RedisL1State:
    task_id: str
    session_id: str
    active_task_id: str
    focus_entities: list
    current_hypothesis: str
    exhausted_paths: list
    active_blocker: Optional[str]
    version: int = 0

class RedisL1Client:
    KEY_PREFIX = "genesis:state:task:"
    async def get_state(self, task_id: str) -> Optional[RedisL1State]: ...
    async def set_state(self, state: RedisL1State, ttl_seconds: int = 3600) -> bool: ...
```

*Module 3 — MVFLTrigger:*
```python
@dataclass
class MVFLTriggerResult:
    triggered: bool
    trigger_type: Optional[str]  # "syntax" | "semantic" | "external_rejection"
    severity: int  # 1=warning, 2=error, 3=critical
    details: str

class MVFLTrigger:
    def evaluate(self, output: dict, task_payload: dict) -> MVFLTriggerResult:
        if self._check_external_rejection(output):
            return MVFLTriggerResult(True, "external_rejection", 3, ...)
        if self._check_semantic_inconsistency(output, task_payload):
            return MVFLTriggerResult(True, "semantic", 2, ...)
        if self._check_syntax_errors(output, task_payload):
            return MVFLTriggerResult(True, "syntax", 1, ...)
        return MVFLTriggerResult(False, None, 0, "Clean output")
```

*Module 4 — StranglerFig decorator:*
```python
WRAPPED_FUNCTIONS = {}

def strangler_fig(task_type: str, tier: Optional[str] = None):
    def decorator(func):
        WRAPPED_FUNCTIONS[func.__name__] = {"task_type": task_type, "tier": tier}
        @functools.wraps(func)
        async def wrapper(*args, **kwargs):
            original_result = await func(*args, **kwargs)
            task_payload = {"type": task_type, "tier": tier, "args": args, "kwargs": kwargs, "original_result": original_result}
            try:
                return await dispatch_to_swarm(task_payload)
            except Exception:
                return original_result  # Fail-safe fallback
        return wrapper
    return decorator
```

*Module 5 — swarm_sagas DDL:*
```sql
CREATE TABLE IF NOT EXISTS swarm_sagas (
    saga_id          UUID PRIMARY KEY,
    session_id       UUID REFERENCES sessions(id),
    orchestrator_dag JSONB,
    proposed_deltas  JSONB,
    resolved_state   JSONB,
    status           VARCHAR(20),
    created_at       TIMESTAMP DEFAULT NOW()
);
```

*Module 5 — ColdLedger:*
```python
class ColdLedger:
    def __init__(self):
        self.pool = psycopg2.pool.ThreadedConnectionPool(2, 10, **PostgresConfig.get_connection_params())
    def write_event(self, session_id, event_type, payload) -> str:
        event_id = str(uuid.uuid4())
        conn = self.pool.getconn()
        try:
            with conn.cursor() as cur:
                cur.execute("INSERT INTO events (id, session_id, event_type, payload) VALUES (%s, %s, %s, %s)", ...)
            conn.commit()
        finally:
            self.pool.putconn(conn)
        return event_id
```

*Module 6 — StateDelta (frozen dataclass):*
```python
@dataclass(frozen=True)
class StateDelta:
    agent_id: str
    session_id: str
    version_at_read: int
    patch: tuple
    submitted_at: datetime
    def apply_to(self, state: dict) -> dict:
        return jsonpatch.apply_patch(state, list(self.patch))
```

*Module 6 — RedisMasterState OCC:*
```python
async def commit_patch(self, session_id, version, patch) -> CommitResult:
    key = f"{self.KEY_PREFIX}{session_id}"
    async with self.redis.pipeline(transaction=True) as pipe:
        try:
            await pipe.watch(key)
            current = json.loads(await pipe.get(key) or '{"version":0,"data":{}}')
            if current["version"] != version:
                return CommitResult(False, current["version"], conflict=True)
            new_data = jsonpatch.apply_patch(current["data"], patch)
            pipe.multi()
            await pipe.set(key, json.dumps({"version": version+1, "data": new_data}))
            await pipe.execute()
            return CommitResult(True, version+1, conflict=False)
        except redis.WatchError:
            return CommitResult(False, -1, conflict=True)
```

*Module 6 — SwarmWorkerBase (XREADGROUP):*
```python
class SwarmWorkerBase(ABC):
    PEL_TIMEOUT_MS = 60000
    @abstractmethod
    async def process(self, task: dict) -> StateDelta: ...
    async def run_worker_loop(self, group, consumer_id) -> None:
        self._running = True
        while self._running:
            entries = await self.redis.xreadgroup(group, consumer_id, {"genesis:swarm:tasks": ">"}, count=1, block=5000)
            if entries:
                for entry_id, fields in entries[0][1]:
                    try:
                        delta = await self.process(task)
                        await self.staging.submit_delta(delta)
                        await self.redis.xack("genesis:swarm:tasks", group, entry_id)
                    except Exception:
                        pass  # No XACK → re-queues after PEL timeout
```

*Module 9 — EpochScheduler:*
```python
class EpochScheduler:
    def start(self) -> None:
        self.scheduler.add_job(
            self.runner.run_epoch_safe, 'cron',
            day_of_week='sun', hour=16, minute=0, id='nightly_epoch'
        )
        self.scheduler.start()
```

*Module 9 — RedisEpochLock:*
```python
EPOCH_LOCK_KEY = "epoch:lock:nightly"
EPOCH_LOCK_TTL = 7200

class RedisEpochLock:
    async def acquire(self, epoch_id: str) -> bool:
        value = f"{epoch_id}:{socket.gethostname()}:{os.getpid()}"
        result = await self.redis.set(EPOCH_LOCK_KEY, value, nx=True, ex=EPOCH_LOCK_TTL)
        return result is not None
```

*Module 9 — Distillation Prompt:*
```python
DISTILLATION_PROMPT = """
You are the Genesis Memory Distiller. Review this week's Genesis execution data...
Return ONLY valid JSON:
{"axioms": [{"id": "epoch_{date}_001", "content": "...", "category": "...", "confidence": 0.9, "source_saga_ids": [...]}],
 "week_summary": "2-sentence summary"}
"""
```

**`E:\genesis-system\docs\plans\AIVA_RLM_NEXUS_PRD_V2.md`** (modified in prior session — 3222 lines, COMPLETE)
- Completed in prior session — all 9 modules fully specified (59 stories total)
- Referenced as "Track A" complement to this PRD

---

4. Errors and Fixes:

- **No errors in this session** — all file writes succeeded. The Desktop Commander tool warned about large append operations (>30 lines recommendation) but all writes completed successfully.
- The prior session had a large edit warning (1962 lines) on AIVA_RLM_NEXUS_PRD_V2.md that still succeeded.
- Prior session also had Bash shell parsing issues with pipe characters — fixed by using Desktop Commander's list_directory and get_file_info tools instead.

---

5. Problem Solving:

**Solved**: Kinan's mid-task directive about 1000-agent swarm was immediately incorporated into the PRD's opening section as a hardwired execution mandate, including the 4-Wave parallel execution plan showing exactly how modules fire simultaneously.

**Solved**: The PRD was written in chunks to avoid large single writes, using `mode='rewrite'` for the first chunk and `mode='append'` for all subsequent chunks.

**Ongoing**: The PRD writing task is essentially complete (all 9 modules written, 73 stories). A closing section (verification checklist, component interface contracts, execution manifest) may still be needed to formally complete the PRD document.

---

6. All User Messages:

- **Mid-session message (while PRD was being written)**: "can you put somehwere that this will be a 1000 agent swarm operation? I need this completed today ! non-negotiable 1000 agent swarm with all teams firing. hive delegation in action!"

*(All prior session messages were captured in the compacted summary — the concurrent question about 7 DT architecture, and the requests to complete the AIVA PRD and write Track B PRD)*

---

7. Pending Tasks:

- **Complete the Genesis Persistent Context Architecture PRD closing section**: The 9 modules are written (73 stories). The PRD may need a formal closing section with: Component Interface Contracts (boundaries between A/B/C/D/E), Global Verification Checklist, Execution Manifest linking all 73 stories to their wave/order, and possibly a CLAUDE.md-style quick reference for agents executing this PRD.
- **Execute Track B Wave 1**: Once PRD is complete and confirmed, fire Wave 1 Ralph Wiggum Loops for Modules 1, 2, and 3 simultaneously (BaseInterceptor + JIT Hydration + MVFL) — each module gets its own fresh context window session.
- **George's Cairns phone number**: Still pending — check Telnyx portal for provisioning status of Request ID `c6c5e6a4-286d-4df7-ae1f-a17e2e44a7a3`, then assign to George's assistant.
- **Execute AIVA RLM Nexus PRD v2 Wave 1**: Track A Module 1 (Memory Schema) and Module 2 (Registry Layer) can begin — they only need Redis/Postgres, not the full interceptor chain.

---

8. Current Work:

Immediately before this summary request, the assistant had been writing the final module (Module 9 — Nightly Epoch) of the Genesis Persistent Context Architecture PRD. The last append written was:

**File**: `E:\genesis-system\docs\plans\GENESIS_PERSISTENT_CONTEXT_ARCHITECTURE_PRD.md`
**Last operation**: Appended Module 9 (Stories 9.01–9.10, 325 lines) in `mode='append'`

The last story written was **Story 9.10: Test Suite — Module 9 Integration (Full Epoch Run)**:
```
### Story 9.10: Test Suite — Module 9 Integration (Full Epoch Run)
File: tests/epoch/test_epoch_integration.py

Acceptance Criteria:
- Full epoch run with mocked LLM + real Redis + real Postgres (test DB)
- Verifies all 7 phases execute in order
- Verifies KG file updated post-run
- Verifies epoch log entry in epoch_log.jsonl
- Verifies lock released after epoch completes
- 10+ test cases

Budget: 7 implement + 1 run + 2 fix + 1 re-run + 1 stamp = 12 iterations
```

The PRD now covers all 9 modules and 73 atomic stories:
- Module 1: 7 stories (BaseInterceptor + dispatch_to_swarm)
- Module 2: 8 stories (JIT Hydration + Zero-Amnesia)
- Module 3: 7 stories (MVFL + Voyager Defense)
- Module 4: 7 stories (80/20 Tier Router + Strangler Fig)
- Module 5: 9 stories (CQRS + Event Sourcing + Postgres)
- Module 6: 9 stories (Multi-Agent Coherence + Redis Streams + OCC)
- Module 7: 7 stories (SemanticMergeInterceptor + Bulkheads)
- Module 8: 9 stories (Dual-Loop + Shadow Arena)
- Module 9: 10 stories (Nightly Epoch)

---

9. Optional Next Step:

Write the closing/footer section of the PRD to formally complete it — including: Global Verification Checklist, Component Interface Contract summary, and the Ralph Wiggum Loop execution manifest. This directly continues the in-progress task of "Write Genesis Persistent Context Architecture v2 Epic PRD (Track B)."

Direct quote from the todo list confirming what was being worked on:
> `{"content": "Write Genesis Persistent Context Architecture v2 Epic PRD (Track B)", "status": "in_progress", "activeForm": "Writing Genesis Persistent Context Epic PRD"}`

And from Kinan's directive that was incorporated into the PRD header:
> "can you put somehwere that this will be a 1000 agent swarm operation? I need this completed today ! non-negotiable 1000 agent swarm with all teams firing. hive delegation in action!"

The next step is to write the PRD closing section and mark the task complete.

If you need specific details from before compaction (like exact code snippets, error messages, or content you generated), read the full transcript at: C:\Users\P3\.claude\projects\E--genesis-system--claude-worktrees-eloquent-austin\ba014860-73a9-4c7e-97c0-b782fc8fb989.jsonl
Please continue the conversation from where we left off without asking the user any further questions. Continue with the last task that you were asked to work on.


TodoWrite

Answer Kinan's question: 7 DT architecture is SEPARATE from AIVA PRD

Answering question about DT architecture separation

Complete AIVA RLM Nexus PRD v2 — write full specs for Modules 4-9 (42 stories)

Writing Modules 4-9 full atomic story specs

Write Genesis Persistent Context Architecture v2 Epic PRD (Track B)

Writing Genesis Persistent Context Epic PRD closing section


Todos have been modified successfully. Ensure that you continue to use the todo list to track your progress. Please proceed with the current tasks if applicable

Answer Kinan's question: 7 DT architecture is SEPARATE from AIVA PRD

Answering question about DT architecture separation

Complete AIVA RLM Nexus PRD v2 — write full specs for Modules 4-9 (42 stories)

Writing Modules 4-9 full atomic story specs

Write Genesis Persistent Context Architecture v2 Epic PRD (Track B)

Writing Genesis Persistent Context Epic PRD closing section


PRD nearly complete — all 9 modules written (73 stories). Writing the closing section now: Component Interface Contracts, Global Verification Checklist, and Ralph Wiggum Loop Execution Manifest.


size: 130619
created: "2026-02-24T19:34:42.447Z"
modified: "2026-02-24T19:46:41.761Z"
accessed: "2026-02-24T19:49:37.697Z"
isDirectory: false
isFile: true
permissions: 666
fileType: text
lineCount: 2589
lastLine: 2588
appendPosition: 2589


2589 lines written. Now appending the closing section — Component Interface Contracts, Global Verification Checklist, and Execution Manifest.