# WebSocket_Hibernation_Patterns

**Generated**: 2026-02-26T14:12:58.988391
**Model**: gemini-3-flash-preview
**Tokens**: 1242
**Cost**: $0.0002

---

# WebSocket Hibernation Patterns for Real-Time AI Chat

In the era of LLMs, real-time feedback is non-negotiable. However, maintaining stateful WebSocket connections for thousands of concurrent users can be prohibitively expensive. Cloudflare’s **WebSocket Hibernation API** solves this by allowing Durable Objects (DO) to terminate their execution state (saving memory/CPU costs) while keeping the TCP connection alive at the edge.

This guide outlines the architectural patterns for building a production-grade AI chat interface using this technology.

---

## 1. WebSocket Hibernation API Reference

The Hibernation API moves the event handling from the DO instance memory to the Durable Object runtime itself.

*   **`state.acceptWebSocket(ws, tags)`**: Instead of `ws.accept()`, this hands the socket to the DO runtime. The DO can then safely shut down.
*   **`state.getWebSockets(tag?)`**: Retrieves a list of currently active or hibernated WebSockets.
*   **`ws.serializeAttachment(value)`**: Attaches metadata (e.g., `userId`, `chatId`) to the socket. This data persists even when the DO hibernates and is available when the DO wakes up.
*   **`ws.deserializeAttachment()`**: Retrieves the saved metadata.
*   **`webSocketMessage(ws, message)`**: The handler that wakes the DO when a client sends data.
*   **`webSocketClose/Error(ws, ...)`**: Handlers for connection termination.

---

## 2. Connection Lifecycle & Authentication

Authentication must happen during the initial HTTP Upgrade request. Once the DO "accepts" the socket via hibernation, it is harder to perform initial validation.

```typescript
export class ChatRoom implements DurableObject {
  constructor(private state: DurableObjectState) {}

  async fetch(request: Request) {
    const url = new URL(request.url);
    const token = url.searchParams.get("token");

    // 1. Authenticate before upgrading
    const user = await this.validateToken(token);
    if (!user) return new Response("Unauthorized", { status: 401 });

    const [client, server] = Object.values(new WebSocketPair());

    // 2. Accept and Hibernate
    // We attach the userId so we know who sent messages later
    this.state.acceptWebSocket(server, [user.id]);
    server.serializeAttachment({ userId: user.id, username: user.username });

    return new Response(null, { status: 101, webSocket: client });
  }

  async webSocketMessage(ws: WebSocket, message: string | ArrayBuffer) {
    const attachment = ws.deserializeAttachment();
    // Handle message...
  }
}
```

---

## 3. Cost Model: Hibernated vs. Active

| State | Billing Metric | Cost Impact |
| :--- | :--- | :--- |
| **Active** | Duration (GB-s) + Requests | Charged for every millisecond the DO is in memory. |
| **Hibernated** | Requests Only | **Zero duration charges.** You only pay for the initial "wake up" and the time spent processing a message. |

**The "AI Gap":** When waiting for a slow LLM response, the DO is **Active**. However, between user messages (the "thinking time" of the human), the DO **Hibernates**, saving ~90% of idle costs.

---

## 4. Streaming AI Responses over WebSocket

To provide a "typewriter" effect, we stream chunks from the AI provider (OpenAI, Anthropic, or Workers AI) directly to the WebSocket.

```typescript
async webSocketMessage(ws: WebSocket, message: string) {
  const { content } = JSON.parse(message);
  const attachment = ws.deserializeAttachment();

  // 1. Call AI (Streaming)
  const stream = await ai.run("@cf/meta/llama-3-8b-instruct", {
    prompt: content,
    stream: true,
  });

  // 2. Pipe chunks to the WebSocket
  for await (const chunk of stream) {
    ws.send(JSON.stringify({
      type: "chunk",
      text: chunk.response,
      userId: attachment.userId
    }));
  }
  
  ws.send(JSON.stringify({ type: "done" }));
}
```

---

## 5. Broadcasting Patterns (Multi-Device Sync)

If a user has three tabs open, they should all see the AI's response. We use `state.getWebSockets()` to broadcast to all connections tagged with the same `userId`.

```typescript
private broadcastToUser(userId: string, data: any) {
  const sockets = this.state.getWebSockets(userId);
  for (const ws of sockets) {
    try {
      ws.send(JSON.stringify(data));
    } catch (e) {
      // Handle broken connections
    }
  }
}
```

---

## 6. Heartbeats and Keepalive

Cloudflare terminates WebSockets after **lose-of-connectivity** or **long periods of inactivity**.
*   **Hibernation Strategy:** You do **not** need to implement `setInterval` pings inside the DO (which would keep it active and cost money).
*   **Client-Side Strategy:** The client should send a "ping" every 30-45 seconds. This wakes the DO momentarily (minimal cost) and prevents edge timeout.
*   **Auto-Close:** If a DO hasn't seen activity in the timeout window, Cloudflare closes the socket automatically.

---

## 7. PartyServer vs. Raw Durable Objects

| Feature | Raw Durable Objects | PartyServer (PartyKit) |
| :--- | :--- | :--- |
| **Complexity** | High (Manual lifecycle) | Low (Class-based hooks) |
| **Hibernation** | Manual `acceptWebSocket` | Built-in by default |
| **Broadcasting** | Manual `getWebSockets` | `this.room.broadcast()` |
| **Best For** | Bespoke high-perf systems | Rapid development & complex rooms |

---

## 8. Client-Side: Reconnection with `partysocket`

Standard WebSockets are brittle. Use `partysocket` (from the PartyKit team) for robust reconnection logic that handles the DO waking up.

```typescript
import PartySocket from "partysocket";

const socket = new PartySocket({
  host: "your-worker.dev",
  room: "chat-room-abc",
  params: async () => ({
    token: await getAuthToken(),
  }),
});

socket.addEventListener("message", (e) => {
  const data = JSON.parse(e.data);
  updateUIMessage(data);
});
```

---

## 9. Scaling and Concurrent Limits

*   **Max Connections:** A single Durable Object can handle up to **32,768 concurrent connections**.
*   **CPU/Memory:** While hibernated, memory usage is zero. However, when broadcasting a message to 30k users, the DO wakes up and must iterate through all sockets. This can hit the **128MB/256MB RAM limit**.
*   **Scaling Strategy:**
    *   **Room-based:** One DO per Chat Room (Standard).
    *   **Sharded:** If a "Global" chat exceeds 30k users, shard users across multiple DOs based on `userId` hash and use a "Pub/Sub" DO to bridge them.

---

## 10. Summary Checklist for Production

1.  **Use `serializeAttachment`** to store session data; avoid external DB lookups in the `webSocketMessage` handler.
2.  **Tag WebSockets** with `userId` to allow targeted broadcasting across devices.
3.  **Implement `webSocketError`** to clean up state or log disconnects.
4.  **Buffer AI chunks** if they are too small (e.g., send every 3-5 tokens) to reduce the number of WebSocket frames and UI re-renders.
5.  **Set a `reconnect` logic** on the client to handle the 24-hour max DO lifetime restart.