# Gemini Deep Think Result

**Model**: gemini-2.5-pro
**Elapsed**: 74.98s
**Prompt**: prompts/DT17_CONDENSED.md

## Response

Of course. This is an excellent and deeply practical problem space. Building an "Audio Doctor" is a high-leverage investment that can transform the user experience. Here is a comprehensive breakdown of the design and architecture, addressing all ten questions.

---

## The Audio Doctor: A Deep Dive

This document outlines the architecture, algorithms, and implementation strategy for the "Audio Doctor," a universal audio diagnostic and repair engine for Genesis Voice Products.

### 1. Browser-Based Diagnostic Decision Tree

This flow executes when the user initiates a voice session (e.g., clicks "Start Call"). It's designed to be fast, non-intrusive, and progressively more detailed as problems are detected.

**Status Codes:**
*   **GREEN:** All good. Stream active and healthy.
*   **YELLOW:** Minor issue detected (e.g., low volume). Potentially self-repaired. Non-blocking.
*   **ORANGE:** Action required. A solvable problem was found (e.g., permission needed, wrong device). Blocking.
*   **RED:** Critical failure. An unsolvable problem (e.g., no devices, hardware error). Blocking.

**Decision Flow (with pseudocode):**

```javascript
async function diagnoseAudio(): Promise<DiagnosisReport> {
  // 1. API Availability Check
  if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
    return { status: 'RED', code: 'API_UNSUPPORTED', message: 'Your browser does not support voice input.' };
  }

  // 2. Permission State Query (Non-intrusive)
  const perm = await navigator.permissions.query({ name: 'microphone' });
  if (perm.state === 'denied') {
    return { status: 'RED', code: 'PERMISSION_DENIED_HARD', message: 'Microphone access is permanently blocked.' };
  }

  let stream = null;
  try {
    // 3. Device Enumeration & Initial getUserMedia Request
    const devices = await navigator.mediaDevices.enumerateDevices();
    const audioDevices = devices.filter(d => d.kind === 'audioinput');

    if (audioDevices.length === 0) {
      return { status: 'RED', code: 'NO_DEVICES_FOUND', message: 'No microphone was found on your system.' };
    }

    // Attempt to get the stream. This will trigger the prompt if perm.state is 'prompt'.
    stream = await navigator.mediaDevices.getUserMedia({ audio: true });

    // 4. Stream Validation via AudioContext
    const audioContext = new AudioContext();
    const source = audioContext.createMediaStreamSource(stream);
    const analyser = audioContext.createAnalyser();
    analyser.fftSize = 256;
    source.connect(analyser);

    const dataArray = new Uint8Array(analyser.frequencyBinCount);

    // Wait a brief moment for the stream to stabilize
    await new Promise(resolve => setTimeout(resolve, 250)); 
    analyser.getByteFrequencyData(dataArray);

    const rms = dataArray.reduce((acc, val) => acc + val * val, 0) / dataArray.length;
    if (rms < 1) { // Threshold for silence
      // Stop the silent stream before returning
      stream.getTracks().forEach(track => track.stop());
      return { status: 'ORANGE', code: 'MIC_MUTED_OR_SILENT', message: 'Your microphone is connected but we are not receiving any audio.' };
    }

    // 5. Signal Quality Checks (Simplified for initial diagnosis)
    // More detailed checks happen in real-time monitoring (Section 4)
    const averageVolume = dataArray.reduce((sum, value) => sum + value) / dataArray.length;
    if (averageVolume < 10) { // Arbitrary low volume threshold
        return { status: 'YELLOW', stream, code: 'LOW_VOLUME', message: 'Your volume seems low. You may need to speak up or adjust your settings.' };
    }

    // Success!
    return { status: 'GREEN', stream, code: 'OK', message: 'Audio check passed.' };

  } catch (err) {
    // 6. Error Code Mapping
    if (stream) stream.getTracks().forEach(track => track.stop());
    
    switch (err.name) {
      case 'NotAllowedError': // User denied permission at the prompt
        return { status: 'RED', code: 'PERMISSION_DENIED_SOFT', message: 'You denied microphone access.' };
      case 'NotFoundError': // No device matches constraints (or no devices at all)
        return { status: 'RED', code: 'NO_DEVICES_FOUND', message: 'No microphone was found.' };
      case 'NotReadableError': // Hardware/OS/Driver error
        return { status: 'RED', code: 'HARDWARE_ERROR', message: 'Your microphone is being used by another application or there is a hardware error.' };
      case 'OverconstrainedError': // Constraints cannot be met
        return { status: 'ORANGE', code: 'CONSTRAINTS_ERROR', message: 'Could not find a microphone that meets the requirements.' };
      default:
        return { status: 'RED', code: 'UNKNOWN_ERROR', message: `An unknown error occurred: ${err.name}` };
    }
  }
}
```

### 2. Intelligent Device Selection

When `enumerateDevices()` returns multiple devices, we need a robust scoring algorithm to pick the best one.

**Scoring Heuristics (applied in order):**

1.  **User Preference (Weight: 1000):** If the user has previously selected a device in our application, and that device is still available (`deviceId` is stored in `localStorage`), select it. This is the highest priority.
2.  **Device Type Priority (Weight: 100-500):** We use pattern matching on the device `label` to infer its type.
    *   `Bluetooth` / `Headset` / `AirPods` / `Jabra` / `Poly`: **500**
    *   `USB` / `Yeti` / `HyperX`: **400**
    *   `Webcam` / `C920` / `Brio`: **200**
    *   `Built-in` / `Internal` / `Realtek` / `Array`: **100**
3.  **Windows Default Device Handling (Weight: 50):** On Windows, the `default` device is often the system-wide default, but applications like Teams or Discord use the "Default Communications Device". The browser's `default` `deviceId` usually maps to the communications device. We give a moderate boost to the device with `deviceId === 'default'`.
4.  **Active Signal Score (Tie-breaker):** If multiple devices have the same top score, we can momentarily request streams from the top 2-3 candidates, measure their RMS amplitude, and select the one with the strongest signal. This is more intensive and should be used sparingly.
5.  **Recency (Minor Tie-breaker):** If a device was used successfully in the last session, it gets a small bonus.

**Fallback Chain (Mid-call Failure):**
Triggered by `devicechange` event or real-time monitor detecting a dead stream.
1.  **Attempt Reconnect:** Try `getUserMedia` with the same `deviceId`. It might be a transient driver glitch.
2.  **Switch to Next Best:** If reconnection fails, run the selection algorithm again, excluding the failed `deviceId`.
3.  **Notify User:** Display a non-blocking notification: "Your microphone disconnected. We've switched you to [New Device Name]. [Click here to change]."
4.  **Degrade to Listen-Only:** If no other input devices are found, transition the user to a listen-only mode.
5.  **Terminate:** If audio output also fails, gracefully end the session with a diagnostic report.

### 3. Bluetooth Audio Profile Management

Direct control of A2DP/HFP profiles from the browser is impossible. Our strategy is **detection and mitigation**.

**Bluetooth Health Monitor:**

1.  **Detection:** Identify a device as Bluetooth using label keywords (`Bluetooth`, `AirPods`, etc.). This flags it for special monitoring.
2.  **Latency Tracking:**
    *   Use `audioContext.outputLatency` and `audioContext.baseLatency`. A sudden spike from ~50ms to ~250ms can indicate a profile switch.
    *   If using WebRTC, monitor `RTCRtpSender.getStats()` for `roundTripTime`.
3.  **Dropout Detection:**
    *   In the real-time monitor, a "silence" flag that persists for >1 second after a period of normal audio is a strong indicator of a profile switch hang or dropout.
4.  **Signal Strength Proxy (WebRTC only):**
    *   Monitor `RTCIceCandidatePair` stats for `bytesReceived` and `bytesSent`. A drop-off indicates a connectivity problem.
    *   Monitor `RTCInboundRtpStreamStats` for `packetsLost` and `jitter`. Spikes correlate with poor Bluetooth signal.

**Auto-Recovery Strategies:**

*   **On Profile Switch Hang (>3s silence):**
    1.  Show a non-blocking UI: "Reconnecting your Bluetooth headset..."
    2.  Programmatically stop the current stream's tracks.
    3.  Request a new stream using `getUserMedia` with the same `deviceId`. This can often "kick" the OS/device into finalizing the connection.
*   **On High Jitter/Packet Loss:**
    1.  Display a warning: "Your Bluetooth connection appears unstable."
    2.  Suggest user moves closer to their computer or removes sources of interference.

### 4. Real-Time Audio Health Monitoring

This runs continuously during the call using a `ScriptProcessorNode` or `AudioWorklet` for performance.

**Metrics & Thresholds:**

| Metric | Description | Normal (GREEN) | Warning (YELLOW) | Critical (RED) | Auto-Recovery Action |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **RMS Amplitude** | Average volume | 20-80 | < 20 or > 80 | < 5 or > 95 | **Yellow:** "You sound quiet/loud." **Red:** Auto-gain (if possible), else "Your mic is silent/distorted." |
| **Clipping Rate** | % of samples at max/min value | < 0.1% | 0.1% - 1% | > 1% | **Yellow/Red:** "You sound distorted. Try moving away from the mic or lowering gain." |
| **Silence %** | % of audio buffer with near-zero RMS | < 70% | 70% - 95% | > 95% (for >3s) | **Red:** Trigger full diagnostic for muted/disconnected mic. |
| **Packet Loss (WebRTC)** | % of lost packets | < 1% | 1% - 5% | > 5% | **Yellow:** "Your network is unstable." **Red:** Degrade audio quality (codec), suggest switching network. |
| **Jitter (WebRTC)** | Variation in packet arrival time | < 30ms | 30ms - 80ms | > 80ms | Same as Packet Loss. |

**3-Level Auto-Recovery:**

1.  **Soft Recovery (Internal):** The system tries to fix it without user knowledge. Examples: adjusting codec bitrate, enabling WebRTC's automatic gain control.
2.  **Moderate Recovery (Guidance):** The system displays a non-blocking toast notification with a clear, actionable suggestion. Example: "You sound a bit quiet. Try moving closer to your microphone."
3.  **Hard Recovery (Intervention):** The system detects a critical failure. It shows a modal, pauses the session if necessary, and runs a mini-diagnostic or guides the user through a specific fix (e.g., "It looks like your headset disconnected. Please reconnect it.").

### 5. Cross-Platform Diagnostic Engine

The core logic remains the same, but it's wrapped in a platform-aware abstraction layer.

**Universal API:**

```typescript
interface DiagnosisOptions {
  runSignalTest?: boolean;
  targetDeviceId?: string;
}

interface DiagnosisReport {
  status: 'GREEN' | 'YELLOW' | 'ORANGE' | 'RED';
  code: string; // e.g., 'PERMISSION_DENIED'
  message: string; // User-facing message
  data?: any; // e.g., device list, metrics
  stream?: MediaStream;
}

class AudioDoctor {
  static diagnose(options: DiagnosisOptions): Promise<DiagnosisReport>;
  static getPlatformAdapter(): PlatformAdapter;
}
```

**Platform-Specific Adaptations (`PlatformAdapter`):**

*   **iOS (Safari):**
    *   `enumerateDevices()` only returns device IDs after `getUserMedia` is called and a stream is active. Labels are generic ("Front Microphone").
    *   **Adaptation:** The diagnostic flow must first get a stream, *then* enumerate devices. We cannot perform intelligent selection upfront.
*   **Android (Chrome):**
    *   Aggressive battery optimization can throttle or kill background tab audio processing.
    *   **Adaptation:** The real-time monitor must check `document.visibilityState`. If `hidden`, it should alert the user upon return if the audio stream was lost.
*   **Windows:**
    *   Three-level default devices (System Default, Communications Default, Per-App Default).
    *   **Adaptation:** The device selection heuristics must heavily weight the `default` deviceId from the browser, as it usually correctly maps to the Communications device. The UI should explain this nuance if the user tries to select a different device and it doesn't work.
*   **macOS:**
    *   Safari has "sticky" per-site permissions that can be confusing.
    *   **Adaptation:** If `permission.state` is `denied`, the guided fix (Level 4 UI) must show screenshots of Safari's "Website Settings" panel.
*   **Linux (PulseAudio/PipeWire):**
    *   Virtual "loopback" and "monitor" devices are common, which can confuse selection.
    *   **Adaptation:** The device selection algorithm should de-prioritize devices with labels containing "Monitor of" or "Loopback".

### 6. Self-Healing Audio Pipeline

This is an event-driven system designed for resilience.

**Event Triggers & Listeners:**

*   `navigator.mediaDevices.ondevicechange`: Re-run device enumeration and check if the active device is still present. If not, trigger the fallback chain.
*   `stream.oninactive` / `track.onended`: The stream died for an unknown reason (e.g., driver crash, device unplugged). Trigger the fallback chain.
*   `document.addEventListener('visibilitychange')`: On becoming visible again, re-validate the audio stream.
*   `window.addEventListener('online' / 'offline')`: On network change, re-check WebRTC connection health.
*   **System Sleep/Wake:** Most browsers handle this, but on wake, we should run a quick health check (is RMS > 0?).

**Graceful Degradation Hierarchy:**

1.  **Level 5 (Full Duplex):** Normal operation. Real-time, two-way audio.
2.  **Level 4 (Half Duplex):** If high latency or packet loss is detected, switch to a "Push-to-Talk" mode. This reduces network strain and prevents users from talking over each other.
3.  **Level 3 (Playback Only):** If the microphone fails entirely but the speaker is fine, switch the user to a listen-only mode.
4.  **Level 2 (Text Only):** If all audio fails, degrade the experience to a real-time text chat interface within the same session.
5.  **Level 1 (Session End):** If all else fails or the user can't resolve the issue, end the session and automatically prompt to send a diagnostic report.

### 7. User-Facing Diagnostic UI/UX

The principle is **progressive disclosure**. We only show the user what they absolutely need to know.

*   **Level 1 (Silent Auto-Fix):** The most common case. Audio Doctor detects the wrong mic is selected (e.g., laptop mic instead of headset), switches it automatically, and the user never knows. (Target: 70-80% of issues).
*   **Level 2 (Gentle Toast):** A non-blocking notification for minor issues. "Your volume is a bit low." or "Switched to your Jabra headset." It disappears on its own.
*   **Level 3 (Blocking Modal with Checklist):** For solvable issues requiring user action (`ORANGE` status).
    *   **UI Elements:**
        *   Clear title: "Let's get your microphone working"
        *   Visual mic indicator (waveform/bar that reacts to sound).
        *   A simple dropdown to select a different microphone.
        *   A "Test Your Mic" button that plays back a short recording.
        *   A single, prominent button like "Retry" or "Continue".
*   **Level 4 (Guided Walkthrough):** For platform-specific issues (`RED` status like permission denied).
    *   The modal expands to show step-by-step instructions with screenshots/GIFs specific to the user's OS and Browser.
    *   Example: "1. Click the lock icon in your address bar. 2. Find 'Microphone' and set it to 'Allow'. 3. Click 'Reload' when prompted."
*   **Level 5 (Human Escalation):** If all else fails.
    *   A button appears: "Get Help".
    *   Clicking it generates a unique diagnostic ID and pre-fills a support request with the report.
    *   **Good Error Message:** "We're sorry, we couldn't connect to your microphone despite several checks. This could be a hardware or driver issue. Please share this Diagnostic ID with our support team: `[unique_id]`."
    *   **Bad Error Message:** "Error: NotReadableError."

### 8. Embeddable Widget Design

**Architecture:**

*   **Technology:** TypeScript, Preact/SolidJS for a tiny footprint, Vite for optimized bundling.
*   **Isolation:** Encapsulate the entire UI in a **Shadow DOM** to prevent CSS conflicts with the host page.
*   **Modularity (Tree-shakeable):**
    1.  `@genesis/audiov-doctor/core`: Contains the main `diagnose()` logic. (~8KB)
    2.  `@genesis/audio-doctor/analyzer`: Contains the `AudioContext` analysis tools. (~12KB)
    3.  `@genesis/audio-doctor/ui`: The Preact components for the modals and toasts. (~15KB)
    4.  `@genesis/audio-doctor/recovery`: Contains the self-healing/fallback logic. (~10KB)
    5.  `@genesis/audio-doctor/reporter`: The logic for packaging and sending diagnostic data. (~5KB)
*   **API:**
    ```javascript
    import { AudioDoctor } from '@genesis/audio-doctor';

    const doctor = new AudioDoctor({
      element: '#widget-container', // Where to mount the UI
      apiKey: 'YOUR_API_KEY', // For reporting
      theme: 'dark',
      onComplete: (report) => {
        if (report.status === 'GREEN') {
          // start the main application logic with report.stream
        }
      }
    });

    doctor.runPreflightCheck();
    ```
*   **Privacy:** No audio is ever recorded or sent to servers, only metadata and derived metrics (RMS, clipping, etc.). This must be clearly stated in the privacy policy.

### 9. Data Collection Pipeline

This is the key to creating the learning flywheel.

**Analytics Schema (per session):**

```json
{
  "sessionId": "uuid-v4",
  "timestamp": "iso-8601",
  "userId": "hashed-user-id",
  "platform": {
    "os": "Windows 11",
    "browser": "Chrome",
    "version": "108.0.0"
  },
  "initialState": {
    "permission": "prompt",
    "devices": [
      { "label": "Default - Microphone Array (Realtek(R) Audio)", "kind": "audioinput", "groupId": "grp1" },
      { "label": "Microphone (Logitech BRIO)", "kind": "audioinput", "groupId": "grp2" },
      { "label": "Headset Microphone (Jabra Evolve 20)", "kind": "audioinput", "groupId": "grp3" }
    ]
  },
  "diagnosticTrace": [
    { "step": "enumerateDevices", "status": "SUCCESS", "duration": 50 },
    { "step": "autoSelectDevice", "result": "Jabra Evolve 20", "reason": "TypePriority" },
    { "step": "getUserMedia", "status": "SUCCESS", "duration": 300 },
    { "step": "validateStream", "result": "MIC_MUTED_OR_SILENT", "rms": 0.5 }
  ],
  "userActions": [
    { "action": "showMutedModal" },
    { "action": "userUnmutesAndRetries" }
  ],
  "finalResolution": {
    "status": "GREEN",
    "code": "OK",
    "timeToResolve": 4500 // ms
  }
}
```

**Aggregation Queries (for product improvement):**

*   `SELECT device.label, COUNT(*) FROM events WHERE resolution.status = 'RED' GROUP BY device.label ORDER BY COUNT(*) DESC LIMIT 10;` -> **Find the most problematic devices.**
*   `SELECT platform.os, platform.browser, resolution.code, COUNT(*) FROM events GROUP BY 1, 2, 3;` -> **Find the most common errors per platform.**
*   `AVG(timeToResolve) WHERE resolution.status = 'GREEN'` -> **Measure the effectiveness of the tool.**

### 10. Integration with Genesis Products

The modular widget and unified API allow for tailored integration.

*   **Talking Widget (Customer-facing chat):**
    *   **Policy:** Zero-tolerance, aggressive recovery. The goal is instant success.
    *   **Integration:** Run a silent `diagnose()` on page load. If not GREEN, the "voice" button is disabled and replaced by a "Fix Mic" button that launches the full UI. Self-healing is extremely aggressive, immediately switching devices on failure.
*   **AIVA (Internal agent assistant):**
    *   **Policy:** Conservative recovery, detailed metrics. The user is more technical.
    *   **Integration:** A persistent "Audio Health" icon is visible. Green/Yellow/Red. Clicking it shows detailed real-time metrics (latency, jitter, etc.). Recovery is less aggressive, preferring to notify the agent ("Connection unstable, suggest switching to a wired headset") rather than auto-switching mid-conversation.
*   **ReceptionistAI (B2B phone system):**
    *   **Policy:** Mandatory pre-call check, business owner alerts. Reliability is paramount.
    *   **Integration:** Before a user can go "on duty," they must pass a mandatory Audio Doctor pre-flight check. The results (pass/fail, device used) are logged. If an agent has recurring audio issues (flagged by our data pipeline), an alert can be sent to the business owner/admin.

This comprehensive Audio Doctor system will directly address the #1 friction point, dramatically improve first-time user success rates, reduce support load, and create a powerful data-driven competitive advantage.
