Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import CsUiGuard from "@/entrypoints/contexts/content-scripts/services/ui-guard/CsUiGuard";
import { csUiMount } from "@/entrypoints/contexts/content-scripts/ui-groups/_root/CsUiRoot";
import { ThreadMessageFooterSecondaryComponentRegister } from "@/entrypoints/contexts/content-scripts/ui-groups/routes/Thread/message-footer-secondary/Group";

import TtsDownloadButton from "./components/TtsDownloadButton";

function TtsDownloadButtonWrapper() {
return (
<CsUiGuard requiresLoggedIn dependentPluginIds={["thread:ttsDownload"]}>
<ThreadMessageFooterSecondaryComponentRegister id="plugin:thread:ttsDownload">
<TtsDownloadButton />
</ThreadMessageFooterSecondaryComponentRegister>
</CsUiGuard>
);
}

TtsDownloadButtonWrapper.displayName = "TtsDownloadButtonWrapper";

export default function () {
csUiMount({
id: "plugin:thread:ttsDownload:messageFooter",
component: <TtsDownloadButtonWrapper />,
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import CsUiGuard from "@/entrypoints/contexts/content-scripts/services/ui-guard/CsUiGuard";
import { csUiMount } from "@/entrypoints/contexts/content-scripts/ui-groups/_root/CsUiRoot";
import { ThreadNavbarAttributesComponentRegister } from "@/entrypoints/contexts/content-scripts/ui-groups/routes/Thread/navbar-attributes/Group";

import ThreadTtsDownloadButton from "./components/ThreadTtsDownloadButton";

function ThreadTtsDownloadButtonWrapper() {
return (
<CsUiGuard
location={["thread"]}
dependentPluginIds={["thread:ttsDownload"]}
>
<ThreadNavbarAttributesComponentRegister id="plugin:thread:ttsDownload">
<ThreadTtsDownloadButton />
</ThreadNavbarAttributesComponentRegister>
</CsUiGuard>
);
}

ThreadTtsDownloadButtonWrapper.displayName = "ThreadTtsDownloadButtonWrapper";

export default function () {
csUiMount({
id: "plugin:thread:ttsDownload:navbar",
component: <ThreadTtsDownloadButtonWrapper />,
});
}
124 changes: 124 additions & 0 deletions perplexity/extension/src/plugins/_thread/tts-download/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# TTS Download Plugin

A Complexity plugin that enables downloading Perplexity AI messages and conversations as audio files using text-to-speech.

## Features

- **Individual Message Download**: Download any assistant message as a WAV audio file
- **Full Conversation Download**: Download all messages in a thread as a single concatenated audio file
- **Multiple Voice Options**: Choose from 4 TTS voices (Mike, Alex, Kate, Mary)
- **Progress Tracking**: Visual progress indicator when downloading multi-message conversations
- **Seamless Integration**: Download buttons in message action bars and navbar

## How It Works

### Architecture

The plugin uses Perplexity's native `voice_over` WebSocket API to generate TTS audio:

1. **WebSocket Connection**: Establishes connection via `InternalWebSocketManager`
2. **Audio Streaming**: Receives audio as Int16Array chunks via `audio` events
3. **Chunk Collection**: `AudioBufferCollector` accumulates chunks in memory
4. **WAV Encoding**: `WavEncoder` converts PCM chunks to WAV format with proper RIFF headers
5. **File Download**: Uses File System Access API (Chrome) or blob download (fallback)

### Key Components

#### Components
- **`TtsDownloadButton.tsx`**: Individual message download button (in message action bar)
- **`ThreadTtsDownloadButton.tsx`**: Full conversation download button (in navbar)
- **`VoiceSelectionDialog.tsx`**: Voice picker dialog for conversation downloads

#### Hooks
- **`useTtsDownloadRequest.ts`**: Core hook managing WebSocket communication and audio streaming

#### Utilities
- **`AudioBufferCollector`**: Collects and manages audio chunks
- **`WavEncoder`**: Encodes PCM data to WAV format
- **`download-wav.ts`**: Handles WAV file downloads

### Audio Format

- **Sample Rate**: 48kHz
- **Bit Depth**: 16-bit
- **Channels**: Mono (1 channel)
- **Format**: WAV (RIFF header)

## Usage

### Individual Message Download

1. Navigate to any Perplexity thread
2. Hover over an assistant message
3. Click the download icon in the action bar
4. Audio file downloads automatically with default voice

### Full Conversation Download

1. Click the download button in the navbar
2. Select desired voice from the dialog
3. Wait for progress to complete
4. Audio file saves with timestamp in filename

## Settings

- **Default Voice**: Select preferred voice for single-message downloads (Mike, Alex, Kate, Mary)
- **Enabled**: Toggle plugin on/off

## Technical Details

### Dependencies

- `domObservers:thread:messageBlocks` - Required for accessing message content

### WebSocket Protocol

```typescript
// Request
socket.emitWithAck("voice_over", {
is_page: false,
version: "2.13",
completed: true,
uuid: backendUuid,
preset: voice,
});

// Response
socket.on("audio", (packet: { data: ArrayLike<number>; uuid: string }) => {
// Handle audio chunk
});
```

### File Naming

- **Single message**: `message-{uuid.slice(0, 8)}-{voice}.wav`
- **Conversation**: `conversation-{timestamp}-{voice}.wav`

### Error Handling

- WebSocket connection failures
- Missing audio data
- User cancellation (File System Access API)
- Individual message failures in batch downloads (continues with next message)

## Changes to Shared Utilities

### `download-file.ts`

Enhanced to support binary audio data:

- Added `ArrayBuffer` support for `data` parameter
- Added `audio/wav` and `audio/wave` MIME types
- Added `useFilePicker` option for explicit File System Access API control
- Improved error handling to propagate user cancellation

## Future Enhancements

Potential improvements:

- Support for additional audio formats (MP3, OGG)
- Voice selection for individual messages
- Audio quality settings (sample rate, bit depth)
- Batch download with ZIP archive
- Pause/resume for long conversations
- Audio preview before download
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import Tooltip from "@/components/Tooltip";
import { Button } from "@/components/ui/button";
import { useToast } from "@/components/ui/use-toast";
import { useThreadDomObserverStore } from "@/entrypoints/contexts/content-scripts/core-plugins/dom-observers/thread/store";
import { useThreadMessageBlocksDomObserverStore } from "@/entrypoints/contexts/content-scripts/core-plugins/dom-observers/thread/message-blocks/store";
import useTtsDownloadRequest from "@/plugins/_thread/tts-download/hooks/useTtsDownloadRequest";
import type { TtsVoice } from "@/plugins/_thread/tts-download/types";
import { downloadWavFile } from "@/plugins/_thread/tts-download/utils/download-wav";

import TablerDownload from "~icons/tabler/download";
import TablerLoaderCircle from "~icons/tabler/loader-2";

import VoiceSelectionDialog from "./VoiceSelectionDialog";

export default function ThreadTtsDownloadButton() {
const [dialogOpen, setDialogOpen] = useState(false);
const [isDownloading, setIsDownloading] = useState(false);
const [progress, setProgress] = useState({ current: 0, total: 0 });
const { toast } = useToast();

const isThreadInFlight = useThreadDomObserverStore(
(store) => store.states.isInFlight,
);

const messageBlocks = useThreadMessageBlocksDomObserverStore(
(store) => store.messageBlocks,
);

const { downloadTts } = useTtsDownloadRequest({});

const handleDownload = async (voice: TtsVoice) => {
// Get all assistant messages (not in-flight, has answer content)
const assistantMessages =
messageBlocks?.filter(
(block) =>
!block.states.isInFlight &&
block.content.answer &&
block.content.backendUuid,
) ?? [];

if (assistantMessages.length === 0) {
toast({
title: "No Messages",
description: "No assistant messages to download",
variant: "caution",
});
setDialogOpen(false);
return;
}

if (assistantMessages.length > 20) {
toast({
title: "Warning",
description: `Downloading ${assistantMessages.length} messages may take a while`,
});
}

setIsDownloading(true);
setProgress({ current: 0, total: assistantMessages.length });

const allChunks: Int16Array[] = [];

try {
// Download each message sequentially
for (let i = 0; i < assistantMessages.length; i++) {
const message = assistantMessages[i];
if (!message) continue;

setProgress({ current: i + 1, total: assistantMessages.length });

try {
const chunks = await downloadTts({
voice,
backendUuid: message.content.backendUuid,
});

if (chunks.length > 0) {
allChunks.push(...chunks);
}
} catch (error) {
console.error(
`[TtsDownload] Failed to download message ${i + 1}:`,
error,
);
// Continue with next message
}
}

if (allChunks.length === 0) {
toast({
title: "Download Failed",
description: "No audio data received",
variant: "caution",
});
setDialogOpen(false);
return;
}

// Download concatenated audio
const timestamp = new Date()
.toISOString()
.slice(0, 19)
.replace(/:/g, "-");
const filename = `conversation-${timestamp}-${voice}`;

await downloadWavFile({
chunks: allChunks,
filename,
});

toast({
title: "Download Complete",
description: `Downloaded ${assistantMessages.length} messages as audio`,
});

setDialogOpen(false);
} catch (error) {
console.error("[TtsDownload] Conversation download failed:", error);
// Don't show error toast if user just cancelled the save dialog
if (error instanceof Error && error.message === "Download cancelled") {
return;
}
toast({
title: "Download Failed",
description: "Failed to download conversation",
variant: "caution",
});
} finally {
setIsDownloading(false);
setProgress({ current: 0, total: 0 });
}
};

return (
<>
<Tooltip content="Download conversation as audio">
<Button
onClick={() => setDialogOpen(true)}
disabled={isThreadInFlight || isDownloading}
variant="ghost"
size="sm"
className="x:box-content x:h-8 x:px-2.5 x:text-muted-foreground"
>
{isDownloading ? (
<TablerLoaderCircle className="x:size-4 x:animate-spin" />
) : (
<TablerDownload className="x:size-4" />
)}
</Button>
</Tooltip>

<VoiceSelectionDialog
open={dialogOpen}
onOpenChange={setDialogOpen}
onDownload={handleDownload}
isDownloading={isDownloading}
progress={isDownloading ? progress : undefined}
/>
</>
);
}
Loading