Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 159 additions & 12 deletions client-sdks/advanced/attachments.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ The **Attachment Table** is a local-only table that stores metadata about each f
**Metadata stored:**
- `id` - Unique attachment identifier (UUID)
- `filename` - File name with extension (e.g., `photo-123.jpg`)
- `localUri` - Path to file in local storage
- `localUri` - Reference to the file in local storage. The format is platform-specific: a file path on native platforms and Node.js, or an internal `indexeddb://` reference on web
- `size` - File size in bytes
- `mediaType` - MIME type (e.g., `image/jpeg`)
- `state` - Current sync state (see states above)
Expand All @@ -96,6 +96,8 @@ The **Remote Storage Adapter** is an interface you implement to connect PowerSyn
- `downloadFile(attachment)` - Download file from cloud storage
- `deleteFile(attachment)` - Delete file from cloud storage

In the JavaScript/TypeScript SDK, apps that transfer large files can replace this adapter with a streaming transport; see [Transferring Large Files Without Buffering](#transferring-large-files-without-buffering).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we calling it TypeScript SDK anywhere else?

Suggested change
In the JavaScript/TypeScript SDK, apps that transfer large files can replace this adapter with a streaming transport; see [Transferring Large Files Without Buffering](#transferring-large-files-without-buffering).
In the JavaScript SDK, apps that transfer large files can replace this adapter with a streaming transport; see [Transferring Large Files Without Buffering](#transferring-large-files-without-buffering).

This also still makes it sound like all other SDKs necessarily buffer the file, when that is not true for Kotlin and Dart.


**Common pattern:**
For security reasons, client-side implementations should use **signed URLs**
1. Request a signed upload/download URL from your backend
Expand All @@ -115,14 +117,16 @@ The **Local Storage Adapter** handles file persistence on the device. PowerSync
- `fileExists(path)` - Check if file exists
- `getLocalUri(filename)` - Get full path for a filename

In the JavaScript/TypeScript SDK, adapters that can relocate a file without loading it into memory implement the `StreamingLocalStorageAdapter` subinterface, which adds `moveFile(sourceUri, targetUri)`. Configuring the queue with a streaming-capable adapter enables [`saveFileFromUri`](#upload-an-attachment). The Node.js, Expo, and React Native FS adapters are streaming-capable; the web IndexedDB adapter is not.

**Built-in adapters:**
- **IndexedDB** - For web browsers (`IndexDBFileSystemStorageAdapter`)
- **Node.js Filesystem** - For Node/Electron (`NodeFileSystemAdapter`)
- **React Native** - For React Native with Expo or bare React Native we have a dedicated package [(`@powersync/attachments-storage-react-native`)](https://github.com/powersync-ja/powersync-js/tree/main/packages/attachments-storage-react-native)
- **Native mobile storage** - For Flutter, Kotlin, Swift

<Warning>
The React Native local storage adapter requires Expo 54 or later.
The React Native local storage adapter requires Expo 54 or later. The Expo streaming transport requires Expo 56 or later.
</Warning>

### Attachment Queue
Expand All @@ -135,6 +139,8 @@ The **Attachment Queue** is the orchestrator that manages the entire attachment
- **Performs cleanup** - Removes archived files that are no longer needed
- **Verifies integrity** - Checks local files exist and repairs inconsistencies

In the JavaScript/TypeScript SDK, remote transfers buffer each file through JS memory by default. Apps that handle large files can swap in a streaming transport adapter; see [Transferring Large Files Without Buffering](#transferring-large-files-without-buffering).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also only mention this once (it's also mentioned under attachment table, I don't think it belongs there).


**Watched Attachments pattern:**
The queue needs to know which attachments exist in your data model. The `watchAttachments` function you provide monitors your data model and returns a list of attachment IDs that your app references. The queue compares this list with its internal attachment table to determine:
- **New attachments** - Download them
Expand Down Expand Up @@ -653,6 +659,9 @@ const attachmentQueue = new AttachmentQueue({
db: db, // PowerSync database instance
localStorage,
remoteStorage,
// Or a transportAdapter in place of remoteStorage; see "Transferring
// Large Files Without Buffering" under Advanced Topics.
// transportAdapter,

// Define which attachments exist in your data model
watchAttachments: (onUpdate) => {
Expand Down Expand Up @@ -1359,6 +1368,24 @@ async function uploadProfilePhoto(imageBlob: Blob, userId: string) {
// 3. Update user record in same transaction
// 4. Automatically upload file in background
// 5. Update state to SYNCED when complete

// For files already on disk (e.g. a captured video or audio recording),
// saveFileFromUri queues the upload without reading the file into memory.
// Requires a streaming-capable local storage adapter (StreamingLocalStorageAdapter:
// Node.js, Expo, or React Native FS; not available on web).
async function attachRecording(localUri: string, recordingId: string) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add a second snippet for this (to have one with a blob and one streaming example) instead of combining it into one?

return attachmentQueue.saveFileFromUri({
localUri, // path to the existing file
fileExtension: 'm4a',
mediaType: 'audio/mp4',
updateHook: async (tx, attachment) => {
await tx.execute(
'UPDATE recordings SET audio_id = ? WHERE id = ?',
[attachment.id, recordingId]
);
}
});
}
```

```dart Flutter
Expand Down Expand Up @@ -1488,6 +1515,8 @@ The `updateHook` parameter is the recommended way to link attachments to your da
<CodeGroup>

```typescript JavaScript/TypeScript
import { AttachmentState } from '@powersync/web';

// Downloads happen automatically when watchAttachments references a file

async function getProfilePhotoUri(userId: string): Promise<string | null> {
Expand All @@ -1509,42 +1538,61 @@ async function getProfilePhotoUri(userId: string): Promise<string | null> {
return null;
}

if (attachment.state === 'SYNCED' && attachment.local_uri) {
if (attachment.state === AttachmentState.SYNCED && attachment.local_uri) {
return attachment.local_uri;
}

return null;
}

// Example: Display image in React with watch query
// Example: display the image in React on web. On web, local_uri is an
// internal indexeddb:// reference, so read the bytes through the local
// storage adapter and convert them to an object URL. On React Native and
// Node.js, local_uri is a real file path and can be used directly
// (e.g. <Image source={{ uri: localUri }} /> in React Native).
function ProfilePhoto({ userId }: { userId: string }) {
const [photoUri, setPhotoUri] = useState<string | null>(null);
const [photoUrl, setPhotoUrl] = useState<string | null>(null);

useEffect(() => {
let objectUrl: string | null = null;

const watch = db.watch(
`SELECT a.local_uri, a.state
`SELECT a.local_uri, a.media_type, a.state
FROM users u
LEFT JOIN attachments a ON a.id = u.photo_id
WHERE u.id = ?`,
[userId],
{
onResult: (result) => {
onResult: async (result) => {
const row = result.rows?._array[0];
if (row?.state === 'SYNCED' && row?.local_uri) {
setPhotoUri(row.local_uri);
if (row?.state === AttachmentState.SYNCED && row?.local_uri) {
const buffer = await localStorage.readFile(row.local_uri);
const nextUrl = URL.createObjectURL(
new Blob([buffer], { type: row.media_type ?? 'image/jpeg' })
);
if (objectUrl) {
URL.revokeObjectURL(objectUrl);
}
objectUrl = nextUrl;
setPhotoUrl(nextUrl);
}
}
}
);

return () => watch.close();
return () => {
watch.close();
if (objectUrl) {
URL.revokeObjectURL(objectUrl);
}
};
}, [userId]);

if (!photoUri) {
if (!photoUrl) {
return <div>Loading photo...</div>;
}

return <img src={photoUri} alt="Profile" />;
return <img src={photoUrl} alt="Profile" />;
}
```

Expand Down Expand Up @@ -1788,6 +1836,10 @@ internal sealed class PhotoState

</CodeGroup>

<Note>
Comment thread
simolus3 marked this conversation as resolved.
Web SDK only: `local_uri` is an internal `indexeddb://` reference rather than a URL the browser can load. Passing it directly to an `<img src>` fails with `net::ERR_UNKNOWN_URL_SCHEME`. Read the file through the local storage adapter and convert it to an object URL first, as shown in the JavaScript/TypeScript example above. Native SDKs return a real file path that can be used directly.
</Note>

### Delete an Attachment

<CodeGroup>
Expand Down Expand Up @@ -2151,6 +2203,64 @@ var queue = new AttachmentQueue(new AttachmentQueueOptions
```
</CodeGroup>

### Transferring Large Files Without Buffering
Comment thread
khawarizmus marked this conversation as resolved.

This section applies to the JavaScript/TypeScript SDK only. In the Dart and Kotlin SDKs, the remote storage interface is already stream-based (`Stream`/`Flow`), so transfers can avoid buffering. The Swift SDK currently receives files as `Data` and has no streaming equivalent yet.

By default, the queue transfers files by buffering them through JS memory: the entire file is read into an `ArrayBuffer` before it is handed to the remote storage adapter, and the reverse for downloads. This works well for small files but limits the practical attachment size, particularly in React Native, where a large video can exhaust the JS heap on lower-end devices.

To stream instead, configure the queue with a transport adapter in place of the remote storage adapter (you provide one or the other, not both; TypeScript enforces this). A transport owns all remote operations through three methods:

- `upload(attachment)` - Transfer the file at `attachment.localUri` to remote storage
- `download(attachment)` - Fetch the remote file into `attachment.localUri` (the queue assigns the destination path before the call)
- `delete(attachment)` - Remove the file from remote storage

The streaming-capable local storage adapters each create a ready-made transport through their `createTransportAdapter` method, so you don't implement these methods yourself:

- `ExpoFileSystemStorageAdapter` (`@powersync/attachments-storage-react-native`) - The transport streams with Expo's native `File.upload`/`File.downloadFileAsync`. Using the transport requires Expo 56 or later; using only the storage adapter requires Expo 54
- `ReactNativeFileSystemStorageAdapter` (`@powersync/attachments-storage-react-native`) - The transport streams with `uploadFiles`/`downloadFile` from `@dr.pogodin/react-native-fs`, uploading as a raw binary `PUT` by default
- `NodeFileSystemAdapter` (`@powersync/node`) - The transport streams with `fetch` and Node.js filesystem streams

All three take the same options. `resolveUpload` and `resolveDownload` map an attachment to the HTTP request that transfers its bytes, typically a signed URL from your backend. `deleteFile` performs the remote delete, which is a plain remote call rather than a byte transfer.

<Note>
The transport API requires `@powersync/web` v2.2.0, `@powersync/react-native` v2.0.3, or `@powersync/node` v0.21.0 or later. React Native also requires `@powersync/attachments-storage-react-native` v0.1.0 or later.
</Note>

```typescript
import { AttachmentQueue } from '@powersync/react-native';
import { ExpoFileSystemStorageAdapter } from '@powersync/attachments-storage-react-native';

const localStorage = new ExpoFileSystemStorageAdapter();

// Streams bytes natively and owns upload/download/delete. No remoteStorage needed.
const transportAdapter = localStorage.createTransportAdapter({
resolveUpload: async (attachment) => ({
url: await getSignedUploadUrl(attachment.filename), // from your backend
mimeType: attachment.mediaType ?? 'application/octet-stream'
}),
resolveDownload: async (attachment) => ({
url: await getSignedDownloadUrl(attachment.filename)
}),
deleteFile: async (attachment) => {
await deleteFromStorage(attachment.filename); // your backend or storage SDK call
}
});

const attachmentQueue = new AttachmentQueue({
db,
localStorage,
transportAdapter, // owns all remote operations; used in place of remoteStorage
watchAttachments: (onUpdate) => {
// Same as in Initialize Attachment Queue
}
});

await attachmentQueue.startSync();
```

For files your app produces on disk (camera captures, recordings, exports), combine a native transport with [`saveFileFromUri`](#upload-an-attachment). The file moves into managed storage and uploads without ever being read into memory; `saveFile` would read it into an `ArrayBuffer` just to write it back to disk.

### Custom Storage Adapters

The following is an example of how to implement a custom storage adapter for IPFS:
Expand Down Expand Up @@ -2355,6 +2465,43 @@ public sealed class IPFSStorageAdapter(HttpClient http) : IRemoteStorageAdapter

</CodeGroup>

### Custom Transport Adapters

In the JavaScript/TypeScript SDK, you can also implement [`AttachmentTransportAdapter`](#transferring-large-files-without-buffering) yourself. A custom remote storage adapter can already customize where bytes go, but it always receives the file as a full in-memory buffer. A transport works from the file's path instead, which enables:

- **Buffer-free transfers** - Let a native package transfer directly between the file system and the network, bypassing JS entirely, as the built-in transports do
- **Resumable transfers** - The queue retries a failed operation by calling the transport again on the next sync interval. A transport built on a resumable protocol such as [tus](https://tus.io) or S3 multipart upload can continue from the last confirmed offset instead of restarting from zero. Downloads can resume a partial file with HTTP `Range` requests
- **Encryption** - Encrypt files before upload and decrypt them after download for end-to-end encrypted attachments, without holding the whole file in memory

```typescript

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this empty snippet, linking to the relevant interface in our tsdoc reference might be better.

import {
AttachmentRecord,
AttachmentTransportAdapter,
LocatedAttachmentRecord
} from '@powersync/web';

class ResumableTransportAdapter implements AttachmentTransportAdapter {
async upload(attachment: LocatedAttachmentRecord): Promise<void> {
// attachment.localUri points at the source file. Transfer it to remote
// storage, e.g. in chunks that resume from the last confirmed offset
// if a previous attempt was interrupted.
}

async download(attachment: LocatedAttachmentRecord): Promise<void> {
// attachment.localUri is the destination path, assigned by the queue.
// Fetch the remote file into it.
}

async delete(attachment: AttachmentRecord): Promise<void> {
// Remove the file from remote storage.
}
}
Comment thread
khawarizmus marked this conversation as resolved.
```

For a working reference, see the built-in [`NodeFileSystemTransportAdapter`](https://github.com/powersync-ja/powersync-js/blob/main/packages/node/src/attachments/NodeFileSystemTransportAdapter.ts), which streams with `fetch` and Node.js filesystem streams.

Throwing from any method marks the operation as failed; the queue retries it on the next sync interval, subject to your [error handler](#error-handling).
Comment on lines +2502 to +2503

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is obvious enough that it doesn't need to be mentioned.

Suggested change
Throwing from any method marks the operation as failed; the queue retries it on the next sync interval, subject to your [error handler](#error-handling).


### Verification and Recovery

`verifyAttachments()` is always called internally during `startSync()`.
Expand Down