|
| 1 | +--- |
| 2 | +title: Modal |
| 3 | +description: Call deployed Modal functions and endpoints |
| 4 | +--- |
| 5 | + |
| 6 | +import { BlockInfoCard } from "@/components/ui/block-info-card" |
| 7 | + |
| 8 | +<BlockInfoCard |
| 9 | + type="modal" |
| 10 | + color="#000000" |
| 11 | +/> |
| 12 | + |
| 13 | +## Usage Instructions |
| 14 | + |
| 15 | +Integrate Modal into your workflow to reach the serverless compute you already run there. Invoke a deployed Web Function or Server over HTTPS with proxy-token auth, generate completions from a model served by a Modal Endpoint, and list the models a token can reach. |
| 16 | + |
| 17 | + |
| 18 | + |
| 19 | +## Actions |
| 20 | + |
| 21 | +### Modal Call Function |
| 22 | + |
| 23 | +Invoke a deployed Modal Web Function or Server over HTTPS |
| 24 | + |
| 25 | +#### Input |
| 26 | + |
| 27 | +| Parameter | Type | Required | Description | |
| 28 | +| --------- | ---- | -------- | ----------- | |
| 29 | +| `url` | string | Yes | Public URL of the deployed Modal Web Function or Server \(e.g., https://your-workspace--your-app-your-function.modal.run\) | |
| 30 | +| `method` | string | No | HTTP method to use: GET, POST, PUT, PATCH, DELETE, or HEAD | |
| 31 | +| `body` | json | No | JSON request body sent to the function | |
| 32 | +| `queryParams` | json | No | Query parameters to append to the URL as key-value pairs | |
| 33 | +| `headers` | json | No | Additional request headers as key-value pairs | |
| 34 | +| `tokenId` | string | No | Modal proxy token ID \(wk-...\), required for authenticated functions | |
| 35 | +| `tokenSecret` | string | No | Modal proxy token secret \(ws-...\), required for authenticated functions | |
| 36 | + |
| 37 | +#### Output |
| 38 | + |
| 39 | +| Parameter | Type | Description | |
| 40 | +| --------- | ---- | ----------- | |
| 41 | +| `data` | json | Body returned by the function — parsed JSON when it responds with application/json, otherwise the raw text | |
| 42 | +| `status` | number | HTTP status code of the response | |
| 43 | +| `headers` | json | Response headers as key-value pairs | |
| 44 | + |
| 45 | +### Modal Chat Completion |
| 46 | + |
| 47 | +Generate a chat completion from a model served by a Modal Endpoint |
| 48 | + |
| 49 | +#### Input |
| 50 | + |
| 51 | +| Parameter | Type | Required | Description | |
| 52 | +| --------- | ---- | -------- | ----------- | |
| 53 | +| `endpointUrl` | string | No | Endpoint URL from the Modal dashboard or `modal endpoint list`. Defaults to https://inference.us-west.modal.direct, which routes to Shared Endpoints on the model ID | |
| 54 | +| `model` | string | Yes | Model to generate with — the base model repo ID for a dedicated endpoint, or the endpoint hostname for a Shared Endpoint | |
| 55 | +| `content` | string | Yes | The user message content to send to the model | |
| 56 | +| `systemPrompt` | string | No | System prompt to guide the model behavior | |
| 57 | +| `maxTokens` | number | No | Maximum number of tokens to generate | |
| 58 | +| `temperature` | number | No | Sampling temperature \(e.g., 0 for deterministic, 0.7 for creative\) | |
| 59 | +| `topP` | number | No | Nucleus sampling probability mass between 0 and 1 | |
| 60 | +| `tokenId` | string | Yes | Modal proxy token ID \(wk-...\) | |
| 61 | +| `tokenSecret` | string | Yes | Modal proxy token secret \(ws-...\) | |
| 62 | + |
| 63 | +#### Output |
| 64 | + |
| 65 | +| Parameter | Type | Description | |
| 66 | +| --------- | ---- | ----------- | |
| 67 | +| `content` | string | Generated text content | |
| 68 | +| `model` | string | Model that produced the completion | |
| 69 | +| `finishReason` | string | Why generation stopped \(e.g., stop, length\) | |
| 70 | +| `usage` | object | Token usage reported by the endpoint | |
| 71 | +| ↳ `prompt_tokens` | number | Number of tokens in the prompt | |
| 72 | +| ↳ `completion_tokens` | number | Number of tokens in the completion | |
| 73 | +| ↳ `total_tokens` | number | Total number of tokens used | |
| 74 | + |
| 75 | +### Modal List Models |
| 76 | + |
| 77 | +List the model IDs a Modal proxy token can reach on an endpoint |
| 78 | + |
| 79 | +#### Input |
| 80 | + |
| 81 | +| Parameter | Type | Required | Description | |
| 82 | +| --------- | ---- | -------- | ----------- | |
| 83 | +| `endpointUrl` | string | No | Endpoint URL to query. Defaults to https://inference.us-west.modal.direct, which lists every Shared Endpoint the token can reach | |
| 84 | +| `tokenId` | string | Yes | Modal proxy token ID \(wk-...\) | |
| 85 | +| `tokenSecret` | string | Yes | Modal proxy token secret \(ws-...\) | |
| 86 | + |
| 87 | +#### Output |
| 88 | + |
| 89 | +| Parameter | Type | Description | |
| 90 | +| --------- | ---- | ----------- | |
| 91 | +| `models` | array | Models the token can reach on the endpoint | |
| 92 | +| `count` | number | Number of models returned | |
| 93 | + |
| 94 | + |
0 commit comments