Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions changes/fix-provider-recordings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
### Fixed

- **Recordings your telephony provider keeps.** A portal that works with an outside telephony provider, such as a
Beeline cloud PBX, links each recording to the provider's server. b24 refused such a link, so `meetings transcribe`
turned none of these calls into text. It now follows the link over https to an address on the public internet,
never to this computer or a local network, and sends nothing of your Bitrix24 sign-in with it. See
[Calls](../docs/calls.md#recordings).
- **`meetings transcribe --max-meetings` counts calls with a recording.** A call with no recording on the portal is
still listed under `failed`, but it no longer uses up the limit. Before, when the newest calls had no recording,
`--max-meetings 1` turned nothing into text.
2 changes: 1 addition & 1 deletion contracts/wirecat.json
Original file line number Diff line number Diff line change
Expand Up @@ -1531,7 +1531,7 @@
"github.com",
"*.githubusercontent.com"
],
"user_hosts": "the Bitrix24 portal in your webhook address, or the one you sign in to with a local application"
"user_hosts": "your Bitrix24 portal, and your telephony provider's server where a call recording link leads"
},
"desktop": {
"tagline": "Bitrix24 CRM: deals, leads, contacts and companies",
Expand Down
6 changes: 4 additions & 2 deletions docs/calls.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,10 @@ Each recording is saved in b24's state folder, under `recordings/<profile>/`, re
gives its path as `local_path`, and WireCat Desktop plays it. A recording can be a few megabytes per call: check the
space before you turn this on for a busy line.

The link the portal gives for a recording holds an access token. b24 follows it only on the portal's own address,
right after reading it, and never saves or prints it.
The link the portal gives for a recording holds an access token. b24 follows it right after reading it, with no
redirect, and never saves or prints it. When your telephony provider keeps the recordings itself, as a Beeline cloud PBX
does, the link leads to the provider's server. b24 follows such a link only over https and only to an address on the
public internet, never to this computer or your local network, and sends nothing of your Bitrix24 sign-in with it.

## Turn calls into text

Expand Down
8 changes: 6 additions & 2 deletions docs/dev/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,8 +267,12 @@ and `type`); field names are from the REST 3.0 field pages and the `user.get` an
to a contact's mobile. The portal's own link to a CRM record (`CRM_ENTITY_*`) is kept in the metadata, never a
join: cli-core has no rule for it.
- Only the calls of the users the CRM copy is for (`unassigned`, else the signed-in user); `scope: all` refuses.
- A recording is downloaded only with `download_recordings`, from `CALL_RECORD_URL`, on the portal's own host, with
no redirect. The link holds a token: never stored or printed. No `disk` scope.
- A recording is downloaded only with `download_recordings` or `meetings transcribe`, from `CALL_RECORD_URL`, with no
redirect. The link holds a token: never stored or printed. No `disk` scope. A portal with an outside telephony
provider (Beeline cloud PBX, checked live 2026-11-17) links the provider's own host: followed only over https, only
when every address the host resolves to is public (cli-core `isPublicAddress`), through `hostFetch` for that one
host, with no portal sign-in. The name is resolved once for the check and again for the request, so a host that
changes its answer in between is not caught.
- `api call` keeps the whole `voximplant` family denied, the call history too, since its answer holds those links.
- A number without `+` needs `phone_region`: the store's `phone_countries` never decides an identifier.

Expand Down
4 changes: 2 additions & 2 deletions docs/dev/test-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ ran it · ⛔ not tested offline, with the reason and where it is checked instea
Measured from the test run (`coverage/argv.jsonl`), not searched for — see
[TESTING.md](TESTING.md) for how, and for the states and failures that cut across commands.

**227 ✅ · 88 ⛔ · 0 ❌** — 109 commands, 206 options.
**228 ✅ · 87 ⛔ · 0 ❌** — 109 commands, 206 options.

| Command | Option | | Note |
|---|---|---|---|
Expand Down Expand Up @@ -203,7 +203,7 @@ Measured from the test run (`coverage/argv.jsonl`), not searched for — see
| `meetings repair` | `--yes` | ⛔ | cli-core's addMeetingCommands; its meetings/cli tests drive every option, and src/calls/calls.test.ts runs sync, list, show, transcript and people on b24's calls |
| `meetings transcribe` | | ✅ | |
| `meetings transcribe` | `--replace` | ✅ | |
| `meetings transcribe` | `--max-meetings` | ⛔ | src/calls/calls.test.ts passes it through its transcribe helper, with a fake speech engine |
| `meetings transcribe` | `--max-meetings` | ✅ | |
| `meetings transcribe` | `--model` | ⛔ | picks the local speech model; src/calls/calls.test.ts runs transcribe on a fake engine that has none |
| `events list` | | ✅ | |
| `models list` | | ✅ | |
Expand Down
2 changes: 0 additions & 2 deletions scripts/test-matrix-untested.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ const SHARED_MEETINGS =
"cli-core's addMeetingCommands; its meetings/cli tests drive every option, and src/calls/calls.test.ts runs sync, " +
"list, show, transcript and people on b24's calls"
const SHARED_MODELS = "cli-core's modelCatalogueCommand; its tests download a model from a fake catalogue"
const TRANSCRIBE_HELPER = "src/calls/calls.test.ts passes it through its transcribe helper, with a fake speech engine"
const LONG_RUNNING = `starts an HTTP server or a background process that runs until stopped; ${SHARED_MCP}`

export const UNTESTED: { command: string; option?: string; reason: string }[] = [
Expand Down Expand Up @@ -145,7 +144,6 @@ export const UNTESTED: { command: string; option?: string; reason: string }[] =
{ command: "meetings sync", option: "--lookback-days", reason: SHARED_MEETINGS },
{ command: "meetings repair", reason: SHARED_MEETINGS },
{ command: "meetings repair", option: "--yes", reason: SHARED_MEETINGS },
{ command: "meetings transcribe", option: "--max-meetings", reason: TRANSCRIBE_HELPER },
{
command: "meetings transcribe",
option: "--model",
Expand Down
29 changes: 29 additions & 0 deletions src/bitrix/transport.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,35 @@ describe("BitrixTransport", () => {
expect(JSON.stringify(events)).toContain("/rest/:user/:token/download.json")
})

it("downloads a recording a telephony provider keeps, from its public https host, with nothing of the sign-in", async () => {
const portal = fakePortal({ "rec.mp3": { text: "audio", headers: { "content-type": "audio/mpeg" } } })
const events: RequestEvent[] = []
const { transport } = transportOf(portal, {
lookup: async () => ["93.184.216.10"],
events: (event) => events.push(event as RequestEvent),
})
const file = await transport.download("https://records.pbx.example/files/a1b2/rec.mp3", 1_000)
expect(new TextDecoder().decode(file.bytes)).toBe("audio")
expect(portal.calls.map(({ url }) => url)).toEqual(["https://records.pbx.example/files/a1b2/rec.mp3"])
expect(JSON.stringify(portal.calls)).not.toContain(FAKE_SECRET)
expect(JSON.stringify(events)).toContain("records.pbx.example/…")
expect(JSON.stringify(events)).not.toContain("a1b2")
})

it.each([
["is not https", "http://records.pbx.example/rec.mp3", ["93.184.216.10"]],
["names this computer", "https://127.0.0.1/rec.mp3", []],
["resolves to a private network", "https://records.pbx.example/rec.mp3", ["10.0.0.5"]],
["resolves to a public and a private address", "https://records.pbx.example/rec.mp3", ["93.184.216.10", "fd00::1"]],
])("refuses a download link off the portal that %s, before any request", async (_, link, addresses) => {
const portal = fakePortal({ "rec.mp3": { text: "audio" } })
const { transport } = transportOf(portal, { lookup: async () => addresses })
const failure = await failureOf(transport.download(link, 1_000))
expect(failure.details.reason).toBe("bitrix.foreign-link")
expect(failure.message).not.toContain("rec.mp3")
expect(portal.calls).toEqual([])
})

it("keeps REST 2's total and next", async () => {
const portal = fakePortal({ "user.get": ok([{ ID: "1" }], { total: 120, next: 50 }) })
expect(await transportOf(portal).transport.call("user.get")).toEqual({
Expand Down
62 changes: 49 additions & 13 deletions src/bitrix/transport.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import { lookup as dnsLookup } from "node:dns/promises"
import { isIP } from "node:net"
import { CliError, isCliError, realSleep, type SleepLike, wallClock } from "@wirecat/cli-base"
import { hostFetch, providerWaitMs, readJsonResponse } from "@wirecat/cli-base/http"
import { apiJson, currentAttempt, type EventSink, providerEvent, providerRequest } from "@wirecat/cli-core/cli"
import { providerCall } from "@wirecat/cli-core/crm"
import { isPublicAddress } from "@wirecat/cli-core/documents"
import type { Pacer } from "@wirecat/cli-core/sends"
import { type EmployeeRule, employeeRuleOf } from "../api-keys.js"
import { isRecord } from "../body.js"
Expand Down Expand Up @@ -37,9 +40,19 @@ const isTimeout = (error: unknown) =>

export type FetchLike = (url: string, init: RequestInit) => Promise<Response>

/** The addresses a hostname resolves to. */
export type AddressLookup = (hostname: string) => Promise<string[]>

const systemLookup: AddressLookup = async (hostname) =>
(await dnsLookup(hostname, { all: true, verbatim: true })).map(({ address }) => address)

const refusedLink = (message: string) => new CliError("invalid_response", message, { reason: "bitrix.foreign-link" })

export interface BitrixTransportOptions {
auth: Webhook | AppAuth
fetch?: FetchLike
/** Resolves the host of a download link that leads off the portal. */
lookup?: AddressLookup
timeoutMs?: number
signal?: AbortSignal
events?: EventSink
Expand Down Expand Up @@ -78,6 +91,8 @@ const unsent = (error: unknown): never => {
export class BitrixTransport {
readonly #auth: Webhook | AppAuth
readonly #fetch: FetchLike
readonly #plainFetch: FetchLike | undefined
readonly #lookup: AddressLookup
readonly #timeoutMs: number
readonly #signal: AbortSignal | undefined
readonly #events: EventSink | undefined
Expand All @@ -95,6 +110,8 @@ export class BitrixTransport {
...(options.fetch ? { fetch: options.fetch as typeof fetch } : {}),
timeoutMs: this.#timeoutMs * 2,
}) as FetchLike
this.#plainFetch = options.fetch
this.#lookup = options.lookup ?? systemLookup
this.#signal = options.signal
this.#events = options.events
this.#pacer = options.pacer
Expand Down Expand Up @@ -166,26 +183,45 @@ export class BitrixTransport {
})
}

/** Reaches only the link's own host, once the host is known to be on the public internet. */
async #outsideFetch(url: URL): Promise<FetchLike> {
if (url.protocol !== "https:")
throw refusedLink("the portal gave a download link that is not https; b24 does not follow it")
const host = url.hostname.replace(/^\[(.*)\]$/, "$1")
const addresses = isIP(host)
? [host]
: await this.#lookup(host).catch(() => {
throw new CliError("provider_unavailable", `${url.hostname} could not be found`, {
reason: "bitrix.link-host-unknown",
})
})
if (!addresses.length || !addresses.every(isPublicAddress))
throw refusedLink("the portal gave a download link to this computer or a private network; b24 does not follow it")
return hostFetch({
hosts: [url.hostname],
...(this.#plainFetch ? { fetch: this.#plainFetch as typeof fetch } : {}),
timeoutMs: this.#timeoutMs * 2,
}) as FetchLike
}

/**
* A file behind a link the portal gave, such as a call's `CALL_RECORD_URL`. Such a link carries a token, so it is
* never put into an error or a run record, and it is followed only on the portal's own address, with no redirect.
* never put into an error or a run record, and it is followed with no redirect. A portal with an outside telephony
* provider gives the provider's own link to a recording; that one is followed only over https, only to a public
* address, and with nothing of the portal's sign-in.
*/
async download(link: string, maxBytes: number): Promise<{ bytes: Uint8Array; type: string | null }> {
const portal = new URL(this.#auth.origin)
const url = URL.canParse(link, portal.href) ? new URL(link, portal) : null
if (url?.origin !== portal.origin)
throw new CliError(
"invalid_response",
"the portal gave a download link to another address; b24 does not follow it",
{
reason: "bitrix.foreign-link",
},
)
const at = this.#pacer ? (await this.#pacer.reserve()) - this.#now() : 0
if (!url) throw refusedLink("the portal gave a download link b24 cannot read")
const onPortal = url.origin === portal.origin
const fetch = onPortal ? this.#fetch : await this.#outsideFetch(url)
// The portal's request limit does not cover a provider's own server.
const at = onPortal && this.#pacer ? (await this.#pacer.reserve()) - this.#now() : 0
if (at > 0) await this.#wait(at, "a download")
const end = providerRequest("download", this.#events ?? providerEvent, {
method: "GET",
endpointTemplate: loggedPath(url),
endpointTemplate: onPortal ? loggedPath(url) : `${url.hostname}/…`,
transport: "http",
})
const deadline = new AbortController()
Expand All @@ -196,14 +232,14 @@ export class BitrixTransport {
timer.unref()
const signal = AbortSignal.any([deadline.signal, ...(this.#signal ? [this.#signal] : [])])
try {
const response = await this.#fetch(url.href, { method: "GET", redirect: "error", signal }).catch((error) => {
const response = await fetch(url.href, { method: "GET", redirect: "error", signal }).catch((error) => {
throw this.#unanswered("download", signal.aborted ? signal.reason : error, { reads: true, key: undefined })
})
end({ status: response.status })
if (!response.ok)
throw new CliError(
response.status === 401 || response.status === 403 ? "permission_error" : "provider_unavailable",
`the portal refused the download (HTTP ${response.status})`,
`${onPortal ? "the portal" : url.hostname} refused the download (HTTP ${response.status})`,
{ reason: "bitrix.download-refused", status: response.status },
)
const size = Number(response.headers.get("content-length") ?? 0)
Expand Down
36 changes: 33 additions & 3 deletions src/calls/calls.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,12 +158,12 @@ describe("recordings", () => {
expect(portal.calls.filter(({ method }) => method === "download.json")).toHaveLength(1)
})

it("never follows a link to another address, and saves the call without it", async () => {
it("never follows a link into a private network, and saves the call without it", async () => {
expect((await b24(["work", "config", "set", "download_recordings", "true"])).code).toBe(0)
const elsewhere = row(1, { CALL_RECORD_URL: "https://elsewhere.example/rec.mp3?token=faketoken0" })
const elsewhere = row(1, { CALL_RECORD_URL: "https://10.0.0.5/rec.mp3?token=faketoken0" })
const ran = await b24(["work", "meetings", "sync", "--json"], { portal: portalWith([elsewhere]) })
expect(ran.code, ran.err).toBe(0)
expect(ran.err).toContain("another address")
expect(ran.err).toContain("private network")
expect(ran.err).not.toContain("faketoken0")
const listed = (await b24(["work", "meetings", "list", "--json"])).json() as { items: unknown[] }
expect(listed.items).toHaveLength(1)
Expand Down Expand Up @@ -270,6 +270,36 @@ describe("meetings transcribe", () => {
})
})

it("downloads a recording the telephony provider keeps on its own public host", async () => {
// A literal public address, so the test resolves no name.
const provider = "https://93.184.216.10/records/a1b2/rec.mp3"
const portal = portalWith([row(1, { CALL_RECORD_URL: provider })], {
"rec.mp3": { text: AUDIO, headers: { "content-type": "audio/mpeg" } },
})
await b24(["work", "meetings", "sync", "--json"], { portal })
const ran = await transcribe(portal)
expect(JSON.parse(ran.out).item).toMatchObject({ transcribed: 1, failed: [] })
expect(portal.calls.filter(({ method }) => method === "rec.mp3").map(({ url }) => url)).toEqual([provider])
})

it("spends --max-meetings only on calls with a recording", async () => {
const portal = portalWith(
[
row(1, { CALL_RECORD_URL: LINK }),
row(2, { CALL_START_DATE: "2026-11-11T10:00:00+03:00" }),
row(3, { CALL_START_DATE: "2026-11-12T10:00:00+03:00" }),
],
{ "download.json": { text: AUDIO, headers: { "content-type": "audio/mpeg" } } },
)
await b24(["work", "meetings", "sync", "--json"], { portal })
const ran = await transcribe(portal, ["--max-meetings", "1"])
expect(JSON.parse(ran.out).item).toMatchObject({
transcribed: 1,
failed: [{ reason: "recording.missing" }, { reason: "recording.missing" }],
remaining: 0,
})
})

it("keeps the recording with download_recordings, and reads the kept file the next time", async () => {
const portal = portalWith([row(1, { CALL_RECORD_URL: LINK })], {
"download.json": { text: AUDIO, headers: { "content-type": "audio/wav; codec=pcm" } },
Expand Down
10 changes: 7 additions & 3 deletions src/calls/transcribe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,16 +149,20 @@ export const transcribeCalls = async (
report.failed.push(failed(call.id, "recording.too-long", "the call is longer than 60 minutes"))
else pending.push(call)
}
// A call with no recording costs one read of the portal, so it does not count against the limit.
let tried = 0
for (const [index, call] of pending.entries()) {
if (index >= options.limit || options.signal?.aborted) {
if (tried >= options.limit || options.signal?.aborted) {
report.remaining = pending.length - index
report.stopped = index < options.limit
report.stopped = tried < options.limit
break
}
const step = `${index + 1}/${Math.min(pending.length, options.limit)} meeting ${call.id}`
const step = `${tried + 1}/${Math.min(pending.length, options.limit)} meeting ${call.id}`
try {
tried++
const found = await recordingOf(ports, call.externalId)
if (!found) {
tried--
report.failed.push(failed(call.id, "recording.missing", "the call has no recording on the portal"))
options.progress?.(`${step} failed: recording.missing`)
continue
Expand Down
2 changes: 1 addition & 1 deletion src/identity.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export const DESKTOP: ManifestIdentity = {
docs: "https://wirecat.dev/en/docs/b24",
// A local application's sign-in exchanges and refreshes its tokens there (apidocs settings/oauth).
hosts: ["oauth.bitrix.info"],
userHosts: "the Bitrix24 portal in your webhook address, or the one you sign in to with a local application",
userHosts: "your Bitrix24 portal, and your telephony provider's server where a call recording link leads",
tagline: "Bitrix24 CRM: deals, leads, contacts and companies",
syncs: "deals, leads, contacts and companies",
signIn: {
Expand Down
Loading