diff --git a/README.md b/README.md index be273c8..7fc2f03 100644 --- a/README.md +++ b/README.md @@ -47,8 +47,10 @@ asyncio.run(channel.connect()) - [Webhook server adapter](docs/webhook-server.md) - [CardKit streaming](docs/cardkit-streaming.md) - [Deduplication architecture](docs/dedup-architecture.md) +- [Meeting channel](docs/meeting-channel.md) — agents that perceive and respond inside a live meeting - [Release notes](docs/release-notes/v1.1.0.md) - [Echo bot sample](samples/channel/echo_bot.py) +- [Meeting samples](samples/channel/) — [join a meeting](samples/channel/meeting_join_bot.py), [follow one without joining](samples/channel/meeting_follow_agenda.py) ## Migration from `lark_oapi.channel` @@ -69,6 +71,45 @@ your application needs the full OpenAPI SDK surface. See the [migration guide](docs/migration-from-lark-oapi.md) for import mapping, runtime compatibility notes, and a migration checklist. +## Meeting Channel + +Agents that perceive and respond **inside a live meeting** — captions, meeting +chat, participants, shared documents. Two entry points, one session type. + +```python +# The bot joins as a real participant and can speak into the meeting. +channel.on("meetingInvited", lambda inv: channel.join_meeting(inv.meeting_no)) + +# Or follow the meeting a user is already in, without joining it. +session = await channel.follow_my_meeting(user_open_id="ou_...") + +session.on("transcript", lambda e: notes.append(e.text)) +session.on("chat", on_chat) +await session.send_message("noted") # joined sessions only +await session.leave() +``` + +Three things to know before you start: + +- **The bot's own in-meeting messages come back to it.** So a meeting-chat + handler needs `if event.self_echo: return`, or the bot spends the meeting + replying to itself at network speed. +- **Disconnecting does not take the bot out of the meeting.** `disconnect()` + closes the event channel; the bot stays a participant, which is what makes + reconnects safe. A process that is really exiting should `leave()` each + session first. +- **`follow_my_meeting` reads what every participant says, and the bot is not + in the participant list.** Two more things: the `user_open_id` you pass has + to be somebody you have already established is the requester — the SDK + receives a string and cannot check that for you — and what the user grants + in one go is + **every** scope your app applied for, not just the meeting read. See + [Security configuration](docs/security.md#meeting-channel). + +For event ordering, how captions settle from interim to final, how many +sessions can run at once, and what to check when nothing arrives, see +[Meeting channel](docs/meeting-channel.md). + ## Security Mode `SecurityConfig` defaults to compatibility mode so existing bots continue to diff --git a/README.zh.md b/README.zh.md index 6c4fdfb..6cad05a 100644 --- a/README.zh.md +++ b/README.zh.md @@ -46,8 +46,10 @@ asyncio.run(channel.connect()) - [Webhook 服务适配](docs/webhook-server.md) - [CardKit 流式回复](docs/cardkit-streaming.md) - [去重架构](docs/dedup-architecture.md) +- [会议通道](docs/meeting-channel.md) —— 让 Agent 在会议进行中感知与响应 - [发布说明](docs/release-notes/v1.1.0.md) - [Echo bot 示例](samples/channel/echo_bot.py) +- [会议示例](samples/channel/) —— [入会](samples/channel/meeting_join_bot.py)、[不入会跟随](samples/channel/meeting_follow_agenda.py) ## 从 `lark_oapi.channel` 迁移 @@ -67,6 +69,37 @@ from lark_channel import FeishuChannel 详见 [迁移手册](docs/migration-from-lark-oapi.md)，其中包含 import 映射、运行时兼容 说明和迁移检查清单。 +## 会议通道 + +让 Agent 在**会议进行中**感知内容（字幕、会中聊天、参会人进退、文档共享）并作出响应。两个入口，同一个会话类型。 + +```python +# Bot 作为真实参会者入会，可在会中发言 +channel.on("meetingInvited", lambda inv: channel.join_meeting(inv.meeting_no)) + +# 或跟随用户当前所在的会议，不入会 +session = await channel.follow_my_meeting(user_open_id="ou_...") + +session.on("transcript", lambda e: notes.append(e.text)) +session.on("chat", on_chat) +await session.send_message("已记录") # 仅入会的会话可用 +await session.leave() +``` + +使用前请注意三点： + +- **Bot 自己发的会中消息会再推回来一次。** 所以处理会中聊天时要先写 `if event.self_echo: + return`，否则 Bot 会不停地回应自己刚说的话。 +- **断开连接不会让 Bot 退出会议。** `disconnect()` 只是断开事件通道，Bot 仍然留在会议 + 里（正因为如此，重连之后能接着收事件）。进程真要退出前，请先对每个会话调用 `leave()`。 +- **`follow_my_meeting` 会读到会议里所有人的发言，而 Bot 不出现在参会人名单里。** + 另外两点：传给 `user_open_id` 的必须是你已经自行确认过身份的那个人（SDK 只收到一个 + 字符串，没法替你校验）；用户一次授权给出的是你的应用申请过的**全部**权限，不只是读 + 会议。详见 [安全配置](docs/security.md#meeting-channel)。 + +事件的先后顺序、字幕从临时结果到定稿、同时最多能开多少个会话，以及收不到事件时怎么排查， +见 [会议通道](docs/meeting-channel.md)。 + ## 安全模式 `SecurityConfig` 默认使用兼容模式，便于已有机器人平滑迁移。生产发布建议先使用 diff --git a/docs/meeting-channel.md b/docs/meeting-channel.md new file mode 100644 index 0000000..d8f662c --- /dev/null +++ b/docs/meeting-channel.md @@ -0,0 +1,254 @@ +# Meeting channel + +Agents that perceive and respond **inside a live meeting**: captions, meeting +chat, participants arriving and leaving, documents being shared. + +Two entry points, one session type. Moving from one to the other changes the +entry-point line and nothing else. + +| | `follow_my_meeting` — as the user | `join_meeting` — as the bot | +|---|---|---| +| Visible in the meeting | no | yes, a real participant | +| Credential | the user's own access token | the app's tenant token | +| How content arrives | polling `bots/events` | pushed `vc.bot.meeting_activity_v1` | +| Needs `connect()` | **no** — REST only | **yes** — activity is pushed | +| Can speak in the meeting | no, reply over IM | yes, `send_message` | +| Scope | `vc:meeting.meetingevent:read` | `vc:meeting.bot.join:write`, `vc:meeting.message:write` | + +Both require the meeting's "allow agents to join" setting and Feishu client +7.68 or later. Joining as the bot is gated behind an application process. + +## Joining a meeting + +```python +channel = FeishuChannel(app_id=..., app_secret=...) + +async def on_invited(invitation): + session = await channel.join_meeting(invitation.meeting_no) + + def on_chat(event): + if event.self_echo: + return # see "Echoes" below — this line is load-bearing + ... + + session.on("chat", on_chat) + +channel.on("meetingInvited", on_invited) +await channel.connect() +``` + +## Following a meeting + +```python +session = await channel.follow_my_meeting(user_open_id="ou_...") +session.on("transcript", lambda e: notes.append(e.text)) +``` + +No `connect()` needed. Read [Who the ticket belongs to](#who-the-ticket-belongs-to) +before wiring the `user_open_id`. + +## Session events + +`transcript` · `chat` · `participant` · `share` · `document_context` · `end` · +`error`. `on()` is multicast and returns an unsubscribe. + +```python +off = session.on("transcript", handler) +off() +``` + +`document_context` carries **identifiers only** — a comment id, an element +token. Fetching the comment body or the asset is your job, within the shared +document's temporary grant, with permissions you applied for. + +### Ordering, and the price of it + +Events are delivered **in order**, and each handler is awaited before the next +event. Order is the meaning of some of them: swapping the shared document +arrives as `magic_share_ended` then `magic_share_started`, and reordering makes +you reconstruct the wrong document. + +The price: + +- a handler that `await`s and takes a long time holds up **this meeting's** + stream; +- a handler that blocks **without** awaiting (`time.sleep`, a synchronous HTTP + call, heavy CPU) holds up **the entire process** — every meeting, the message + path, the socket heartbeat. There is one thread. Hand blocking work to an + executor. + +### Captions + +The protocol has no "final" marker, so a later item with the same +`sentence_id` supersedes the earlier text. Upsert on `sentence_id`. + +`MeetingOptions(stabilize_seconds=...)` debounces instead: `0.0` (the default) +delivers every revision; a positive value delivers a sentence once, after it +stops changing for that long — at the cost of one window of latency, and of a +sentence never settling while somebody keeps talking. + +### Echoes + +The bot's own contributions come back around: an in-meeting message is pushed +back as meeting chat. Those arrive with `self_echo=True` and **are still +delivered** — a full record wants the bot's own turns. + +So `if event.self_echo: return` is not boilerplate. Without it, a handler that +replies to chat replies to itself, at network speed. The SDK's backstop is +`meeting.send_rate_limit_per_minute` (default 20), after which `send_message` +raises `rate_limited`. + +When the bot's own id is not yet known, `self_echo` is `True` — "maybe" has to +read as "yes", because `False` means "definitely not me" and would let the loop +close. + +## Leaving versus disposing + +| | Departs the meeting | Use for | +|---|---|---| +| `await session.leave()` | **yes** | you are done, the meeting ended | +| `session.dispose()` | **no** | reconnects, in-process cleanup | + +`dispose()` deliberately does not depart: a reconnect must not make the bot +vanish from every meeting it is in. The flip side is that **the bot stays a +participant**, so: + +- `channel.disconnect()` disposes sessions — it does not leave meetings; +- before the process exits, `leave()` every live session, or the bot sits in + those meetings until the server ends them. `leave()` keeps working after + `dispose()` precisely so this is possible. + +Both are idempotent. + +## Limits and reclamation + +```python +FeishuChannel( + app_id=..., app_secret=..., + meeting=MeetingChannelConfig( + max_concurrent_sessions=32, + idle_timeout_seconds=0.0, + liveness_probe_interval_seconds=300.0, + send_rate_limit_per_minute=20, + ), +) +``` + +Sessions are created **from outside** your process — joining starts at an +invitation from anybody who can add the bot to a meeting. Hence a ceiling, +shared by both entry points. + +A seat is released when there is evidence the bot is no longer a participant: +a clean departure, a departure rejected because the meeting is gone, a +meeting-ended event, or a probe confirming absence. An inconclusive departure +(5xx, timeout) keeps the seat, and the next `join_meeting` / +`follow_my_meeting` retries it before comparing the ceiling. + +`idle_timeout_seconds` is **off by default**: the liveness probe already +catches the bot being removed, so what is left for idle reclamation is a +meeting where nobody happens to be talking — and walking out of that one is +visible and wrong. Turning it on makes the session depart the meeting. + +One caveat if you leave it off: idle reclamation is also the only backstop for +a wedged handler. **If your handlers can block for a long time, set a positive +value.** + +## Diagnosing silence + +Failures on this path are silent by nature — an undeclared subscription, a +missing permission and a renamed field all look like "nothing happened". + +```python +health = channel.get_meeting_event_health() +``` + +| Symptom | Reading | Where to look | +|---|---|---| +| the type is absent from `stats` | the platform never sent it | meeting setting, subscription declaration, is the bot in the meeting | +| `empty == received` | nothing could be unpacked | field names or structure changed | +| `0 < empty < received` | some could not | an uncovered sub-type | +| `liveness.consecutive_unknown` climbing | the probe never concludes | its permission assumption may not hold in this tenant | +| `membership.held` never falling | seats are not coming back | `released_by_evidence` shows why | +| `dropped` climbing | a handler is slower than the meeting, so its session's delivery queue hit its ceiling | the handler — and move blocking work to an executor | +| `TOO_MANY_SESSIONS` while `membership.held` is `0` | the seat is held by a live session, not by server-side membership | `sessions` — `held` counts only the latter | + +## When a handler cannot keep up + +Delivery is serial and awaited, so a handler that yields but takes a long time +makes its session's queue grow at whatever rate the meeting produces activity. +That queue has a ceiling; past it, the newest event is refused and counted in +`dropped`. + +The newest is refused rather than the oldest evicted, because order is meaning +here — a document swap arrives as `magic_share_ended` then +`magic_share_started`, and dropping from the front would split such a pair and +leave a queue that still looks complete. Refusing at the tail keeps what is +queued contiguous, so a gap is a gap at the end and `dropped` says so. + +Error reports have their own headroom above the ceiling: they are what explain +why a session went away, and a ceiling full of transcripts must not be able to +drop the explanation. + +## Untrusted content + +`transcript.text`, `chat.content`, `topic`, `actor.name`, `doc.url`, +`doc.title` and the `document_context` sub-objects are written by meeting +participants, who may be external or guest users. Escape them before rendering +and never concatenate them into a log line. + +Prompt injection is a residual risk that no SDK layer can remove — the whole +point of this feature is feeding meeting speech to a model. `actor` and +`self_echo` give you the minimum needed to tier trust; the rest is your call. + +## Subscribing to unwrapped events + +```python +off = channel.on_raw_event("vc.bot.meeting_started_v1", handler) +``` + +Different from the `"raw"` event, which mirrors already-wrapped events and is +controlled by `inbound.emit_raw_events`. This subscribes to types the channel +does not wrap, and ignores that switch. + +The payload is authentic — this runs after signature verification and +decryption — but **unredacted**, and this path sits **outside the safety +pipeline**: no policy gate, no dedup, no processing lock, no loop guard. + +> **Subscribing to a type the channel already handles opens an unpoliced path +> into that type.** With `dm_policy="allowlist"` set, a raw subscription to +> `im.message.receive_v1` still receives direct messages from everybody, and a +> redelivered event runs your handler again. That is what an escape hatch is — +> but know that you have opened one. + +## Errors + +`not_supported` (`send_message` while only following), `meeting_not_found`, +`too_many_sessions`, plus the existing `rate_limited`, `not_connected` and the +rest. + +`FeishuChannelError.context` may carry `console_url` — a **signed one-click +authorization link**. Treat it as a credential: do not echo it into a chat, a +web page or a support ticket. + +Failures inside a session go to the session's `error` event. With no handler +registered they are logged instead, minimally and without `context`. + +## Who the ticket belongs to + +`follow_my_meeting(user_open_id=...)` reads a meeting under **that user's** +authorization. The SDK receives a string; it cannot check whose it is, the +ticket store is shared across the process, and a cached ticket resolves +**without notifying its owner**. + +So `user_open_id` must be somebody you have already established is the +requester. Passing a value taken from an inbound message means listening in on +someone else's meeting with their authorization, invisibly. `prompt_context` +must belong to the same person — pairing one person's `user_open_id` with +another's context sends the authorization card to the wrong person and files +the resulting ticket under the first. + +`meeting.follow_allowlist` is the available gate; `meeting.invite_allowlist` is +its counterpart on the join side. Both default to open, because a closed +default would make the feature unusable out of the box. + +See [Security configuration](security.md#meeting-channel) for the full picture. diff --git a/docs/security.md b/docs/security.md index 50b7979..bf1223d 100644 --- a/docs/security.md +++ b/docs/security.md @@ -110,3 +110,69 @@ security = SecurityConfig(mode="audit", audit_recorder=AuditRecorder()) See the [Channel reference](./reference.md#security-configuration) for the full option table. + +## User access tokens + +`require_user_auth` and `follow_my_meeting` act under a **user's** authorization +rather than the app's. Three properties of that are yours to handle. + +**The open_id decides whose authorization is used, and the SDK cannot check it.** +It receives a string and looks up whatever ticket is filed under it, so passing +a user-controlled value acts as that person — without notifying them. The +`user_open_id` you pass must be somebody you have already established is the +requester, and `prompt_context` must belong to that same person: the +authorization card carries a one-time grant, so sending it elsewhere lets a +different person authorize *their* account while the resulting ticket is filed +under the first one's id. + +**The granted scope is wider than the call suggests.** A call may ask for +`vc:meeting.meetingevent:read`, but the device flow issues a ticket carrying +every scope the application applied for — commonly calendar, documents and IM +as well. That ticket is stored per user and reused by anything else in the +process that resolves a ticket for the same user, for as long as it stays valid. +Where it is stored is your choice: the default `InMemoryTokenStore` keeps it in +process memory and loses it on restart, `FileTokenStore` writes plaintext and is +development-only, and production wants your own `TokenStore` over a secret +manager. + +**Resolution runs on the channel's background loop**, serialized per user, so a +concurrent refresh cannot take a valid authorization away from its owner. Two +consequences: `prompt_context.respond` is invoked from that loop's thread, so an +object bound to a different event loop will not work; and a process that only +calls `require_user_auth` still gets the channel's background thread. + +## Paths outside the message policy + +Two entry points reach your handlers without passing through `PolicyConfig`, +`SeenCache` dedup, the processing lock or the loop guard. Both are deliberate, +and both default to open: + +- **`on_raw_event`** — subscribing to a type the channel already handles opens an + unpoliced path into that type. With `dm_policy="allowlist"` set, a raw + subscription to `im.message.receive_v1` still receives direct messages from + everybody. +- **`meetingInvited`** — the only way into a joined meeting, triggered by anybody + who can add the bot to one. Gate it with + `MeetingChannelConfig.invite_allowlist`. + +## Meeting channel + +`follow_my_meeting` reads a meeting under a user's own authorization — see +[User access tokens](#user-access-tokens) for what that authorization actually +covers — and the bot is **not visible in the meeting**. It collects every +participant's speech for as long as the meeting lasts. Informing them is the +integrating application's responsibility; this SDK does not prompt, and cannot. +The first call in a process logs a warning to that effect. +`MeetingChannelConfig.follow_allowlist` gates it by open_id, but defaults to +`None` (open) — an opt-in, not a safety net you already have. + +Two values on this path are credentials that do not look like one: + +- **`console_url`**, which a permission failure may carry in + `FeishuChannelError.context`, is a signed one-click authorization link — a + capability, not a help page. The redaction layer masks it in logs; it cannot + mask it in your own output. Never echo it into a chat message, a web page or a + support ticket. +- **Meeting passwords**, both the one you pass to `join_meeting` and the one some + meeting responses hand back. Neither reaches logs, `raw` payloads, error + objects or the session. diff --git a/lark_channel/api/vc/__init__.py b/lark_channel/api/vc/__init__.py new file mode 100644 index 0000000..e206673 --- /dev/null +++ b/lark_channel/api/vc/__init__.py @@ -0,0 +1 @@ +"""Minimal VC primitives required by the meeting channel.""" diff --git a/lark_channel/api/vc/bot.py b/lark_channel/api/vc/bot.py new file mode 100644 index 0000000..551bc6d --- /dev/null +++ b/lark_channel/api/vc/bot.py @@ -0,0 +1,178 @@ +"""Request builders for the five ``vc/v1/bots`` endpoints the meeting +channel needs. + +Thin builders in the shape of :mod:`lark_channel.api.drive.comment` and +:mod:`lark_channel.api.wiki.node`: they produce a :class:`BaseRequest` and +nothing else. Execution and response parsing live in the channel layer. + +**Every builder declares exactly one token type.** Declaring two is not a +harmless superset: + +- ``core.token.auth.verify`` walks tenant → app → user and returns at the + first match, rewriting ``token_types`` in place. A request declaring + ``{TENANT, USER}`` therefore resolves to a freshly minted *tenant* token + and silently discards the user token the caller supplied. +- ``Transport._build_header`` iterates ``token_types`` and overwrites + ``Authorization`` once per entry, last write winning. ``AccessTokenType`` + is an ``Enum``, so set iteration order follows ``hash(name)`` and varies + with the process hash seed — the same code would send a different identity + from one run to the next. + +``bots/events`` is a dual-identity endpoint, so it gets two named builders +rather than one with a token-type argument: which identity a meeting read +happens under is the most consequential decision on this path, and it +belongs in the function name rather than in an argument that can default. +""" + +from typing import Optional + +from lark_channel.core.enum import AccessTokenType, HttpMethod +from lark_channel.core.model import BaseRequest + +#: The actor ids in the event stream must share a namespace with the bot's own +#: open_id, or echo detection compares two unrelated random strings and never +#: matches. Pinned here rather than exposed as a parameter so there is no way +#: to get it wrong from the outside. +_USER_ID_TYPE = "open_id" + +#: ``bots/join`` accepts no other value; the protocol reserves the field. +_JOIN_TYPE_BOT = 1 + + +def _request(method: HttpMethod, uri: str, token_type: AccessTokenType) -> BaseRequest: + req = BaseRequest() + req.http_method = method + req.uri = uri + req.token_types = {token_type} + return req + + +def build_bot_join_request( + *, + meeting_no: str, + password: Optional[str] = None, + call_id: Optional[str] = None, +) -> BaseRequest: + """Bot joins a meeting. Tenant token only. + + ``password`` is a credential: it must not reach logs, ``raw`` payloads or + error objects. It is passed straight through to the request body here and + nowhere else. + """ + req = _request(HttpMethod.POST, "/open-apis/vc/v1/bots/join", AccessTokenType.TENANT) + body = { + "join_type": _JOIN_TYPE_BOT, + "join_identify": {"meeting_no": meeting_no}, + } + if password is not None: + body["password"] = password + if call_id is not None: + body["call_id"] = call_id + req.body = body + return req + + +def build_bot_leave_request(*, meeting_id: str) -> BaseRequest: + """Bot leaves a meeting. Tenant token only. + + Takes the long meeting id. The endpoint rejects nine-digit meeting + numbers (HTTP 400 / ``121105 meeting not exist``), so a caller holding + only a meeting number has nothing useful to send here. + """ + req = _request( + HttpMethod.POST, "/open-apis/vc/v1/bots/leave", AccessTokenType.TENANT + ) + req.body = {"meeting_id": meeting_id} + return req + + +def build_bot_message_request( + *, + meeting_id: str, + msg_type: str, + content: str, + uuid: str, +) -> BaseRequest: + """Send an in-meeting message. Tenant token only. + + ``uuid`` is the caller-supplied idempotency key. + """ + req = _request( + HttpMethod.POST, "/open-apis/vc/v1/bots/message", AccessTokenType.TENANT + ) + req.body = { + "meeting_id": meeting_id, + "msg_type": msg_type, + "content": content, + "uuid": uuid, + } + return req + + +def _events_request( + token_type: AccessTokenType, + meeting_id: str, + page_token: Optional[str], + page_size: Optional[int], +) -> BaseRequest: + req = _request(HttpMethod.GET, "/open-apis/vc/v1/bots/events", token_type) + req.add_query("meeting_id", meeting_id) + req.add_query("user_id_type", _USER_ID_TYPE) + if page_token is not None: + req.add_query("page_token", page_token) + if page_size is not None: + req.add_query("page_size", page_size) + return req + + +def build_bot_events_request_as_user( + *, + meeting_id: str, + page_token: Optional[str] = None, + page_size: Optional[int] = None, +) -> BaseRequest: + """Read in-meeting events as the user, for ``follow_my_meeting``. + + Requires ``vc:meeting.meetingevent:read`` on a user access token, and + requires the token to be supplied through ``RequestOption`` — which in + turn requires the client to be built with ``enable_set_token(True)``. + """ + return _events_request( + AccessTokenType.USER, meeting_id, page_token, page_size + ) + + +def build_bot_events_request_as_app( + *, + meeting_id: str, + page_token: Optional[str] = None, + page_size: Optional[int] = None, +) -> BaseRequest: + """Read in-meeting events as the app: liveness and backfill while joined. + + Same endpoint, tenant token, and the scope is exactly the one + ``bots/join`` already needs — so probing a joined meeting costs no + additional credential and no additional authorization. + + The endpoint rejects ``page_size`` below 20 (``99992402``) during field + validation, so a probe cannot ask for a single item. + """ + return _events_request( + AccessTokenType.TENANT, meeting_id, page_token, page_size + ) + + +def build_user_active_meeting_request(*, user_id: Optional[str] = None) -> BaseRequest: + """Look up the meeting the user is currently in. User token only. + + Omitting ``user_id`` means "whoever the token belongs to". + """ + req = _request( + HttpMethod.GET, + "/open-apis/vc/v1/bots/user_active_meeting", + AccessTokenType.USER, + ) + req.add_query("user_id_type", _USER_ID_TYPE) + if user_id is not None: + req.add_query("user_id", user_id) + return req diff --git a/lark_channel/channel/__init__.py b/lark_channel/channel/__init__.py index 83aa361..3fb8150 100644 --- a/lark_channel/channel/__init__.py +++ b/lark_channel/channel/__init__.py @@ -44,6 +44,7 @@ async def on_message(msg): MarkdownConverter, MediaCacheConfig, MediaCapabilities, + MeetingChannelConfig, NameCacheConfig, OutboundConfig, OversizeContext, @@ -176,6 +177,25 @@ async def on_message(msg): VoteContent, ) +from .meeting import ( + ActivityTypeStats, + DocumentContextEvent, + LivenessHealth, + MeetingActor, + MeetingChatEvent, + MeetingEndEvent, + MeetingEventHealth, + MeetingEvents, + MeetingInvitedEvent, + MeetingOptions, + MeetingSession, + MembershipHealth, + ParticipantEvent, + ShareDocInfo, + ShareEvent, + TranscriptEvent, +) + __all__ = [ # Entry points "FeishuChannel", @@ -210,6 +230,24 @@ async def on_message(msg): "TextBatchConfig", "TransportConfig", "UATConfig", + # Meeting channel + "ActivityTypeStats", + "DocumentContextEvent", + "LivenessHealth", + "MeetingActor", + "MeetingChatEvent", + "MeetingEndEvent", + "MeetingEventHealth", + "MeetingEvents", + "MeetingInvitedEvent", + "MeetingOptions", + "MeetingSession", + "MembershipHealth", + "ParticipantEvent", + "ShareDocInfo", + "ShareEvent", + "TranscriptEvent", + "MeetingChannelConfig", # Events "ChannelEventName", "Events", diff --git a/lark_channel/channel/_coerce.py b/lark_channel/channel/_coerce.py index 8972467..0e8b885 100644 --- a/lark_channel/channel/_coerce.py +++ b/lark_channel/channel/_coerce.py @@ -51,6 +51,7 @@ "reject", "comment", "raw", "raw_event", + "meetingInvited", "meeting_invited", "reconnecting", "reconnected", "error", @@ -65,6 +66,7 @@ "bot_leave": "botLeave", "message_read": "messageRead", "raw_event": "raw", + "meeting_invited": "meetingInvited", } diff --git a/lark_channel/channel/auth/uat_runner.py b/lark_channel/channel/auth/uat_runner.py index 7a1eeca..7f1ef76 100644 --- a/lark_channel/channel/auth/uat_runner.py +++ b/lark_channel/channel/auth/uat_runner.py @@ -6,6 +6,7 @@ """ import asyncio +import weakref from typing import Any, Dict, List from lark_channel.core.log import logger @@ -21,15 +22,50 @@ # same user don't both try to refresh an expiring token simultaneously. The # interactive device-flow prompt/poll step runs outside this lock so a waiting # authorization does not block unrelated cache reads forever. -# The locks bind to the loop of the first caller; callers on other loops fall -# back to lock-less behaviour (rare; same-user concurrency across loops is not -# a supported configuration). -_user_locks: Dict[str, asyncio.Lock] = {} +# The locks bind to the loop of the first caller. A caller on a *different* +# loop gets no mutual exclusion at all when the lock is free, and a +# ``RuntimeError`` about a future attached to another loop when it is +# contended — it does not degrade gracefully. Same-user concurrency across +# loops is therefore not a supported configuration; callers that must handle +# it should treat that RuntimeError as a credential failure rather than let it +# escape as an unhandled task exception. +# Keyed by loop first, then by user. An ``asyncio.Lock`` binds to the loop it +# is first awaited on, so a single flat registry hands a lock created on one +# loop to a caller on another — which is the failure described above. The outer +# map holds loops weakly, so a loop that goes away (``stop()`` builds a fresh +# one on the next ``start()``) takes its locks with it instead of leaving +# permanently unusable entries behind. +_loop_user_locks: "weakref.WeakKeyDictionary" = weakref.WeakKeyDictionary() +_loopless_user_locks: Dict[str, asyncio.Lock] = {} def _get_user_lock(user_open_id: str) -> asyncio.Lock: - """Lazily create + memoize a per-user asyncio.Lock on the current loop.""" - return _user_locks.setdefault(user_open_id, asyncio.Lock()) + """The per-user lock for the loop this call is running on.""" + try: + loop = asyncio.get_running_loop() + except RuntimeError: + # No running loop: nothing can contend, so a shared bucket is enough. + return _loopless_user_locks.setdefault(user_open_id, asyncio.Lock()) + try: + per_user = _loop_user_locks[loop] + except (KeyError, TypeError): + # TypeError, not just KeyError: `WeakKeyDictionary.__getitem__` builds a + # weak reference to look the key up, so a loop that cannot be + # weak-referenced raises right here rather than on assignment. + per_user = {} + try: + _loop_user_locks[loop] = per_user + except TypeError: + # A loop implementation that cannot be weak-referenced (some + # third-party loops). Falling back keeps credential refresh working + # — losing the automatic cleanup is a far smaller problem than + # raising on this path, which every UAT caller goes through. + return _loopless_user_locks.setdefault(user_open_id, asyncio.Lock()) + lock = per_user.get(user_open_id) + if lock is None: + lock = asyncio.Lock() + per_user[user_open_id] = lock + return lock async def require_user_auth( @@ -112,3 +148,58 @@ async def require_user_auth( uat.scopes = list(scopes) await token_store.set(user_open_id, uat) return uat + + +async def resolve_user_auth_non_interactive( + *, + device_flow: DeviceFlowClient, + token_store: TokenStore, + uat_config: Any, + user_open_id: str, +) -> UAT: + """Resolve a usable UAT for ``user_open_id`` **without** prompting anybody. + + Cache lookup plus a refresh when the ticket is close to expiry, and + nothing else. Raises :class:`UATAuthError` when there is no usable ticket. + + This lives here, next to :func:`require_user_auth`, because it has to take + the *same* per-user lock — the registry is module-level, so a lock created + anywhere else excludes nothing. Two things go wrong without shared + serialization: a refresh returns a **new** refresh token and retires the + old one, so whichever caller arrives second with the stale one is rejected, + and :func:`require_user_auth` answers a rejection by deleting the ticket — + taking a perfectly good authorization away from its owner, who then gets an + unexpected authorization card. + + Why a separate function rather than a flag on :func:`require_user_auth`: + that one starts a device flow whenever the stored scopes do not contain the + requested one verbatim. Ticket scopes are whatever the platform granted the + app, so that is an ordinary state, not an error — and a polling loop asking + every few seconds would turn it into an unbounded stream of authorization + cards, or a silent six-hundred-second stall inside ``poll``. + + Refresh failures do **not** delete the ticket. Deleting is the interactive + path's prerogative: it can ask for a new authorization immediately, whereas + a polling loop can only arrange for somebody's next unrelated call to fail + with a surprise card. + """ + slack = getattr(uat_config, "refresh_before_expiry_seconds", 0) or 0 + async with _get_user_lock(user_open_id or ""): + existing = await token_store.get(user_open_id or "") + if existing is None: + raise UATAuthError( + "no stored user authorization for this user; authorize once " + "interactively before starting a non-interactive flow" + ) + if not uat_needs_refresh(existing, slack_seconds=slack): + return existing + if not existing.refresh_token: + raise UATAuthError("stored user authorization expired and cannot be renewed") + refreshed = await device_flow.refresh(existing.refresh_token) + refreshed.open_id = user_open_id + if not refreshed.scopes and existing.scopes: + refreshed.scopes = existing.scopes + # Written back inside the lock: the refresh token just rotated, and the + # one we used is already dead. + await token_store.set(user_open_id, refreshed) + return refreshed diff --git a/lark_channel/channel/channel.py b/lark_channel/channel/channel.py index 2e7765c..2d5489e 100644 --- a/lark_channel/channel/channel.py +++ b/lark_channel/channel/channel.py @@ -65,6 +65,12 @@ from .auth.token_store import InMemoryTokenStore, TokenStore from .auth.uat_runner import require_user_auth +from .auth.uat_runner import resolve_user_auth_non_interactive +from .meeting import dedup as _meeting_dedup +from .meeting.loop_affinity import await_on as _await_on_loop +from .meeting import registry as _meeting_registry +from .meeting.types import MeetingEventHealth, MeetingOptions +from .raw_events import RawEventRegistry from .bot_identity import BotIdentity, fetch_bot_identity from .chat_member_cache import ChatMemberCache from .chat_mode import ChatModeCache @@ -104,6 +110,7 @@ from .outbound.streaming.markdown_stream import MarkdownStreamController from .quote import QuoteResolver from .safety import RejectEvent, SafetyPipeline +from .safety.dedup_cache import SeenCache from .types import ( UAT, BotAddedEvent, @@ -286,6 +293,21 @@ def _extract_fetched_sender(item: Dict[str, Any]) -> Any: # --------------------------------------------------------------------------- +#: The only scope the follow path asks for. What the platform actually grants +#: is whatever the app has applied for, which is usually much broader — see +#: `follow_my_meeting` and docs/security.md. +_MEETING_EVENT_SCOPE = "vc:meeting.meetingevent:read" + + +def _meeting_payload(handler): + """Adapt a dispatcher callback to the meeting channel's dict-in signature.""" + + def _dispatch(data): + handler(_coerce.obj_to_dict(data) or {}) + + return _dispatch + + class FeishuChannel: """Single public entry point for the Feishu Channel capability layer. @@ -395,6 +417,13 @@ def __init__( .app_secret(cfg.app_secret) .domain(cfg.domain) .log_level(cfg.log_level) + # Required for `RequestOption.user_access_token` to have any effect + # at all: with it off, `core.token.auth.verify` skips manual tokens + # entirely, so a request declaring the user identity gets a freshly + # minted tenant token instead and the user's authorization becomes + # decoration. Additive — a request that carries no manual token + # still mints from app credentials exactly as before. + .enable_set_token(True) .timeout(cfg.transport.http_timeout_seconds) .proxy_url(cfg.transport.proxy_url) .trust_env_proxy(cfg.transport.trust_env_proxy) @@ -495,6 +524,26 @@ def __init__( self._bg_tasks_lock = threading.Lock() self._shutdown = threading.Event() + self._raw_events = RawEventRegistry( + schedule=self.schedule, report=self._report_raw_error + ) + self._meeting = _meeting_registry.MeetingChannel( + client=self._client, + config=self._config, + # A dedicated cache: platform event ids are global, so sharing the + # message layer's key space would let one layer's mark swallow the + # other layer's event. + seen=SeenCache( + cache=safety_cache, + namespace=_meeting_dedup.NAMESPACE, + ), + bot_open_id_getter=self._meeting_bot_open_id, + schedule=self.schedule, + resolve_ticket_interactive=self._meeting_ticket_interactive, + resolve_ticket_quiet=self._meeting_ticket_quiet, + emit_invited=self._emit_meeting_invited, + timeout_seconds=self._config.transport.http_timeout_seconds or 30.0, + ) self._stop_requested = False self._started = False self._lifecycle_lock = threading.Lock() @@ -758,6 +807,13 @@ def _mark_ready(self) -> None: ev.set() except Exception: # pragma: no cover pass + # One of the three reconciliation points: a seat stranded by an + # inconclusive departure before the connection dropped is exactly what a + # fresh connection wants back. + try: + self._meeting.on_connected() + except Exception as e: # pragma: no cover - never block readiness + logger.debug("channel: meeting reconcile on connect skipped: %s", e) def _ensure_ready_event(self) -> "asyncio.Event": """Lazily create the asyncio.Event so we don't need a running loop in __init__.""" @@ -861,7 +917,14 @@ async def _wait_background_start_ready( await asyncio.sleep(0.05) async def disconnect(self) -> None: - """Gracefully drain safety pipeline batches + stop the WS loop.""" + """Gracefully drain safety pipeline batches + stop the WS loop. + + Meeting sessions are **disposed, not left**: a reconnect must not make + the bot walk out of every meeting it is in. The bot therefore stays a + participant, which is why those seats stay counted and why a process + that is really exiting should ``leave()`` first. + """ + await self._dispose_meeting_sessions() if self._safety is not None: try: await self._safety.dispose() @@ -995,13 +1058,20 @@ def stop(self, *, join_timeout: float = 5.0) -> None: 1. Signal shutdown (sets ``self._shutdown``). 2. Stop the WS client if one was created. - 3. Cancel in-flight futures returned from :meth:`schedule`. - 4. Run ``DeviceFlowClient.close()`` on the bg loop to release httpx. - 5. Stop the bg loop and join its thread. + 3. Dispose meeting sessions so their tasks stop before the loop does. + 4. Cancel in-flight futures returned from :meth:`schedule`. + 5. Run ``DeviceFlowClient.close()`` on the bg loop to release httpx. + 6. Stop the bg loop and join its thread. """ if self._shutdown.is_set(): return self._shutdown.set() + # Before the loop goes away: a session owns tasks and timers created + # directly on it, which `_cancel_bg_tasks` does not know about. Left + # running they are destroyed mid-flight when the loop closes, and the + # resulting "Task was destroyed but it is pending" lands wherever the + # process happens to be logging at the time. + self._dispose_meeting_sessions_blocking() if self._start_future is not None: try: self._start_future.cancel() @@ -1264,7 +1334,12 @@ def _start_bot_identity_retry_loop(self) -> None: future = self._bot_identity_retry_future if future is not None and not future.done(): return - if self._bg_loop is None: + # `run_coroutine_threadsafe` accepts a callback for a loop that has + # stopped but is not yet closed, and that callback never runs — leaving + # the coroutine built below unawaited, and a future here that never + # completes. Retrying on a loop that is not running has nothing to do + # anyway, so check before building anything. + if self._bg_loop is None or not self._bg_loop.is_running(): return coro = self._bot_identity_retry_loop() try: @@ -1286,9 +1361,101 @@ def _cancel_bg_tasks(self) -> None: pass self._drain_cancelled_bg_tasks() + @staticmethod + def _sweep_bg_loop_tasks(loop: Any, *, timeout: float = 5.0) -> None: + """Cancel every task left on ``loop`` and let the cancellations land. + + `timeout` bounds how long cancellations are given to converge. It is + generous because overshooting only costs a slow shutdown, while + undershooting downgrades "every task converged" to "some tasks were + abandoned" on a loaded machine — which surfaces later as "Task was + destroyed but it is pending" from whatever happens to be running then, + the least useful place to see it. + + The bound is enforced *inside* the loop rather than by waiting across + threads, because a task that never accepts its cancellation must not be + able to strand this sweep. Enforcing it from here instead would leave + the sweep's own task pending on a loop about to be stopped — the exact + residue this function exists to prevent. + """ + + async def _sweep() -> None: + current = asyncio.current_task() + pending = [ + task + for task in asyncio.all_tasks() + if task is not current and not task.done() + ] + for task in pending: + task.cancel() + if pending: + # `wait`, not `gather`: it returns when the budget runs out + # instead of hanging on whatever refuses to converge, so this + # coroutine always finishes and can never become the orphan. + await asyncio.wait(pending, timeout=timeout) + + if not loop.is_running(): + return + + # Deliberately not `run_coroutine_threadsafe`: it ties the task to a + # `concurrent.futures.Future` through `_chain_future`, whose cancel + # callback re-enters `loop.call_soon_threadsafe`. During shutdown the + # loop is usually closed by then, and that raises *inside* a + # `concurrent.futures` callback — which that module logs itself, out of + # reach of any `try` here. Scheduling by hand keeps both ownership of + # the coroutine and delivery of the cancellation in code we control. + state = {} + finished = threading.Event() + + def _schedule() -> None: + # Built here, on the loop's own thread, so a callback the loop + # never gets around to running leaves no coroutine behind. + coro = _sweep() + try: + task = loop.create_task(coro) + except RuntimeError: + # A loop that is already closing refuses new tasks, and the + # coroutine is still ours at that point. + coro.close() + finished.set() + return + state["task"] = task + task.add_done_callback(lambda _task: finished.set()) + + try: + loop.call_soon_threadsafe(_schedule) + except RuntimeError: + return # the loop closed; nothing was scheduled and nothing leaked + + # The sweep bounds itself by `timeout`, so this only expires when the + # loop stops servicing callbacks at all. One extra second covers the + # scheduling hop rather than racing it. + if finished.wait(timeout + 1.0): + return + + task = state.get("task") + if task is None: + return # the loop never ran `_schedule`, so there is no task + logger.debug( + "FeishuChannel.stop: bg loop stopped servicing callbacks within " + "%.1fs; cancelling the sweep", + timeout + 1.0, + ) + try: + loop.call_soon_threadsafe(task.cancel) + except RuntimeError: + pass # the loop closed, taking the task with it + def _stop_bg_loop(self, *, join_timeout: float) -> None: loop = self._bg_loop thread = self._bg_thread + if loop is not None: + # Anything still pending on this loop is about to be destroyed + # mid-flight, which asyncio reports as "Task was destroyed but it is + # pending". Collaborators create tasks directly on the loop + # (delivery queues, polling loops), so `_cancel_bg_tasks` does not + # see them. Cancel whatever is left and give it one turn. + self._sweep_bg_loop_tasks(loop) if loop is not None: try: loop.call_soon_threadsafe(loop.stop) @@ -1438,15 +1605,48 @@ def _build_dispatcher(self) -> EventDispatcherHandler: # the legacy callback channel uses p1 (event has ``uuid``), but # the modern WS frontier wraps the same event in a p2 envelope # (``schema=2.0``). Register the customized-event handler under - # both so neither path logs "processor not found". + # both so neither entry point logs "processor not found". b = b.register_p1_customized_event( "drive.notice.comment_add_v1", self._on_p1_comment_add ) b = b.register_p2_customized_event( "drive.notice.comment_add_v1", self._on_p1_comment_add ) + problems = self._register_meeting_events(b) + # Raw subscriptions go on last, so they can compose with whatever the + # built-ins above installed. The whole table is rebuilt on every + # start(), so this replay is what keeps them alive across a restart. + replay_problem = self._raw_events.apply(b) + problem = problems or replay_problem + self._meeting.mark_registration(ok=not problems, reason=problem) return b.build() + def _register_meeting_events(self, builder) -> Optional[str]: + """Install the three ``vc.bot.*`` processors. Returns a failure note. + + Not fatal: an application that has not declared these subscriptions in + the developer console still wants its message path to start. The health + readout is where the failure becomes visible. + """ + wiring = ( + (_meeting_registry.ACTIVITY_EVENT, self._meeting.on_activity), + (_meeting_registry.INVITED_EVENT, self._meeting.on_invited), + (_meeting_registry.ENDED_EVENT, self._meeting.on_ended), + ) + for event_type, handler in wiring: + try: + builder.register_p2_customized_event( + event_type, _meeting_payload(handler) + ) + except Exception as e: + logger.warning( + "channel: could not subscribe to %s (%s)", + event_type, + type(e).__name__, + ) + return "%s: %s" % (event_type, type(e).__name__) + return None + # ------------------------------------------------------------------ # Raw sync entry points — schedule async work on the bg loop # ------------------------------------------------------------------ @@ -2983,6 +3183,214 @@ async def _fetch_message_payload(self, message_id: str) -> Dict[str, Any]: # ------------------------------------------------------------------ # UAT (user access token) — exposed for callers that need it explicitly # ------------------------------------------------------------------ + # ------------------------------------------------------------------ + # Meeting channel + # ------------------------------------------------------------------ + async def follow_my_meeting( + self, + *, + user_open_id: str, + prompt_context: Any = None, + meeting_no: Optional[str] = None, + options: Optional[MeetingOptions] = None, + ) -> Any: + """Follow the meeting ``user_open_id`` is currently in, without joining — + and read the trust boundary on ``user_open_id`` and ``prompt_context`` + before the parameters, because this SDK cannot enforce either of them. + + ``user_open_id`` **must** be somebody the caller has already + established is the requester, and ``prompt_context`` must belong to that + same person. This SDK cannot check either: it receives a string, the + ticket store is shared per process, and a cached ticket resolves without + notifying its owner — so passing user-controlled input here listens in + on somebody else's meeting with their authorization, invisibly. Pairing + one person's ``user_open_id`` with another's ``prompt_context`` sends the + authorization card to the wrong person and files the resulting ticket + under the first. ``meeting.follow_allowlist`` is the available gate. + + The bot is not a participant and is not visible in the meeting, so there + is no way to speak into it — respond over IM instead. Telling + participants and obtaining consent is the integrating application's + responsibility. + + Unlike :meth:`join_meeting` this does not need ``connect()``: the path + is REST plus a user ticket, and opening a socket for it is pure + overhead. + """ + return await self._on_bg_loop( + self._meeting.follow_my_meeting( + user_open_id=user_open_id, + prompt_context=prompt_context, + meeting_no=meeting_no, + options=options, + ) + ) + + async def join_meeting( + self, + meeting_no: str, + *, + password: Optional[str] = None, + call_id: Optional[str] = None, + options: Optional[MeetingOptions] = None, + ) -> Any: + """Put the bot in a meeting as a participant. Needs ``connect()``. + + ``password`` is a credential and is passed to the platform and nowhere + else — it does not reach logs, ``raw`` payloads or error objects. + + Remember that ``dispose()`` does **not** leave the meeting, so a + reconnect leaves the bot where it is; only an explicit ``leave()`` + removes it. + """ + try: + return await self._on_bg_loop( + self._meeting.join_meeting( + meeting_no, + connected=bool(self.is_ready or self._started), + password=password, + call_id=call_id, + options=options, + ) + ) + finally: + # Dropped from this frame's locals. A wrong password is the ordinary + # way this call fails, and on the failing path this frame is part of + # the raised error's `__traceback__` — where a crash reporter that + # reads frame locals would find it. + password = None + + def get_meeting_event_health(self) -> MeetingEventHealth: + """Diagnostics for the in-meeting event path. + + Failures here are silent by nature — an undeclared subscription, a + missing permission and a renamed field all look like "nothing + happened". ``received`` versus ``stats[type].empty`` separates "the + platform never sent it" from "it sent it and we could not read it". + """ + return self._meeting.health() + + def on_raw_event(self, event_type: str, handler: Callable) -> Unsubscribe: + """Subscribe to a Feishu event type the channel has not wrapped. + + Multicast; returns an unsubscribe. Pass the event type **without** a + schema prefix, e.g. ``"vc.bot.meeting_started_v1"``. + + Distinct from the ``"raw"`` event: that one mirrors already-wrapped + events and is controlled by ``inbound.emit_raw_events``. This one + subscribes to types the channel does not wrap and ignores that switch. + + The payload is authentic — this runs after signature verification and + decryption — but **unredacted**, and this path sits **outside** the + safety pipeline: no policy gate, no dedup, no processing lock, no loop + guard. Subscribing to a type the channel already handles therefore opens + an unpoliced path into that type: with ``dm_policy="allowlist"`` set, + a raw subscription to ``im.message.receive_v1`` still receives direct + messages from everybody, and redelivered events run the handler again. + """ + subscription = self._raw_events.subscribe(event_type, handler) + dispatcher = self._dispatcher + if dispatcher is not None: + # Installed onto the dispatcher **in place**. Replacing + # `self._dispatcher` with a freshly built one would only work for + # consumers that re-read the attribute on every event; anything + # holding the instance it was given — which is what a transport + # does — would not see this subscription until the next `start()`. + self._raw_events.install(dispatcher) + return subscription + + def _dispose_meeting_sessions_blocking(self, *, timeout: float = 10.0) -> None: + """Dispose meeting sessions from a synchronous teardown path. + + Bounded by the sessions' own drain deadlines rather than by this + number, which only has to be larger than theirs. Sessions drain + concurrently, so the floor is one session — but that one is + ``dispose_drain_timeout_seconds`` **twice**: once waiting for its + loops to unwind and once for the delivery queue. With the 5s default + that is 10s — exactly this budget, with nothing to spare. So raising + that configuration, or lowering this number, times out here; lowering + the configuration is safe. Timing out abandons the remaining sessions' + tasks to the loop shutdown below. + """ + loop = self._bg_loop + if loop is None or not loop.is_running(): + self._meeting.dispose_all() + return + try: + future = asyncio.run_coroutine_threadsafe( + self._dispose_meeting_sessions_inner(), loop + ) + future.result(timeout=timeout) + except Exception as e: # pragma: no cover - teardown must not raise + logger.warning( + "FeishuChannel.stop: disposing meeting sessions failed: %s", + type(e).__name__, + ) + + async def _dispose_meeting_sessions(self) -> None: + """Stop local meeting work on the loop that owns it.""" + if self._bg_loop is None: + self._meeting.dispose_all() + return + try: + await self._on_bg_loop(self._dispose_meeting_sessions_inner()) + except Exception as e: # pragma: no cover - teardown must not raise + logger.warning("channel: disposing meeting sessions failed: %s", e) + + async def _dispose_meeting_sessions_inner(self) -> None: + sessions = self._meeting.dispose_all() + await self._meeting.drain_sessions(sessions) + + async def _on_bg_loop(self, coro): + """Run ``coro`` on the channel's background loop and await its result. + + Everything a session owns — the delivery queue, the polling tasks, the + debounce timers — has to live on **one** loop, and it has to be the loop + the dispatcher schedules onto. Building them on whatever loop happened + to call ``join_meeting()`` puts an ``asyncio.Queue``'s consumer on one + loop and its producer on another, and a put there simply never wakes the + getter: events are accepted and silently never delivered. + """ + self._ensure_bg_loop() + # Deliberately not `schedule()`: that one logs anything the coroutine + # raises, and the caller of this method already receives the exception. + # A wrong meeting password would otherwise be reported twice, once as an + # unhandled background failure. + return await _await_on_loop(self._bg_loop, lambda: coro) + + def _meeting_bot_open_id(self) -> Optional[str]: + identity = self._bot_identity + return getattr(identity, "open_id", None) if identity is not None else None + + async def _meeting_ticket_interactive( + self, *, user_open_id: str, prompt_context: Any + ) -> str: + uat = await require_user_auth( + device_flow=self._device_flow, + token_store=self._token_store, + uat_config=self._config.uat, + user_open_id=user_open_id, + scopes=[_MEETING_EVENT_SCOPE], + context=prompt_context, + ) + return uat.access_token + + async def _meeting_ticket_quiet(self, user_open_id: str) -> str: + """Per-round ticket lookup: cache and refresh only, never a prompt.""" + uat = await resolve_user_auth_non_interactive( + device_flow=self._device_flow, + token_store=self._token_store, + uat_config=self._config.uat, + user_open_id=user_open_id, + ) + return uat.access_token + + async def _emit_meeting_invited(self, event: Any) -> None: + await self._invoke("meetingInvited", event) + + def _report_raw_error(self, exc: BaseException) -> Any: + return self._invoke("error", exc) + async def require_user_auth( self, user_open_id: str, @@ -2993,14 +3401,36 @@ async def require_user_auth( """Resolve a user access token for ``user_open_id``, running the device flow if needed. ``prompt_context`` must expose ``respond(card)`` if the user needs a prompt card (usually the - original interaction carrier).""" - return await require_user_auth( - device_flow=self._device_flow, - token_store=self._token_store, - uat_config=self._config.uat, - user_open_id=user_open_id, - scopes=scopes, - context=prompt_context, + original interaction carrier). + + Runs on the channel's background loop, so every ticket resolution in the + process contends on one per-user lock — including the meeting channel's + per-round lookups. Those locks are ``asyncio.Lock``s bound to the loop + that created them, so callers spread across loops would get no mutual + exclusion at all, and the loser of a concurrent refresh has its still + valid ticket deleted. + + Two consequences worth knowing: ``prompt_context.respond`` is invoked + from that loop's thread, so an object bound to a different loop will not + work; and a process that only ever calls this method still gets the + channel's background thread. + """ + # Routed onto the background loop so every ticket resolution in this + # process — this one, and the meeting channel's per-round lookups — + # contends on the same per-user lock. Those locks are `asyncio.Lock`s + # bound to the loop that created them, so callers spread across loops + # get no mutual exclusion at all; funnelling through one loop restores + # it, which matters because the loser of a concurrent refresh has its + # (still valid) ticket deleted. + return await self._on_bg_loop( + require_user_auth( + device_flow=self._device_flow, + token_store=self._token_store, + uat_config=self._config.uat, + user_open_id=user_open_id, + scopes=scopes, + context=prompt_context, + ) ) def _track_sent_message(self, message_id: str) -> None: diff --git a/lark_channel/channel/config.py b/lark_channel/channel/config.py index ce94dda..d8de07c 100644 --- a/lark_channel/channel/config.py +++ b/lark_channel/channel/config.py @@ -487,6 +487,96 @@ def _security_audit_recorder_default(): # --------------------------------------------------------------------------- +@dataclass +class MeetingChannelConfig: + """Meeting-channel knobs. + + Session creation is triggered from **outside** the application — joining + starts at an invitation from anybody who can add the bot to a meeting, and + following typically starts at an inbound instruction. Both therefore need a + ceiling and a reclamation story, and neither is optional. + """ + + #: Concurrent meeting sessions allowed. Over the ceiling ``join_meeting`` + #: and ``follow_my_meeting`` raise ``too_many_sessions`` **without** + #: calling the platform. The reading covers live local sessions plus + #: meetings the bot has joined and not yet left, deduplicated by meeting. + max_concurrent_sessions: int = 32 + + #: Reclaim a ``tat`` session after this long with no in-meeting activity: + #: end it, **depart the meeting**, and give the seat back. ``0.0`` + #: disables it, which is the default because the liveness probe already + #: catches the bot being removed, and probe backfill keeps resetting this + #: clock while the bot really is present. What is left for idle + #: reclamation is a meeting where nobody is talking — departing there is a + #: visible and wrong move. + #: + #: Idle reclamation doubles as the only backstop for a wedged handler. With + #: it off there is none, so applications whose handlers may block for a + #: long time should set a positive value. + idle_timeout_seconds: float = 0.0 + + #: How often to confirm the bot is still in the meeting. Covers the cases + #: that produce no meeting-ended event at all: removed by a host, meeting + #: transferred. ``0`` disables. + liveness_probe_interval_seconds: float = 300.0 + + #: Default transcript settle window, in seconds, for sessions that do not + #: pass their own :class:`~.meeting.MeetingOptions`. ``0.0`` delivers every + #: revision of a sentence; a positive value delivers each sentence once, + #: after it stops changing for that long. + stabilize_seconds: float = 0.0 + + #: Per-session ceiling on in-meeting messages, per minute. Over it, + #: ``send_message`` raises ``rate_limited``. Guards against a self-echo + #: feedback loop turning into in-meeting flooding at network speed. + send_rate_limit_per_minute: int = 20 + + #: ``follow_my_meeting`` empty-poll backoff: first interval and its ceiling. + poll_min_interval_seconds: float = 3.0 + poll_max_interval_seconds: float = 10.0 + #: ``follow_my_meeting`` failure backoff ceiling. Counted separately from + #: empty polls: a failing token must not keep retrying on that cadence. + poll_failure_max_interval_seconds: float = 60.0 + #: Consecutive failures after which the event source gives up. + poll_max_consecutive_failures: int = 10 + #: ``follow_my_meeting``, how often to re-check the meeting is still active. + active_meeting_check_interval_seconds: float = 30.0 + + #: Upper bound on waiting for the delivery queue to drain during + #: reclamation. Past it the queue is cancelled and a warning is logged. + #: Seat return and unregistration never wait for this — otherwise one + #: wedged handler would burn a seat permanently. + dispose_drain_timeout_seconds: float = 5.0 + + #: Fallback deadline on a membership entry, in seconds. Reaching it means + #: our accounting is wrong; releasing the seat beats locking the process + #: out. Two hours rather than a full day because the cost of holding is + #: that both entry points stop working. + membership_max_age_seconds: float = 7200.0 + #: Minimum gap between two reconciliation attempts on the same entry. + #: Reconciliation is lazy — driven by admission, by arriving evidence, and + #: by ``connect()`` — not by a timer, so this throttles rather than + #: schedules. ``0`` disables reconciliation, leaving only the deadline. + membership_reconcile_interval_seconds: float = 60.0 + #: Bounds on reconciliation performed inline on an admission path, so it + #: cannot stretch ``join``/``follow`` latency without limit. + membership_reconcile_max_concurrency: int = 4 + membership_reconcile_attempt_timeout_seconds: float = 3.0 + + #: ``follow_my_meeting`` open_id allowlist. ``None`` accepts any open_id. + #: The SDK cannot verify that the supplied open_id belongs to whoever + #: triggered the call, so passing user-controlled input straight through + #: means listening in on somebody else's meeting with their ticket. + follow_allowlist: Optional[List[str]] = None + + #: Inviter open_id allowlist for ``meeting_invited_v1``. ``None`` accepts + #: invitations from anybody — the default, because a closed default would + #: make the feature unusable out of the box. This path does **not** pass + #: through ``PolicyConfig``. + invite_allowlist: Optional[List[str]] = None + + @dataclass class ChannelConfig: """Top-level configuration for :class:`FeishuChannel`. @@ -538,3 +628,7 @@ class ChannelConfig: # match the requested ``id_type``. Results flow through the internal roster # cache. (Typed ``Any`` to avoid an import cycle with ``types``.) resolve_chat_members: Optional[Callable[..., Any]] = None + + # Meeting channel. Appended last so the existing positional field + # order (locked by test_security_config) stays stable. + meeting: MeetingChannelConfig = field(default_factory=MeetingChannelConfig) diff --git a/lark_channel/channel/errors.py b/lark_channel/channel/errors.py index 52f33a5..9943ec9 100644 --- a/lark_channel/channel/errors.py +++ b/lark_channel/channel/errors.py @@ -1,6 +1,6 @@ """Channel error types and classification. -Single canonical enum: `FeishuChannelErrorCode` — 10 values covering the +Single canonical enum: `FeishuChannelErrorCode` — 13 values covering the taxonomy of failures surfaced by the outbound / inbound pipelines. """ @@ -10,7 +10,7 @@ class FeishuChannelErrorCode(str, Enum): - """Channel-layer error taxonomy (10 canonical values).""" + """Channel-layer error taxonomy (13 canonical values).""" FORMAT_ERROR = "format_error" TARGET_REVOKED = "target_revoked" @@ -22,6 +22,16 @@ class FeishuChannelErrorCode(str, Enum): SEND_TIMEOUT = "send_timeout" NOT_CONNECTED = "not_connected" UNKNOWN = "unknown" + # Appended for the meeting channel. Existing values are unchanged, and + # `classify_error` keeps its current mapping: none of these three come + # from a Feishu error code, they are all local decisions. + #: The operation does not exist in this session's mode — `send_message` + #: on a `uat` session, where the bot is not a participant. + NOT_SUPPORTED = "not_supported" + #: No active meeting to follow, or the target meeting is gone. + MEETING_NOT_FOUND = "meeting_not_found" + #: The concurrent-session ceiling is reached. + TOO_MANY_SESSIONS = "too_many_sessions" @dataclass diff --git a/lark_channel/channel/events.py b/lark_channel/channel/events.py index a03cf22..28c254a 100644 --- a/lark_channel/channel/events.py +++ b/lark_channel/channel/events.py @@ -38,6 +38,7 @@ "reject", "comment", "raw", + "meetingInvited", "reconnecting", "reconnected", "error", @@ -63,6 +64,7 @@ class Events: REJECT = "reject" COMMENT = "comment" RAW = "raw" + MEETING_INVITED = "meetingInvited" RECONNECTING = "reconnecting" RECONNECTED = "reconnected" ERROR = "error" diff --git a/lark_channel/channel/meeting/__init__.py b/lark_channel/channel/meeting/__init__.py new file mode 100644 index 0000000..bb4db63 --- /dev/null +++ b/lark_channel/channel/meeting/__init__.py @@ -0,0 +1,53 @@ +"""Meeting channel: agents that perceive and respond inside a live meeting. + +Two entry points, one session type: + +* :meth:`~..channel.FeishuChannel.follow_my_meeting` follows the meeting a user + is currently in, under that user's own authorization. The bot is **not** a + participant and is not visible; the only way to respond is a direct message. +* :meth:`~..channel.FeishuChannel.join_meeting` puts the bot in the meeting as + a real participant, so it can also speak into the meeting chat. + +Both return a :class:`~.session.MeetingSession` with the same event stream, so +moving from one to the other changes the entry-point line and nothing else. +""" + +from .session import MeetingSession +from .types import ( + ActivityTypeStats, + DocumentContextEvent, + LivenessHealth, + MeetingActor, + MeetingChatEvent, + MeetingEndEvent, + MeetingEventBase, + MeetingEventHealth, + MeetingEvents, + MeetingInvitedEvent, + MeetingOptions, + MembershipHealth, + ParticipantEvent, + ShareDocInfo, + ShareEvent, + TranscriptEvent, +) + +__all__ = [ + "ActivityTypeStats", + "DocumentContextEvent", + "LivenessHealth", + "MeetingActor", + "MeetingChatEvent", + "MeetingEndEvent", + "MeetingEventBase", + "MeetingEventHealth", + "MeetingEvents", + "MeetingInvitedEvent", + "MeetingOptions", + "MeetingSession", + "MembershipHealth", + "ParticipantEvent", + "ShareDocInfo", + "ShareEvent", + "TranscriptEvent", +] diff --git a/lark_channel/channel/meeting/admission.py b/lark_channel/channel/meeting/admission.py new file mode 100644 index 0000000..22428e0 --- /dev/null +++ b/lark_channel/channel/meeting/admission.py @@ -0,0 +1,71 @@ +"""The gate in front of both entry points. + +Sessions are created from **outside** the process: joining starts at an +invitation from anybody who can add the bot to a meeting, following typically at +an inbound instruction. So the ceiling is not a tuning knob, it is the only +thing standing between a hostile-or-clumsy caller and unbounded growth of +sessions, timers, settle buffers and dedup entries. + +Both entry points share it, and the reading covers **live local sessions plus +server-side participation**. Counting only live sessions makes it drop to zero +after a ``dispose()`` while the bot is still sitting in the meeting; counting +only participation misses follow sessions, which take no seat server-side and +whose reclamation depends on nothing but the gate. + +Reclamation runs here rather than on a timer: this is the moment a seat is +actually wanted, and it needs no resource that outlives ``disconnect()``. +""" + +import asyncio +from typing import Any, Callable, Dict, Optional, Set + +from ..errors import FeishuChannelError, FeishuChannelErrorCode + + +class AdmissionGate: + """Decides whether another meeting session may be created.""" + + def __init__( + self, + *, + max_concurrent_sessions: int, + live_meetings: Callable[[], Set[str]], + held_meetings: Callable[[], Set[str]], + reconcile: Callable[[], Any], + ) -> None: + self._ceiling = max_concurrent_sessions + self._live_meetings = live_meetings + self._held_meetings = held_meetings + self._reconcile = reconcile + self._joining: Dict[str, "asyncio.Future"] = {} + + async def admit(self) -> None: + """Reclaim what can be reclaimed, then compare against the ceiling.""" + await self._reconcile() + occupied = self._live_meetings() | self._held_meetings() + if len(occupied) >= self._ceiling: + raise FeishuChannelError( + FeishuChannelErrorCode.TOO_MANY_SESSIONS, + "the meeting session ceiling (%d) is reached" % self._ceiling, + ) + + def joining(self, meeting_no: str) -> Optional["asyncio.Future"]: + """The in-flight join for ``meeting_no``, if there is one.""" + return self._joining.get(meeting_no) + + def claim(self, meeting_no: str) -> "asyncio.Future": + """Register an in-flight join. **Call before the first await.** + + Claiming after the admission check leaves a window where two concurrent + calls both pass it, both call the platform, and the loser's session is + evicted from the routing table with its probe loop still running. + """ + future: "asyncio.Future" = asyncio.get_event_loop().create_future() + self._joining[meeting_no] = future + return future + + def release(self, meeting_no: str) -> None: + self._joining.pop(meeting_no, None) + + +__all__ = ["AdmissionGate"] diff --git a/lark_channel/channel/meeting/api.py b/lark_channel/channel/meeting/api.py new file mode 100644 index 0000000..8ed18f9 --- /dev/null +++ b/lark_channel/channel/meeting/api.py @@ -0,0 +1,218 @@ +"""Executing meeting requests, and reporting what came back. + +The result object exists because two callers need the platform's error code +rather than an exception: the liveness probe has to tell ``120004`` (the bot +is not a participant) from ``120003`` (a *user* is not) and from everything +else, and membership accounting has to tell "the meeting is gone" from "we do +not know". Raising would flatten all of that into one class. + +Credential handling, in one place: + +* every call builds a **fresh** ``BaseRequest`` and ``RequestOption``. The + transport writes ``Authorization`` back onto the request object it was given, + so a reused request carries the previous call's identity into the next one + and keeps a token reachable on a long-lived object. +* the original transport exception is never stored or re-raised — it keeps the + outgoing headers reachable. Only its type name survives, for diagnosis. +* the request and option are **scrubbed after the call**. A raised error carries + ``__traceback__``, and every frame in it exposes its locals — so a request + object still holding ``Authorization`` or a meeting password is reachable from + the error object that a crash reporter walks, even though ``repr()`` is clean. +""" + +import asyncio +from typing import Any, Dict, Optional + +from lark_channel.core.http.transport import Transport +from lark_channel.core.json import JSON +from lark_channel.core.model import RequestOption +from lark_channel.core.token import auth as _token_auth + +from ..errors import FeishuChannelError +from .errors import _first_console_url, build_api_error + + +#: Request-body fields that are credentials in their own right. +_CREDENTIAL_BODY_FIELDS = ("password",) +#: Every credential slot on a ``RequestOption``. +_CREDENTIAL_OPTION_FIELDS = ( + "user_access_token", + "tenant_access_token", + "app_access_token", + "app_ticket", +) + + +def _scrub_credentials(request: Any, option: Any) -> None: + """Remove credentials from a spent request and its options. + + Both objects are single-use, and both are locals of the frames an error + unwinds through — which means they stay reachable through the raised + error's ``__traceback__``. Clearing them here is the one place that covers + every caller, including the ones that raise. + """ + headers = getattr(request, "headers", None) + if isinstance(headers, dict): + for key in list(headers): + if key.lower() == "authorization": + headers.pop(key, None) + body = getattr(request, "body", None) + if isinstance(body, dict): + for field in _CREDENTIAL_BODY_FIELDS: + if field in body: + body[field] = None + # Every token slot, not just the user one: `core.token.auth.verify` writes + # the freshly minted tenant token onto the option too, and both live on an + # object that stays reachable from a raised error's frames. + for slot in _CREDENTIAL_OPTION_FIELDS: + if getattr(option, slot, None): + setattr(option, slot, None) + + +class ApiResult: + """What one meeting request produced. + + ``transport_error`` means the request never got an answer at all, which is + the case that must never be read as "the bot is not in the meeting". + """ + + __slots__ = ( + "ok", + "status", + "feishu_code", + "msg", + "data", + "transport_error", + "console_url", + ) + + def __init__( + self, + *, + ok: bool, + status: Optional[int] = None, + feishu_code: Optional[int] = None, + msg: str = "", + data: Optional[Dict[str, Any]] = None, + transport_error: bool = False, + console_url: Optional[str] = None, + ) -> None: + self.ok = ok + self.status = status + self.feishu_code = feishu_code + self.msg = msg + self.data = data or {} + self.transport_error = transport_error + # Only the already-validated link is carried, never the response body: + # some meeting responses include a plaintext password, and a body kept + # on a result object ends up wherever that object ends up. + self.console_url = console_url + + def __repr__(self) -> str: # pragma: no cover - debugging aid + return "" % ( + self.ok, + self.status, + self.feishu_code, + ) + + +class MeetingApi: + """Runs meeting requests against the channel's client.""" + + def __init__(self, client: Any, *, timeout_seconds: float = 30.0) -> None: + self._client = client + self._timeout = timeout_seconds or 30.0 + + async def call( + self, + request: Any, + *, + user_access_token: Optional[str] = None, + what: str = "meeting request", + ) -> ApiResult: + """Execute ``request``; never raises for an API-level failure. + + ``user_access_token`` is attached to a fresh option for this one call + and referenced nowhere else. It only takes effect because the channel + builds its client with ``enable_set_token(True)`` and because the + request declares exactly one token type. + """ + option = RequestOption() + if user_access_token: + option.user_access_token = user_access_token + try: + resp = await asyncio.wait_for( + self._run(request, option), timeout=self._timeout + ) + except FeishuChannelError: + raise + except Exception as exc: + # Deliberately drops `exc`: an httpx exception keeps the request's + # Authorization header reachable through attribute walks. + return ApiResult( + ok=False, msg=type(exc).__name__, transport_error=True + ) + finally: + _scrub_credentials(request, option) + return self._interpret(resp) + + async def _run(self, request: Any, option: RequestOption) -> Any: + loop = asyncio.get_running_loop() + # Token verification may fetch or refresh a tenant token over the + # network; keep it off the event loop. + await loop.run_in_executor( + None, _token_auth.verify, self._client.config, request, option + ) + return await Transport.aexecute(self._client.config, request, option) + + @staticmethod + def _interpret(resp: Any) -> ApiResult: + status = getattr(resp, "status_code", None) + raw = getattr(resp, "content", None) + body: Dict[str, Any] = {} + if raw: + try: + parsed = JSON.unmarshal(raw.decode("utf-8"), dict) + if isinstance(parsed, dict): + body = parsed + except Exception: + body = {} + code = body.get("code") + feishu_code = int(code) if isinstance(code, int) and code else None + msg = body.get("msg") or "" + http_ok = status is None or 200 <= int(status) < 300 + if feishu_code is None and http_ok: + data = body.get("data") + return ApiResult( + ok=True, + status=status, + msg=msg, + data=data if isinstance(data, dict) else {}, + ) + return ApiResult( + ok=False, + status=status, + feishu_code=feishu_code, + msg=msg, + console_url=_first_console_url(body), + ) + + @staticmethod + def error_for(result: ApiResult, *, what: str) -> FeishuChannelError: + """The error to hand a caller, with credentials already stripped.""" + body: Dict[str, Any] = {} + if result.feishu_code is not None: + body["code"] = result.feishu_code + if result.msg: + body["msg"] = result.msg + if result.console_url: + body["console_url"] = result.console_url + return build_api_error( + what=what, + status=result.status, + body=body, + transport_error=RuntimeError() if result.transport_error else None, + ) + + +__all__ = ["ApiResult", "MeetingApi"] diff --git a/lark_channel/channel/meeting/coerce.py b/lark_channel/channel/meeting/coerce.py new file mode 100644 index 0000000..4e45201 --- /dev/null +++ b/lark_channel/channel/meeting/coerce.py @@ -0,0 +1,91 @@ +"""Identifier and timestamp coercion for meeting payloads. + +Every function here exists because the generated ``vc`` models disagree with +what the platform actually sends on at least one transport. Getting any of +them wrong fails silently — no exception, no log line, just an event stream +that quietly routes nowhere or an echo check that never matches. +""" + +from typing import Any, Optional + +#: Order matters: the bot's own identity is an open_id, so an actor id has to +#: resolve to the same namespace for echo detection to work at all. +_ID_KEYS = ("open_id", "union_id", "user_id") + + +def actor_id(actor: Any) -> Optional[str]: + """The actor's open_id, whichever shape the transport used. + + The generated model declares ``id: str``, and the poll transport does send + a bare string — it pins ``user_id_type=open_id`` in the query. The push + transport takes no such parameter, so it sends every namespace it has: + ``{"open_id": ..., "union_id": ..., "user_id": ...}``. + + An implementation that only reads the string form gets ``None`` for every + pushed actor, and ``self_echo`` then compares the bot's open_id against + nothing for the rest of the process. + """ + if not isinstance(actor, dict): + return None + raw = actor.get("id") + if isinstance(raw, str) and raw: + return raw + if isinstance(raw, dict): + for key in _ID_KEYS: + value = raw.get(key) + if isinstance(value, str) and value: + return value + # Some payloads omit `id` and put the namespaces directly on the actor. + for key in _ID_KEYS: + value = actor.get(key) + if isinstance(value, str) and value: + return value + return None + + +def meeting_id_str(value: Any) -> Optional[str]: + """The long meeting id as a string, whichever type it arrived as. + + ``bots/join`` and ``user_active_meeting`` return it as a string; the push + envelope sends the same value as an ``int``. Python does not compare those + as equal and ``sessions["7654321"]`` does not find ``sessions[7654321]``, + so skipping this normalization drops every pushed event — and dropping + events for an unrecognized meeting is by design, so nothing complains. + """ + if value is None: + return None + if isinstance(value, bool): # bool is an int subclass; never a meeting id + return None + if isinstance(value, str): + return value or None + if isinstance(value, int): + return str(value) + return None + + +def to_ms(value: Any) -> Optional[int]: + """A millisecond timestamp as an int, or ``None`` if it is not one. + + Item-level time fields (``start_time_ms``, ``send_time``, ``join_time``, + ``time``, ...) arrive as strings; meeting-level ones arrive as ints. + Unparsable input yields ``None`` rather than raising: a malformed + timestamp is not a reason to drop a transcript. + """ + if value is None or isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, float): + return int(value) + if isinstance(value, str): + text = value.strip() + if not text: + return None + try: + return int(text) + except ValueError: + try: + return int(float(text)) + except ValueError: + return None + return None diff --git a/lark_channel/channel/meeting/dedup.py b/lark_channel/channel/meeting/dedup.py new file mode 100644 index 0000000..f444477 --- /dev/null +++ b/lark_channel/channel/meeting/dedup.py @@ -0,0 +1,81 @@ +"""Suppressing redeliveries without suppressing distinct content. + +Three levels, all applied at the activity level: + +1. ``event_id`` — present on every polled event, absent from every pushed + activity item. So it catches the poll/push overlap and platform redelivery, + but the push transport relies on level 2. +2. a digest of the content — the identifying tuple plus text, timestamp and + actor. Stands in for a missing ``event_id``. +3. ``sentence_id`` is deliberately **not** a dedup key. It is an upsert handle: + the platform resends a sentence as the speaker keeps talking and the text + grows, so level 2 lets each revision through on its own merits. + +Every key carries the meeting id. Two meetings running at once produce +byte-identical greetings from the same person seconds apart, and a key without +the meeting in it makes one meeting's transcript disappear from the other. +""" + +import hashlib +import json +from typing import Any, Dict, List, Optional + + +#: Same window as the message-layer cache, but a separate namespace: platform +#: event ids are global, so sharing a key space would let one layer's mark +#: swallow the other layer's event. +NAMESPACE = "channel:meeting:seen:" + +def _digest(payload: Any) -> str: + """A fixed-length digest of ``payload``. + + Hashing rather than joining keeps whole transcripts from living in memory + for the meeting's duration, and removes the ambiguity a separator + introduces when the content itself contains it. + """ + encoded = json.dumps(payload, sort_keys=True, ensure_ascii=False, default=str) + return hashlib.sha256(encoded.encode("utf-8")).hexdigest()[:32] + + +def _item_identity(item: Dict[str, Any]) -> Any: + """What makes this item this item. + + The **whole** item, not a hand-picked tuple of identifier, text, timestamp + and actor. That narrower key over-suppresses whenever the distinguishing + detail lives outside it: three ``document_context_changed`` items differing + only in which sub-object they carry share one identifier, one timestamp and + one actor, so the narrow key collapses all three into one — and a dropped + event is silent data loss, strictly worse than a duplicate delivery. + + Redelivery of the same activity is byte-identical, so the full item still + collapses it. What the full item gives up is robustness to the platform + adding a per-delivery volatile field, which would weaken dedup to + "duplicates get through" — the direction that fails loudly and that the + poll path's ``event_id`` covers anyway. + + Items with genuinely thin content still collide: the same person joining + twice within one timestamp's resolution is one entry. That residual is + accepted and documented. + """ + return item + + +def activity_key( + activity: Dict[str, Any], + items: List[Dict[str, Any]], + *, + meeting_id: str, + activity_event_type: str, + event_id: Optional[str] = None, +) -> str: + """The dedup key for one activity object.""" + if event_id: + return "%s|evt|%s" % (meeting_id, event_id) + identities = [_item_identity(item) for item in items] + return "%s|content|%s" % ( + meeting_id, + _digest([activity_event_type, identities]), + ) + + +__all__ = ["NAMESPACE", "activity_key"] diff --git a/lark_channel/channel/meeting/errors.py b/lark_channel/channel/meeting/errors.py new file mode 100644 index 0000000..bdbf1df --- /dev/null +++ b/lark_channel/channel/meeting/errors.py @@ -0,0 +1,156 @@ +"""Error construction, capability-link validation, and log sanitizing. + +Two rules shape this module: + +**Stripping happens where the error is built, not where it is logged.** The +same :class:`FeishuChannelError` that reaches the fallback logger also reaches +the application's ``error`` handler, and applications hand those straight to a +crash reporter — which walks the cause chain and every attribute on it. The +transport exception keeps the outgoing request, headers included, reachable +that way while its ``repr()`` stays clean. So the original exception is never +attached; only the fields needed to tell failures apart survive. + +**Untrusted text is escaped before it can reach a log line.** Passing it as a +lazy logging argument moves it out of the message template but the logging +layer still formats it into the same output line, so a newline in a meeting +title still forges a log entry. +""" + +from typing import Any, Dict, Optional +from urllib.parse import urlsplit + +from ..errors import FeishuChannelError, FeishuChannelErrorCode, classify_api_error +from ..errors import classify_http_status + +#: C0 plus C1. Tab is escaped along with the rest: a log line is a line, and +#: nothing in a meeting payload needs to lay out columns in it. +_CONTROL_ORDINALS = tuple(range(0x00, 0x20)) + tuple(range(0x7F, 0xA0)) +_CONTROL_TRANSLATION = dict((code, "\\x%02x" % code) for code in _CONTROL_ORDINALS) + + +def sanitize_for_log(value: Any) -> str: + """``value`` as a string with every control character escaped. + + Meeting titles, transcripts, chat bodies and document headings are written + by participants, who may be external or guest users. A newline in any of + them forges a log line; an ANSI escape repaints a terminal. + """ + if value is None: + return "" + return str(value).translate(_CONTROL_TRANSLATION) + + +def safe_console_url(value: Any) -> Optional[str]: + """``value`` unchanged if it is a plausible authorization link, else ``None``. + + The link is a signed one-click grant — a capability, and therefore itself a + credential. It is opaque, so it is validated and never rewritten: + re-encoding or reassembling any part of it stops it working. + + Two checks, and nothing else: + + * the scheme is ``https``. The domain this arrives from is configurable, so + the field is not a trusted source, and whoever receives it renders it as + a link — where ``javascript:`` or ``data:`` is script execution. + * there is no userinfo. ``https://open.feishu.cn@elsewhere.example/x`` + passes a scheme check while actually pointing at ``elsewhere.example``, + and the entire purpose of this field is that an administrator clicks it. + + The host itself is *not* checked: self-hosted and proxied deployments make + a host allowlist reject legitimate links. + + Parsing failures count as invalid rather than propagating. This runs while + an error object is being built, and an exception here would replace a real + API failure with a URL-parsing one. + """ + if not isinstance(value, str) or not value: + return None + try: + parts = urlsplit(value) + if parts.scheme != "https": + return None + if parts.username is not None: + return None + except Exception: + return None + return value + + +def _first_console_url(body: Dict[str, Any]) -> Optional[str]: + """The console link, from either place the platform puts it.""" + for candidate in (body, body.get("error") if isinstance(body, dict) else None): + if isinstance(candidate, dict): + found = safe_console_url(candidate.get("console_url")) + if found is not None: + return found + return None + + +def build_api_error( + *, + what: str, + status: Optional[int], + body: Optional[Dict[str, Any]] = None, + transport_error: Optional[BaseException] = None, +) -> FeishuChannelError: + """A :class:`FeishuChannelError` carrying no credentials. + + ``transport_error`` is used only to pick a code and is deliberately **not** + attached as the cause: an ``httpx`` exception keeps the request's + ``Authorization`` header reachable by attribute walk. + """ + body = body if isinstance(body, dict) else {} + feishu_code = body.get("code") or None + msg = body.get("msg") or "" + if feishu_code: + code = classify_api_error(int(feishu_code), msg) + elif status is not None: + code = classify_http_status(status) + else: + code = FeishuChannelErrorCode.UNKNOWN + + context: Dict[str, Any] = {} + if status is not None: + context["status"] = status + if feishu_code: + context["feishu_code"] = int(feishu_code) + console_url = _first_console_url(body) + if console_url is not None: + # Passed through byte for byte; equally a credential, so it is excluded + # from the fallback log and masked by the redaction layer. + context["console_url"] = console_url + + detail = sanitize_for_log(msg) if msg else "" + if transport_error is not None and not detail: + # The exception type, never the exception: its repr is clean but its + # attributes are not. + detail = type(transport_error).__name__ + parts = [what] + if feishu_code: + parts.append("code=%s" % int(feishu_code)) + elif status is not None: + parts.append("status=%s" % status) + if detail: + parts.append(detail) + return FeishuChannelError(code, ": ".join(parts), context=context) + + +def log_error_fields(meeting_id: str, error: FeishuChannelError) -> Dict[str, Any]: + """The minimal shape for the fallback log: no ``context``. + + ``context`` may hold a console link, which is a credential. + """ + return { + "meeting_id": sanitize_for_log(meeting_id), + "code": error.code.value, + "message": sanitize_for_log(error.message), + "feishu_code": (error.context or {}).get("feishu_code"), + } + + +__all__ = [ + "build_api_error", + "log_error_fields", + "safe_console_url", + "sanitize_for_log", +] diff --git a/lark_channel/channel/meeting/health_view.py b/lark_channel/channel/meeting/health_view.py new file mode 100644 index 0000000..6c3372b --- /dev/null +++ b/lark_channel/channel/meeting/health_view.py @@ -0,0 +1,68 @@ +"""Assembling the channel-wide health readout. + +Per-activity-type counters live on the sessions that saw the traffic, but the +readout is channel-level and has to survive the sessions ending — otherwise the +numbers reset every time a meeting finishes, which is exactly when somebody +goes looking at them. +""" + +from typing import Any, Callable, Dict, Iterable, Optional + +from .normalize import MAX_STAT_KEYS, OTHER_STAT_KEY +from .types import ActivityTypeStats, MeetingEventHealth + + +class MeetingHealthView: + """Owns the channel-level health object and folds session stats into it.""" + + def __init__(self, sessions: Callable[[], Iterable[Any]]) -> None: + self._sessions = sessions + self._health = MeetingEventHealth() + self._retired: Dict[str, ActivityTypeStats] = {} + self._retired_dropped = 0 + + @property + def health(self) -> MeetingEventHealth: + """The mutable health object; callers may bump counters on it.""" + return self._health + + def mark_registration(self, *, ok: bool, reason: Optional[str]) -> None: + self._health.registered = ok + self._health.reason = reason + + def retire(self, session: Any) -> None: + """Fold a departing session's counters in before it disappears. + + Bounded the same way the per-session map is: these keys come from the + server and this map lives for the whole process, so the per-session + ceiling alone would not hold it. + """ + for key, stats in session.get_stats().items(): + self._merge(self._retired, key, stats) + self._retired_dropped += session.dropped_deliveries + + def snapshot(self) -> MeetingEventHealth: + aggregate: Dict[str, ActivityTypeStats] = {} + dropped = self._retired_dropped + for session in self._sessions(): + dropped += session.dropped_deliveries + for key, stats in session.get_stats().items(): + self._merge(aggregate, key, stats) + for key, stats in self._retired.items(): + self._merge(aggregate, key, stats) + self._health.stats = aggregate + self._health.dropped = dropped + return self._health + + @staticmethod + def _merge( + target: Dict[str, ActivityTypeStats], key: str, stats: ActivityTypeStats + ) -> None: + if key not in target and len(target) >= MAX_STAT_KEYS: + key = OTHER_STAT_KEY + merged = target.setdefault(key, ActivityTypeStats()) + merged.received += stats.received + merged.empty += stats.empty + + +__all__ = ["MeetingHealthView"] diff --git a/lark_channel/channel/meeting/liveness.py b/lark_channel/channel/meeting/liveness.py new file mode 100644 index 0000000..1c197a9 --- /dev/null +++ b/lark_channel/channel/meeting/liveness.py @@ -0,0 +1,103 @@ +"""Confirming the bot is still a participant. + +Being removed by a host, or a meeting being transferred, produces **no** +meeting-ended event at all. Without a probe those sessions live until the idle +timeout — which is off by default — so this is the mechanism that makes +reclamation work in the ordinary case. + +The probe reuses ``bots/events`` with the app's own credential. That endpoint +takes either identity, and under the app identity it needs exactly the scope +``bots/join`` already needs: being able to join implies being able to probe, so +this costs no new credential and no new authorization. The same call also +backfills whatever the push transport missed, so one request does two jobs. + +**Fail open.** Only a request that succeeded *and* said the bot is absent ends +a session. Everything else is "unknown". Probes for every session run on the +same schedule, so their failures are correlated — one network blip or one +missing scope would otherwise end every live session in a single tick. +""" + +import time +from typing import Any, Dict, List, Optional, Tuple + +from lark_channel.api.vc.bot import build_bot_events_request_as_app + +from .api import MeetingApi +from .types import LivenessHealth + +#: ``120004`` is a statement about the *bot*: it is not in this meeting. +NOT_IN_MEETING_CODES = frozenset({120004}) +#: The evidence key recorded when a probe establishes absence. Pinned rather +#: than derived from the set above: it is part of the health readout's public +#: shape, and the set is explicitly open to gaining more codes. +ABSENCE_EVIDENCE = "120004" +#: ``120003`` is the same HTTP status about a *user*. Treating it as the bot's +#: departure ends live sessions — in follow mode it is not even about the bot. +USER_NOT_IN_MEETING_CODES = frozenset({120003}) + +#: The endpoint rejects anything below twenty during field validation, so a +#: probe asking for a single item never learns anything about anything. +MIN_PAGE_SIZE = 20 + +IN_MEETING = "in_meeting" +NOT_IN_MEETING = "not_in_meeting" +UNKNOWN = "unknown" + + +class LivenessProbe: + """One place where "is the bot still in this meeting" is decided. + + Kept as its own object so the verdict logic has a single home: what the + endpoint returns when its documented precondition is violated was only + established by running it, and the next surprise should be a change to one + class. + """ + + def __init__(self, api: MeetingApi, health: LivenessHealth) -> None: + self._api = api + self._health = health + + async def probe( + self, *, meeting_id: str, page_token: Optional[str] + ) -> Tuple[str, List[Dict[str, Any]], Optional[str]]: + """``(verdict, backfilled events, next page token)``.""" + request = build_bot_events_request_as_app( + meeting_id=meeting_id, + page_token=page_token, + page_size=MIN_PAGE_SIZE, + ) + result = await self._api.call(request, what="meeting liveness probe") + if result.ok: + events = result.data.get("events") + events = [e for e in events if isinstance(e, dict)] if isinstance(events, list) else [] + next_token = result.data.get("page_token") or page_token + # An empty list is indistinguishable from a quiet meeting, so it + # says nothing about participation either way. + verdict = IN_MEETING if events else UNKNOWN + self._record(verdict) + return verdict, events, next_token + if result.feishu_code in NOT_IN_MEETING_CODES: + self._record(NOT_IN_MEETING) + return NOT_IN_MEETING, [], page_token + self._record(UNKNOWN) + return UNKNOWN, [], page_token + + def _record(self, verdict: str) -> None: + self._health.last_probe_at = time.time() + self._health.last_verdict = verdict + if verdict == UNKNOWN: + self._health.consecutive_unknown += 1 + else: + self._health.consecutive_unknown = 0 + + +__all__ = [ + "ABSENCE_EVIDENCE", + "IN_MEETING", + "LivenessProbe", + "MIN_PAGE_SIZE", + "NOT_IN_MEETING", + "NOT_IN_MEETING_CODES", + "UNKNOWN", + "USER_NOT_IN_MEETING_CODES", +] diff --git a/lark_channel/channel/meeting/loop_affinity.py b/lark_channel/channel/meeting/loop_affinity.py new file mode 100644 index 0000000..d9abf87 --- /dev/null +++ b/lark_channel/channel/meeting/loop_affinity.py @@ -0,0 +1,59 @@ +"""Running work on the loop that owns it. + +A meeting session owns an ``asyncio.Queue``, a set of tasks and a few timer +handles. All three are bound to one loop: a queue whose consumer is on loop A +never wakes for a producer on loop B, ``Task.cancel()`` from another loop is not +safe, and a ``TimerHandle`` belongs to the loop that scheduled it. + +The public surface — ``join_meeting``, ``follow_my_meeting``, ``dispose()``, +``leave()`` — can be called from any loop. These two helpers are the single +place that reconciles those two facts, so the policy (including what to do with +a loop that has already stopped) is written once instead of three times. +""" + +import asyncio +from typing import Any, Callable, Optional + + +def _current_loop() -> Optional[Any]: + try: + return asyncio.get_running_loop() + except RuntimeError: + return None + + +def run_on(loop: Optional[Any], fn: Callable[[], None]) -> None: + """Run ``fn`` on ``loop``, from a synchronous caller. + + Falls back to running inline when there is no loop, when we are already on + it, or when it has stopped — teardown has to make progress even after the + loop it was using is gone. + """ + if loop is None or _current_loop() is loop or not loop.is_running(): + fn() + return + try: + loop.call_soon_threadsafe(fn) + except RuntimeError: # pragma: no cover - closed between the check and here + fn() + + +async def await_on(loop: Optional[Any], factory: Callable[[], Any]) -> Any: + """Await ``factory()`` on ``loop``, from an asynchronous caller. + + The stopped-loop case matters: ``disconnect()`` stops the background loop, + and the documented shutdown order is "disconnect, then leave the meetings + you are still in". Handing that coroutine to a stopped loop would wait + forever, so it runs on the caller's loop instead — the session's timers and + tasks are already gone by then, and what is left is a REST call. + """ + if loop is None or _current_loop() is loop or not loop.is_running(): + return await factory() + try: + future = asyncio.run_coroutine_threadsafe(factory(), loop) + except RuntimeError: # pragma: no cover - closed between the check and here + return await factory() + return await asyncio.wrap_future(future) + + +__all__ = ["await_on", "run_on"] diff --git a/lark_channel/channel/meeting/membership.py b/lark_channel/channel/meeting/membership.py new file mode 100644 index 0000000..edf5d39 --- /dev/null +++ b/lark_channel/channel/meeting/membership.py @@ -0,0 +1,281 @@ +"""Server-side participation accounting — what the concurrency gate reads. + +The gate cannot count live sessions. ``dispose()`` stops local work but leaves +the bot a participant, and a failed departure removes the session while the +seat is still taken. So a seat is released on **evidence that the bot is no +longer a participant**, which is a different thing from "the departure call +returned 200". + +Both directions have to hold, and each failure is severe in its own way: + +* release only on a clean departure, and every normally-ended meeting leaks a + seat — ending is exactly when that call is most likely to 404. After + ``max_concurrent_sessions`` such meetings both entry points are dead for the + life of the process, and one tenant member looping "invite the bot, end the + meeting" is enough to get there. +* release on any failure and the gate is off. + +The awkward case is a departure whose outcome is unknown (5xx, timeout). The +seat is kept — but three rules stack into a dead end if nothing else is done: +the seat is kept, the session is still removed, and routing drops events for +meetings with no session. So accounting keeps listening after delivery stops +(``release`` is reachable from the router even with no session), and admission +reconciles the leftovers. + +Reconciliation is **lazy**: it runs on admission, when evidence arrives, and on +connect. Not on a timer. A timer that survived ``disconnect()`` would need its +own thread, and a thread still running after the call whose entire job is +releasing resources is a dangling resource. A seat only needs reclaiming when +somebody wants one, and that is exactly when admission runs. +""" + +import asyncio +import time +from typing import Any, Awaitable, Callable, Dict, List, Optional, Set + +from lark_channel.core.log import logger + +from .errors import sanitize_for_log +from .types import MembershipHealth + +#: Evidence keys, as they appear in ``MembershipHealth.released_by_evidence``. +EVIDENCE_OK = "ok" +EVIDENCE_MEETING_ENDED = "meeting_ended" +EVIDENCE_TTL = "ttl" + +#: Departure failures that prove the seat is already gone server-side. +#: ``121105`` is "meeting not exist" — but it also fires when *we* have been +#: sending the wrong meeting id all along, so releasing on it is reported. +ABSENCE_FEISHU_CODES = frozenset({121105, 120004}) +ABSENCE_HTTP_STATUSES = frozenset({404}) +#: Absence codes that are *also* what a systematically wrong meeting id looks +#: like, so releasing on them is reported rather than absorbed. +AMBIGUOUS_ABSENCE_CODES = frozenset({"121105"}) + + +class _Entry: + __slots__ = ( + "meeting_id", + "added_at", + "confirmed_at", + "last_attempt_at", + "departure_unresolved", + ) + + + def __init__(self, meeting_id: str) -> None: + self.meeting_id = meeting_id + self.added_at = time.monotonic() + #: Last time something proved this meeting is still ours. + self.confirmed_at = self.added_at + # `None`, not 0.0: `time.monotonic()` has an arbitrary origin — on some + # platforms it starts near zero at process start — so a zero sentinel + # reads as "attempted just now" for the first minute of the process and + # silently suppresses the first reconciliation. + self.last_attempt_at = None + # Set only once a departure has actually been attempted and come back + # inconclusive. Reconciliation keys on this rather than on "the session + # is gone", because `dispose()` deliberately does *not* depart: a + # reconnect must not walk the bot out of its meetings, so reconciling + # every session-less entry would turn dispose into leave. + self.departure_unresolved = False + + +class MembershipLedger: + """Which meetings the bot is a participant of, as far as we can tell.""" + + def __init__( + self, + *, + health: MembershipHealth, + max_age_seconds: float, + reconcile_interval_seconds: float, + reconcile_max_concurrency: int, + reconcile_attempt_timeout_seconds: float, + depart: Callable[[str], Awaitable[Any]], + live_meetings: Callable[[], Set[str]], + ) -> None: + self._health = health + self._max_age = max_age_seconds + self._interval = reconcile_interval_seconds + self._max_concurrency = max(1, reconcile_max_concurrency) + self._attempt_timeout = reconcile_attempt_timeout_seconds + self._depart = depart + # Derived, never stored. "Does this meeting have a live session" used to + # be a flag on the entry, and keeping it correct depended on the order + # of `add()` and `dispose()` — which the supersede path gets backwards: + # the new session is accounted for first, then the old one's teardown + # clears the flag, then the new session is registered. Reading the + # routing table instead removes the ordering as a correctness concern. + self._live_meetings = live_meetings + self._entries: Dict[str, _Entry] = {} + + # -- accounting ------------------------------------------------------ + def add(self, meeting_id: str) -> None: + """Record that the bot is a participant of ``meeting_id``. + + Re-joining a meeting we still have an entry for resets it. Carrying the + old ``departure_unresolved`` over would count a meeting with a live + session as one whose departure is unresolved, and send reconciliation + after it every interval — each attempt correctly refused by the + requester guard, each one logging a warning that says the opposite of + what is true. + """ + if not meeting_id: + return + existing = self._entries.get(meeting_id) + if existing is None: + self._entries[meeting_id] = _Entry(meeting_id) + else: + existing.departure_unresolved = False + existing.confirmed_at = time.monotonic() + self._sync_health() + + def touch(self, meeting_id: str) -> None: + """Note that this meeting is demonstrably still ours. + + The deadline below is a backstop for *mis-accounting*, so it has to be + measured from the last evidence rather than from when the entry was + created. Without this, a meeting that genuinely runs longer than + ``membership_max_age_seconds`` has its seat released while the session + is alive and the bot is still in the room — and the release logs a + warning claiming our accounting is wrong. + """ + entry = self._entries.get(meeting_id) + if entry is not None: + entry.confirmed_at = time.monotonic() + + def release(self, meeting_id: str, *, evidence: str) -> bool: + """Give the seat back. ``True`` if it was held.""" + if self._entries.pop(meeting_id, None) is None: + return False + counts = self._health.released_by_evidence + counts[evidence] = counts.get(evidence, 0) + 1 + if evidence in AMBIGUOUS_ABSENCE_CODES: + # Also what a systematically wrong meeting id looks like. Absorbing + # it silently would turn the gate off and keep every test green. + logger.warning( + "meeting: released a seat on %s (meeting not exist) for meeting %s; " + "if this repeats, the meeting id being sent is likely wrong", + evidence, + sanitize_for_log(meeting_id), + ) + self._sync_health() + return True + + def held(self) -> Set[str]: + """The seats currently taken. + + Expiry runs here because this is read on every admission and every + health check, and it is purely local. Leaving it only inside + ``reconcile`` would make the deadline depend on somebody attempting a + new join — so a process that stops joining never releases anything. + """ + self._expire_overdue() + return set(self._entries) + + def note_departure_outcome(self, meeting_id: str, result: Any) -> None: + """Apply whatever a departure attempt proved about the seat.""" + if result is None: + return + if getattr(result, "ok", False): + self.release(meeting_id, evidence=EVIDENCE_OK) + return + code = getattr(result, "feishu_code", None) + status = getattr(result, "status", None) + if code in ABSENCE_FEISHU_CODES: + self.release(meeting_id, evidence=str(code)) + return + if status in ABSENCE_HTTP_STATUSES: + self.release(meeting_id, evidence=str(status)) + return + # Unknown outcome: keep the seat and mark it for reconciliation. + entry = self._entries.get(meeting_id) + if entry is not None: + entry.departure_unresolved = True + self._sync_health() + + # -- lazy reconciliation -------------------------------------------- + async def reconcile(self) -> None: + """Work through seats whose session is gone. Cheap when there are none. + + Called from admission (before the ceiling is compared), when evidence + arrives, and from ``connect()``. Deliberately not scheduled. + """ + self._expire_overdue() + if self._interval <= 0: + return + now = time.monotonic() + candidates = [ + entry + for entry in list(self._entries.values()) + if entry.departure_unresolved + and ( + entry.last_attempt_at is None + or (now - entry.last_attempt_at) >= self._interval + ) + ] + if not candidates: + return + for batch_start in range(0, len(candidates), self._max_concurrency): + batch = candidates[batch_start : batch_start + self._max_concurrency] + await asyncio.gather( + *[self._attempt(entry) for entry in batch], return_exceptions=True + ) + + async def _attempt(self, entry: _Entry) -> None: + entry.last_attempt_at = time.monotonic() + self._health.reconcile_attempts += 1 + try: + result = await asyncio.wait_for( + self._depart(entry.meeting_id), timeout=self._attempt_timeout + ) + except Exception: + # Still unknown; the deadline is the backstop. + return + self.note_departure_outcome(entry.meeting_id, result) + + def _expire_overdue(self) -> None: + if self._max_age <= 0: + return + now = time.monotonic() + live = self._live_meetings() + for meeting_id, entry in list(self._entries.items()): + if meeting_id in live: + # A seat with a live session is not mis-accounted, and this + # deadline exists only to catch mis-accounting. Expiring it + # would release the seat of a meeting the bot is demonstrably + # still in — a two-hour meeting where nobody happens to speak + # produces no activity and no conclusive probe, so the clock + # would run out on a perfectly healthy session. + continue + if (now - entry.confirmed_at) < self._max_age: + continue + logger.warning( + "meeting: releasing seat for meeting %s after %.0fs without a " + "conclusive departure; the accounting for it is wrong", + sanitize_for_log(meeting_id), + now - entry.confirmed_at, + ) + self.release(meeting_id, evidence=EVIDENCE_TTL) + + # -- health ---------------------------------------------------------- + def _sync_health(self) -> None: + self._health.held = len(self._entries) + # Entries whose departure was attempted and left unresolved — the ones + # reconciliation works through. Not merely "has no session": a disposed + # session leaves the bot in the meeting on purpose. + self._health.retained_without_session = sum( + 1 for entry in self._entries.values() if entry.departure_unresolved + ) + + +__all__ = [ + "ABSENCE_FEISHU_CODES", + "AMBIGUOUS_ABSENCE_CODES", + "ABSENCE_HTTP_STATUSES", + "EVIDENCE_MEETING_ENDED", + "EVIDENCE_OK", + "EVIDENCE_TTL", + "MembershipLedger", +] diff --git a/lark_channel/channel/meeting/normalize.py b/lark_channel/channel/meeting/normalize.py new file mode 100644 index 0000000..ec0303a --- /dev/null +++ b/lark_channel/channel/meeting/normalize.py @@ -0,0 +1,279 @@ +"""Turning one wire event into a stream of session events. + +Two shapes, one output. The push transport flattens ``*_items`` onto the +activity object; the poll transport nests them under ``payload`` along with +``activity_event_type``. Both are read, because both are what the platform +sends, and an implementation that reads only one produces an event stream that +is empty for half its inputs without ever raising. +""" + +import re +from typing import Any, Dict, List, Optional, Tuple + +from .coerce import actor_id, meeting_id_str, to_ms +from .types import ( + ACTIVITY_ACTOR_FIELDS, + ACTIVITY_ITEM_FIELDS, + ActivityTypeStats, + DOCUMENT_CONTEXT_KINDS, + DocumentContextEvent, + MeetingActor, + MeetingChatEvent, + ParticipantEvent, + ShareDocInfo, + ShareEvent, + TranscriptEvent, +) + +#: Bucket for activity types we will not turn into a metric key. +OTHER_STAT_KEY = "__other__" +#: Distinct stat keys allowed before everything new lands in the bucket. +MAX_STAT_KEYS = 5000 +#: Shape a server-provided type must have to become a key of its own. Bounding +#: the *count* does not stop one two-hundred-character key, and does not stop a +#: key with a newline in it from reshaping a log line. +_STAT_KEY_SHAPE = re.compile(r"^[a-z0-9_]{1,64}$") + +_ACTION_BY_TYPE = { + "participant_joined": "joined", + "participant_left": "left", + "magic_share_started": "started", + "magic_share_ended": "ended", +} + +#: Session event name per activity type. +_EVENT_NAME_BY_TYPE = { + "transcript_received": "transcript", + "chat_received": "chat", + "participant_joined": "participant", + "participant_left": "participant", + "magic_share_started": "share", + "magic_share_ended": "share", + "document_context_changed": "document_context", +} + + +def stat_key(activity_event_type: Any) -> str: + """The metric key for ``activity_event_type``, or the shared bucket.""" + if isinstance(activity_event_type, str) and _STAT_KEY_SHAPE.match( + activity_event_type + ): + return activity_event_type + return OTHER_STAT_KEY + + +def bump_stats( + stats: Dict[str, ActivityTypeStats], key: str, *, empty: bool +) -> None: + """Account for one activity object, keeping the key space bounded.""" + if key not in stats and len(stats) >= MAX_STAT_KEYS: + key = OTHER_STAT_KEY + entry = stats.get(key) + if entry is None: + entry = ActivityTypeStats() + stats[key] = entry + entry.received += 1 + if empty: + entry.empty += 1 + + +def activity_type_of(activity: Dict[str, Any]) -> Optional[str]: + """``activity_event_type``, from wherever this transport put it.""" + direct = activity.get("activity_event_type") + if isinstance(direct, str) and direct: + return direct + payload = activity.get("payload") + if isinstance(payload, dict): + nested = payload.get("activity_event_type") + if isinstance(nested, str) and nested: + return nested + return None + + +def items_of(activity: Dict[str, Any], activity_event_type: str) -> List[Dict[str, Any]]: + """The inner ``*_items`` list, from either nesting.""" + field = ACTIVITY_ITEM_FIELDS.get(activity_event_type) + if field is None: + return [] + flat = activity.get(field) + if isinstance(flat, list): + return [item for item in flat if isinstance(item, dict)] + payload = activity.get("payload") + if isinstance(payload, dict): + nested = payload.get(field) + if isinstance(nested, list): + return [item for item in nested if isinstance(item, dict)] + return [] + + +def meeting_id_of(activity: Dict[str, Any], fallback: Optional[str]) -> Optional[str]: + """The long meeting id this activity belongs to, always as a string.""" + for candidate in (activity.get("meeting_id"), (activity.get("meeting") or {}).get("id") + if isinstance(activity.get("meeting"), dict) else None): + resolved = meeting_id_str(candidate) + if resolved: + return resolved + return fallback + + +def _actor_of(item: Dict[str, Any], activity_event_type: str) -> MeetingActor: + field = ACTIVITY_ACTOR_FIELDS.get(activity_event_type) + raw = item.get(field) if field else None + if not isinstance(raw, dict): + raw = {} + return MeetingActor( + id=actor_id(raw), + name=raw.get("name") or raw.get("user_name"), + user_type=raw.get("user_type"), + user_role=raw.get("user_role"), + ) + + +def _doc_of(item: Dict[str, Any]) -> Optional[ShareDocInfo]: + # The field is `share_doc`; reading `doc` yields None for every share. + raw = item.get("share_doc") + if not isinstance(raw, dict): + return None + return ShareDocInfo(url=raw.get("url"), title=raw.get("title")) + + +def _context_type_of(item: Dict[str, Any]) -> Optional[str]: + """Which kind of document context this is. + + Derived from whichever sub-object is present, because the generated model + has no discriminator. An explicit ``context_type`` in the payload wins: + that is the platform having moved ahead of the generated types, and there + is no reason to prefer our inference over its statement. + """ + explicit = item.get("context_type") + if isinstance(explicit, str) and explicit: + return explicit + for kind in DOCUMENT_CONTEXT_KINDS: + if isinstance(item.get(kind), dict): + return kind + return None + + +def build_event( + activity_event_type: str, + item: Dict[str, Any], + *, + meeting_id: str, + include_raw: bool, +) -> Optional[Tuple[str, Any]]: + """``(session event name, event)`` for one inner item, or ``None`` to skip.""" + name = _EVENT_NAME_BY_TYPE.get(activity_event_type) + if name is None: + return None + common = { + "meeting_id": meeting_id, + "actor": _actor_of(item, activity_event_type), + "raw": dict(item) if include_raw else None, + } + if activity_event_type == "transcript_received": + return name, TranscriptEvent( + text=item.get("text") or "", + sentence_id=item.get("sentence_id"), + language=item.get("language"), + start_ms=to_ms(item.get("start_time_ms")), + end_ms=to_ms(item.get("end_time_ms")), + **common + ) + if activity_event_type == "chat_received": + return name, MeetingChatEvent( + content=item.get("content") or "", + message_id=item.get("message_id"), + message_type=item.get("message_type"), + send_time=to_ms(item.get("send_time")), + **common + ) + if activity_event_type in ("participant_joined", "participant_left"): + return name, ParticipantEvent( + action=_ACTION_BY_TYPE[activity_event_type], + join_time=to_ms(item.get("join_time")), + leave_time=to_ms(item.get("leave_time")), + leave_reason=item.get("leave_reason"), + **common + ) + if activity_event_type in ("magic_share_started", "magic_share_ended"): + return name, ShareEvent( + action=_ACTION_BY_TYPE[activity_event_type], + share_id=item.get("share_id"), + doc=_doc_of(item), + time=to_ms(item.get("time")), + **common + ) + context_type = _context_type_of(item) + if context_type is None: + # A fourth kind of context the platform has started sending. Forward + # compatibility, not a parse failure — see `unpack`. + return None + return name, DocumentContextEvent( + context_type=context_type, + share_id=item.get("share_id"), + doc=_doc_of(item), + time=to_ms(item.get("time")), + comment_focus=item.get("comment_focus"), + section_location=item.get("section_location"), + element_preview=item.get("element_preview"), + **common + ) + + +def unpack( + activity: Dict[str, Any], + *, + meeting_id: str, + include_raw: bool, + stats: Dict[str, ActivityTypeStats], +) -> List[Tuple[str, Any]]: + """Every session event in one activity object, in array order. + + Also does the stat accounting, because "how many arrived" and "how many + unpacked to nothing" are only knowable here: + + * an activity type we do not know counts as ``empty`` — that is us having + fallen behind the platform, which is what the counter is for; + * a ``document_context_changed`` item with none of the three known + sub-objects is skipped and **not** counted as empty. That is the platform + adding a kind of context, and counting it would report a field-shape + regression that did not happen. + """ + activity_event_type = activity_type_of(activity) + if activity_event_type is None: + bump_stats(stats, OTHER_STAT_KEY, empty=True) + return [] + key = stat_key(activity_event_type) + items = items_of(activity, activity_event_type) + built = [] + for item in items: + event = build_event( + activity_event_type, + item, + meeting_id=meeting_id, + include_raw=include_raw, + ) + if event is not None: + built.append(event) + known = activity_event_type in ACTIVITY_ITEM_FIELDS + if known: + # Items present but all skipped is forward compatibility, not a + # failure; no items at all is what `empty` means. + empty = not items + else: + empty = True + bump_stats(stats, key, empty=empty) + return built + + +__all__ = [ + "MAX_STAT_KEYS", + "OTHER_STAT_KEY", + "activity_type_of", + "build_event", + "bump_stats", + "items_of", + "meeting_id_of", + "stat_key", + "unpack", +] diff --git a/lark_channel/channel/meeting/rate_limit.py b/lark_channel/channel/meeting/rate_limit.py new file mode 100644 index 0000000..722a8a3 --- /dev/null +++ b/lark_channel/channel/meeting/rate_limit.py @@ -0,0 +1,38 @@ +"""Per-session send budget. + +The bot's own in-meeting messages come back as meeting chat, so a handler that +replies without checking the echo flag amplifies itself at network speed. The +flag is the real fix, but it lives in application code; this is the backstop +that keeps a missing check from becoming in-meeting flooding, quota exhaustion, +and platform-side risk controls. +""" + +import time +from collections import deque +from typing import Deque + +_WINDOW_SECONDS = 60.0 + + +class SendRateLimiter: + """Sliding one-minute window over send attempts.""" + + def __init__(self, per_minute: int) -> None: + self._limit = per_minute + self._sent: Deque[float] = deque() + + def try_acquire(self) -> bool: + """Record a send and return whether it is within budget.""" + if self._limit <= 0: + return True + now = time.monotonic() + cutoff = now - _WINDOW_SECONDS + while self._sent and self._sent[0] < cutoff: + self._sent.popleft() + if len(self._sent) >= self._limit: + return False + self._sent.append(now) + return True + + +__all__ = ["SendRateLimiter"] diff --git a/lark_channel/channel/meeting/registry.py b/lark_channel/channel/meeting/registry.py new file mode 100644 index 0000000..88197a8 --- /dev/null +++ b/lark_channel/channel/meeting/registry.py @@ -0,0 +1,622 @@ +"""The meeting channel: routing, the concurrency gate, and both entry points. + +Pushed activity is **application-wide** — every meeting this app is in arrives +on one stream, distinguished by ``meeting.id``. Business code faces one meeting +at a time, so this is where that one routing step happens; each +:class:`MeetingSession` then only ever sees its own meeting. + +Session creation is triggered from **outside** the process: joining starts at +an invitation from anybody who can add the bot to a meeting, following typically +at an inbound instruction. So both entry points share one ceiling, and both get +an identity filter — a count gate and an identity gate answer different +questions and neither substitutes for the other. +""" + +import asyncio +import time +from typing import Any, Callable, Dict, List, Optional, Set, Tuple + +from lark_channel.api.vc.bot import ( + build_bot_join_request, + build_bot_leave_request, + build_user_active_meeting_request, +) +from lark_channel.core.log import logger + +from ..errors import FeishuChannelError, FeishuChannelErrorCode +from .admission import AdmissionGate +from .api import MeetingApi +from .coerce import meeting_id_str +from .errors import sanitize_for_log +from .health_view import MeetingHealthView +from .liveness import ABSENCE_EVIDENCE, LivenessProbe +from .membership import EVIDENCE_MEETING_ENDED, MembershipLedger +from .normalize import meeting_id_of +from .session import MeetingSession +from .sources import PollSource, PushSource +from .types import ( + MeetingActor, + MeetingEventHealth, + MeetingInvitedEvent, + MeetingOptions, +) + +ACTIVITY_EVENT = "vc.bot.meeting_activity_v1" +INVITED_EVENT = "vc.bot.meeting_invited_v1" +ENDED_EVENT = "vc.bot.meeting_ended_v1" +INTERNAL_EVENT_TYPES = (ACTIVITY_EVENT, INVITED_EVENT, ENDED_EVENT) + +#: Warned about once per process, on the first ``follow_my_meeting`` call. +_COMPLIANCE_NOTICE = ( + "meeting: follow_my_meeting reads every participant's speech for the whole " + "meeting under a user's own authorization, and the bot is not visible in " + "the meeting. Telling participants and obtaining their consent is the " + "integrating application's responsibility; this SDK does not do it." +) + + +class MeetingChannel: + """Owns every meeting session on one channel.""" + + def __init__( + self, + *, + client: Any, + config: Any, + seen: Any, + bot_open_id_getter: Callable[[], Optional[str]], + schedule: Callable[[Any], Any], + resolve_ticket_interactive: Callable[..., Any], + resolve_ticket_quiet: Callable[[str], Any], + emit_invited: Callable[[MeetingInvitedEvent], Any], + timeout_seconds: float = 30.0, + ) -> None: + self._config = config + self._meeting_config = config.meeting + self._seen = seen + self._bot_open_id_getter = bot_open_id_getter + self._schedule = schedule + self._resolve_ticket_interactive = resolve_ticket_interactive + self._resolve_ticket_quiet = resolve_ticket_quiet + self._emit_invited = emit_invited + + self._api = MeetingApi(client, timeout_seconds=timeout_seconds) + self._sessions: Dict[str, MeetingSession] = {} + self._health_view = MeetingHealthView(lambda: list(self._sessions.values())) + self._health = self._health_view.health + self._probe = LivenessProbe(self._api, self._health.liveness) + self._membership = MembershipLedger( + health=self._health.membership, + max_age_seconds=self._meeting_config.membership_max_age_seconds, + reconcile_interval_seconds=( + self._meeting_config.membership_reconcile_interval_seconds + ), + reconcile_max_concurrency=( + self._meeting_config.membership_reconcile_max_concurrency + ), + reconcile_attempt_timeout_seconds=( + self._meeting_config.membership_reconcile_attempt_timeout_seconds + ), + depart=self._depart, + live_meetings=self._live_meeting_ids, + ) + self._follow_by_user: Dict[Tuple[str, str], MeetingSession] = {} + self._gate = AdmissionGate( + max_concurrent_sessions=self._meeting_config.max_concurrent_sessions, + live_meetings=self._live_meeting_ids, + held_meetings=self._membership.held, + reconcile=self._membership.reconcile, + ) + self._compliance_warned = False + + def _live_meeting_ids(self) -> Set[str]: + """The meetings this process currently has a session for. + + One method, two readers — the accounting ledger and the admission gate. + The value of not storing this is that both read the same source; two + copies of the same lambda is the one edit that could let them diverge + again. + """ + return set(self._sessions) + + # -- health ---------------------------------------------------------- + def health(self) -> MeetingEventHealth: + # Reading `held()` is what expires overdue entries, so a caller that + # only ever reads health still sees a current seat count rather than + # seats that passed their deadline hours ago. + self._membership.held() + return self._health_view.snapshot() + + def mark_registration(self, *, ok: bool, reason: Optional[str]) -> None: + self._health_view.mark_registration(ok=ok, reason=reason) + + def on_connected(self) -> None: + """The channel just became ready. One of the three reconciliation points. + + Reconnecting is a good moment for it: a seat stranded by an + inconclusive departure before the connection dropped is exactly what a + fresh connection wants back, and this needs no timer. + """ + self._schedule(self._membership.reconcile()) + + # -- dispatcher entry points ---------------------------------------- + def on_activity(self, payload: Dict[str, Any]) -> None: + self._schedule(self._handle_activity(payload)) + + def on_invited(self, payload: Dict[str, Any]) -> None: + self._schedule(self._handle_invited(payload)) + + def on_ended(self, payload: Dict[str, Any]) -> None: + self._schedule(self._handle_ended(payload)) + + async def _handle_activity(self, payload: Dict[str, Any]) -> None: + event = payload.get("event") or {} + envelope_meeting = meeting_id_str((event.get("meeting") or {}).get("id")) + activities = event.get("meeting_activity_items") + if not isinstance(activities, list): + return + self._health.received += 1 + self._health.last_at = time.time() + grouped: Dict[Optional[str], List[Dict[str, Any]]] = {} + for activity in activities: + if not isinstance(activity, dict): + continue + meeting_id = meeting_id_of(activity, envelope_meeting) + grouped.setdefault(meeting_id, []).append(activity) + for meeting_id, batch in grouped.items(): + session = self._sessions.get(meeting_id) if meeting_id else None + if session is None: + # Not a meeting this process runs, so there is nobody to + # deliver to. Accounting is not skipped by this: the paths that + # release a seat — `_handle_ended`, the liveness probe — do not + # go through here. + continue + # Activity is evidence that this meeting is still ours, which is + # what keeps a long meeting from hitting the accounting deadline. + self._membership.touch(meeting_id) + await session.ingest(batch) + + async def _handle_invited(self, payload: Dict[str, Any]) -> None: + event = payload.get("event") or {} + meeting = event.get("meeting") or {} + inviter = _actor_from(event.get("inviter")) + allowlist = self._meeting_config.invite_allowlist + if allowlist is not None and (inviter.id or "") not in allowlist: + logger.warning( + "meeting: ignoring an invitation from an inviter outside " + "invite_allowlist (meeting_no=%s)", + sanitize_for_log(meeting.get("meeting_no")), + ) + return + invited = MeetingInvitedEvent( + meeting_no=meeting.get("meeting_no") or "", + meeting_id=meeting_id_str(meeting.get("id")), + topic=meeting.get("topic"), + inviter=inviter, + bot=_actor_from(event.get("bot")), + call_id=event.get("call_id"), + invite_time=event.get("invite_time"), + ) + await self._emit_invited(invited) + + async def _handle_ended(self, payload: Dict[str, Any]) -> None: + event = payload.get("event") or {} + meeting_id = meeting_id_str((event.get("meeting") or {}).get("id")) + if not meeting_id: + return + # Accounting first, and unconditionally: the meeting is over + # server-side, so the seat is gone whether or not a departure call ever + # succeeded — and whether or not a session still exists to deliver to. + # This is the rule that keeps a 5xx departure from stranding a seat. + self._membership.release(meeting_id, evidence=EVIDENCE_MEETING_ENDED) + # Evidence arriving is one of the three moments reconciliation runs: + # a meeting ending is often when a *previous* inconclusive departure + # becomes resolvable. + await self._membership.reconcile() + session = self._sessions.get(meeting_id) + if session is None: + return + await self._retire(session, reason="meeting_ended", depart=True) + + # -- entry points ---------------------------------------------------- + async def join_meeting( + self, + meeting_no: str, + *, + connected: bool, + password: Optional[str] = None, + call_id: Optional[str] = None, + options: Optional[MeetingOptions] = None, + ) -> MeetingSession: + if not connected: + raise FeishuChannelError( + FeishuChannelErrorCode.NOT_CONNECTED, + "join_meeting needs the event socket: in-meeting activity is " + "pushed, so without connect() the session would receive nothing", + ) + inflight = self._gate.joining(meeting_no) + if inflight is not None: + return await asyncio.shield(inflight) + # Claimed **before** the first await. Registering it after the admission + # check leaves a window where two concurrent calls both pass the check, + # both call the platform, and the loser's session is silently evicted + # from the routing table with its probe loop still running. + future = self._gate.claim(meeting_no) + try: + await self._gate.admit() + session = await self._do_join( + meeting_no, password=password, call_id=call_id, options=options + ) + except BaseException as exc: + future.set_exception(exc) + # Nobody may be awaiting this future; retrieving it here keeps the + # loop from reporting it as never-retrieved. + future.exception() + raise + else: + future.set_result(session) + return session + finally: + self._gate.release(meeting_no) + # Dropped from this frame's locals. On the failing path this frame + # is part of the raised error's `__traceback__`, and a crash + # reporter that reads frame locals reads the password with it. + password = None + + async def _do_join( + self, + meeting_no: str, + *, + password: Optional[str], + call_id: Optional[str], + options: Optional[MeetingOptions], + ) -> MeetingSession: + request = build_bot_join_request( + meeting_no=meeting_no, password=password, call_id=call_id + ) + result = await self._api.call(request, what="meeting join") + password = None # see `join_meeting`: this frame unwinds on failure + if not result.ok: + if result.transport_error: + # The platform may already have admitted the bot while the + # answer was lost. There is nothing useful to send: departure + # rejects a nine-digit number, and the long id was in the reply + # that never arrived. + logger.warning( + "meeting: join for %s failed without an answer; the bot may " + "already be a participant and cannot be removed automatically", + sanitize_for_log(meeting_no), + ) + raise self._api.error_for(result, what="meeting join") + meeting = result.data.get("meeting") or {} + meeting_id = meeting_id_str(meeting.get("id")) + if not meeting_id: + # The response echoes the password back, so the same reasoning as + # the request side applies in the other direction: this frame is + # part of the raised error's `__traceback__`, and its locals still + # reach that echo. `docs/security.md` promises passwords stay out + # of error objects in *both* directions. + meeting = None + result = None + raise FeishuChannelError( + FeishuChannelErrorCode.UNKNOWN, "meeting join returned no meeting id" + ) + self._membership.add(meeting_id) + # Only these three fields are kept. Some meeting responses carry a + # plaintext password, and anything kept here lives as long as the + # session does. + session = self._new_session( + meeting_id=meeting_id, + meeting_no=meeting.get("meeting_no") or meeting_no, + topic=meeting.get("topic"), + mode="tat", + options=options, + ) + source = PushSource( + meeting_id=meeting_id, + probe=self._probe, + probe_interval_seconds=self._meeting_config.liveness_probe_interval_seconds, + idle_timeout_seconds=self._meeting_config.idle_timeout_seconds, + deliver=lambda events: session.ingest(events), + on_absent=lambda: self._probe_said_absent(session), + on_idle=lambda: self._retire(session, reason="idle_timeout", depart=True), + confirm_membership=lambda: self._membership.touch(meeting_id), + ) + session.attach(source) + return session + + async def follow_my_meeting( + self, + *, + user_open_id: str, + prompt_context: Any = None, + meeting_no: Optional[str] = None, + options: Optional[MeetingOptions] = None, + ) -> MeetingSession: + # Identity gate first: before the ticket store is touched, before the + # network, and before session reuse — reuse is keyed on the meeting, so + # checking afterwards would let an unlisted caller inherit somebody + # else's live session and with it that person's transcript. + allowlist = self._meeting_config.follow_allowlist + if allowlist is not None and user_open_id not in allowlist: + raise FeishuChannelError( + FeishuChannelErrorCode.PERMISSION_DENIED, + "this open_id is not in meeting.follow_allowlist", + ) + if not self._compliance_warned: + self._compliance_warned = True + logger.warning(_COMPLIANCE_NOTICE) + # Reuse is keyed on the resolved meeting, not on the requested + # `meeting_no` — which is optional and usually `None`, so keying on it + # degrades to "one session per user" and hands back a stale session + # after the user has moved to a different meeting. + await self._gate.admit() + token = await self._resolve_ticket_interactive( + user_open_id=user_open_id, prompt_context=prompt_context + ) + try: + selected = await self._select_active_meeting( + user_open_id=user_open_id, token=token, meeting_no=meeting_no + ) + finally: + token = None + meeting_id, resolved_no, topic = selected + reuse_key = (user_open_id, meeting_id) + existing = self._follow_by_user.get(reuse_key) + if existing is not None: + return existing + session = self._sessions.get(meeting_id) + if session is not None: + self._follow_by_user[reuse_key] = session + return session + session = self._new_session( + meeting_id=meeting_id, + meeting_no=resolved_no, + topic=topic, + mode="uat", + options=options, + ) + self._follow_by_user[reuse_key] = session + source = PollSource( + meeting_id=meeting_id, + api=self._api, + config=self._meeting_config, + resolve_ticket=lambda: self._resolve_ticket_quiet(user_open_id), + deliver=lambda events: session.ingest(events), + on_ended=lambda reason: self._retire(session, reason=reason, depart=False), + on_terminated=lambda error: self._terminate(session, error), + on_error=session.report, + ) + session.attach(source) + return session + + async def _select_active_meeting( + self, *, user_open_id: str, token: str, meeting_no: Optional[str] + ) -> Tuple[str, str, Optional[str]]: + result = await self._api.call( + build_user_active_meeting_request(), + user_access_token=token, + what="active meeting lookup", + ) + # "no active meeting" is the most common failure on this path, and this + # frame is in the traceback when it raises. + token = None + if not result.ok: + raise self._api.error_for(result, what="active meeting lookup") + meetings = [m for m in (result.data.get("meetings") or []) if isinstance(m, dict)] + if meeting_no is not None: + meetings = [m for m in meetings if m.get("meeting_no") == meeting_no] + if not meetings: + raise FeishuChannelError( + FeishuChannelErrorCode.MEETING_NOT_FOUND, + "no active meeting found for this user", + ) + chosen = meetings[0] + if len(meetings) > 1: + # Meeting titles are written by their creators, so they travel as + # lazy arguments and get their control characters escaped — a + # newline here would forge a log line. + logger.warning( + "meeting: following the first of %d active meetings (%s); " + "pass meeting_no to choose. others: %s", + len(meetings), + sanitize_for_log(chosen.get("meeting_no")), + sanitize_for_log( + ", ".join(str(m.get("meeting_no")) for m in meetings[1:]) + ), + ) + meeting_id = meeting_id_str(chosen.get("meeting_id")) + if not meeting_id: + raise FeishuChannelError( + FeishuChannelErrorCode.MEETING_NOT_FOUND, + "the active meeting carried no meeting id", + ) + return meeting_id, chosen.get("meeting_no") or "", chosen.get("meeting_title") or chosen.get("topic") + + # -- teardown -------------------------------------------------------- + def _new_session( + self, + *, + meeting_id: str, + meeting_no: str, + topic: Optional[str], + mode: str, + options: Optional[MeetingOptions], + ) -> MeetingSession: + if options is None: + options = MeetingOptions( + stabilize_seconds=self._meeting_config.stabilize_seconds + ) + superseded = self._sessions.get(meeting_id) + if superseded is not None: + # A second session for one meeting would leave the first one out of + # the routing table but still running — for a follow session that + # means it keeps polling the whole meeting with the user's ticket + # after the application believes it is gone, and `dispose_all()` + # can no longer reach it. + logger.warning( + "meeting: replacing an existing %s session for meeting %s; " + "the previous one is being disposed", + superseded.mode, + sanitize_for_log(meeting_id), + ) + superseded.dispose(reason="disposed") + session = MeetingSession( + meeting_id=meeting_id, + meeting_no=meeting_no, + mode=mode, + topic=topic, + options=options, + api=self._api, + config=self._meeting_config, + seen=self._seen, + bot_open_id_getter=self._bot_open_id_getter, + on_teardown=self._forget, + depart=self._depart, + ) + self._sessions[meeting_id] = session + return session + + def _forget(self, session: MeetingSession, reason: str) -> None: + """Called from ``dispose()``: unregister and hand the seat back. + + Identity-checked, not keyed: a superseded session tearing down must not + evict the one that replaced it. + """ + if self._sessions.get(session.meeting_id) is session: + self._sessions.pop(session.meeting_id, None) + for key, tracked in list(self._follow_by_user.items()): + if tracked is session: + self._follow_by_user.pop(key, None) + self._health_view.retire(session) + # Draining belongs to whoever initiated the teardown — see + # `MeetingSession._dispose_on_owner_loop`. Keeping a list of drained + # sessions here would grow for the life of a process that never + # disconnects, holding on to every torn-down session's closures, + # counters and queue. + + + async def _probe_said_absent(self, session: MeetingSession) -> None: + """A probe established the bot is no longer a participant. + + The seat has to be released here. This is the one path with no + ``meeting_ended_v1`` behind it — a host removing the bot, a meeting + being transferred — so nothing else will ever produce the evidence, and + lazy reconciliation only looks at entries whose *departure* came back + inconclusive. Without this the seat waits out the accounting deadline, + and repeating the removal exhausts the ceiling for both entry points. + + No departure call: the bot is already out, and this endpoint rejects a + departure for a meeting it is not in. + """ + self._membership.release(session.meeting_id, evidence=ABSENCE_EVIDENCE) + await self._retire(session, reason="no_longer_active", depart=False) + + async def _retire( + self, session: MeetingSession, *, reason: str, depart: bool + ) -> None: + await session.retire(reason=reason, depart=depart) + + async def _terminate(self, session: MeetingSession, error: Any) -> None: + """Full recovery for a source that cannot continue. + + Stopping the loop is not enough: no idle deadline and no liveness probe + applies to a follow session, so a half-terminated one would never be + collected by anything. + """ + await session.report(error) + await session.retire(reason="error", depart=False) + + async def _depart(self, meeting_id: str, *, requester: Any = None) -> Any: + """Leave ``meeting_id``, unless another session has taken it over. + + A stale handle is easy to hold on to: after a session is superseded the + application may still call ``leave()`` on the old one, and that call + would eject the bot from a meeting the *replacement* is actively + serving. + + The condition is deliberately "another session is registered for this + meeting", not "the caller is absent from the routing table" — the + documented shutdown order is ``disconnect()`` then ``leave()``, and by + then no session is registered at all, yet that departure is exactly the + one that has to go through. + """ + current = self._sessions.get(meeting_id) + if current is not None and current is not requester: + logger.warning( + "meeting: ignoring a departure from a superseded session for " + "meeting %s; another session is serving it", + sanitize_for_log(meeting_id), + ) + return None + result = await self._api.call( + build_bot_leave_request(meeting_id=meeting_id), what="meeting departure" + ) + self._membership.note_departure_outcome(meeting_id, result) + return result + + def dispose_all(self) -> List[MeetingSession]: + """Dispose every live session without departing any meeting. + + A reconnect must not make the bot vanish from the meetings it is in, so + this is what ``disconnect()`` does. The flip side is that the bot stays + a participant, which is why the seats stay counted and why the process + should ``leave()`` before exiting. + """ + sessions = list(self._sessions.values()) + for session in sessions: + session.dispose(reason="disposed") + remaining = self._membership.held() + if remaining: + logger.warning( + "channel: %d meeting(s) still have this bot as a participant " + "after disconnect; call leave() on those sessions before exit " + "or the bot stays in them", + len(remaining), + ) + return sessions + + async def drain_sessions(self, sessions: List[MeetingSession]) -> None: + """Wait out each session's bounded drain. Used by channel teardown. + + ``drain()`` is single-flight, so this is safe even though disposal + already scheduled one. + """ + if not sessions: + return + # Concurrently, so the caller's budget is one session's drain rather + # than the sum over every session. + await asyncio.gather( + *[self._drain_one(session) for session in sessions], + return_exceptions=True, + ) + + @staticmethod + async def _drain_one(session: MeetingSession) -> None: + try: + await session.drain() + except Exception: # pragma: no cover - teardown must not raise + pass + + + +def _actor_from(raw: Any) -> MeetingActor: + from .coerce import actor_id + + if not isinstance(raw, dict): + return MeetingActor() + return MeetingActor( + id=actor_id(raw), + name=raw.get("name") or raw.get("user_name"), + user_type=raw.get("user_type"), + user_role=raw.get("user_role"), + ) + + +__all__ = [ + "ACTIVITY_EVENT", + "ENDED_EVENT", + "INTERNAL_EVENT_TYPES", + "INVITED_EVENT", + "MeetingChannel", +] diff --git a/lark_channel/channel/meeting/serial_queue.py b/lark_channel/channel/meeting/serial_queue.py new file mode 100644 index 0000000..785e94d --- /dev/null +++ b/lark_channel/channel/meeting/serial_queue.py @@ -0,0 +1,162 @@ +"""Per-session serial delivery with a bounded wait on teardown. + +Delivery is serial and awaited because order is meaning: a document swap +arrives as ``magic_share_ended`` followed by ``magic_share_started``, and +reordering them makes the application reconstruct the wrong shared document. +Awaiting each handler before the next event is what makes that guarantee hold +end to end rather than only up to the queue. + +The cost, which the README states: a handler that yields and takes a long time +holds up **this meeting's** stream. A handler that blocks *without* yielding +holds up the whole event loop — every meeting, the message path, the socket's +heartbeat — because there is one thread. That is not defensible from here; +blocking work belongs in an executor. +""" + +import asyncio +import inspect +from typing import Any, Awaitable, Callable, List, Optional, Tuple + +#: Ceiling on queued deliveries per session. A handler that yields but takes a +#: long time makes this queue grow at whatever rate the meeting produces +#: activity, so without a ceiling one parked handler costs unbounded memory for +#: the life of the meeting. +#: +#: Overflow rejects the *newest* delivery rather than evicting the oldest or +#: blocking the producer, and neither alternative is available here: +#: +#: * Evicting the oldest would break the guarantee this class exists for. Order +#: is meaning — a document swap arrives as ``magic_share_ended`` then +#: ``magic_share_started`` — and dropping from the front splits such pairs, so +#: the application would rebuild the wrong state from a queue that still looks +#: complete. Rejecting at the tail leaves what is queued contiguous. +#: * Blocking the producer would park the socket's message handler or the poll +#: loop, and there is one thread: that stalls every meeting, the message path +#: and the heartbeat with it. +MAX_QUEUED_DELIVERIES = 1000 + +#: Headroom above the ceiling, reserved for error reports. Teardown submits +#: here — the ``end`` event and any error raised by the departure call — and +#: those are the reports that explain why a session went away. Subjecting them +#: to a ceiling filled by transcripts would drop the explanation and keep the +#: noise. The reserve is small because these submissions are bounded per +#: session, unlike activity. +REPORT_RESERVE = 32 + + +class SerialDelivery: + """A single-consumer queue that awaits each handler in turn.""" + + def __init__( + self, + *, + on_handler_error: Callable[[BaseException], Any], + max_queued: int = MAX_QUEUED_DELIVERIES, + ) -> None: + self._queue: "asyncio.Queue[Optional[Tuple[List[Callable], Any]]]" = ( + asyncio.Queue() + ) + self._on_handler_error = on_handler_error + self._max_queued = max_queued + self._worker: Optional[asyncio.Task] = None + self._closed = False + self._dropped = 0 + + def start(self) -> None: + if self._worker is None: + self._worker = asyncio.ensure_future(self._run()) + + def submit( + self, handlers: List[Callable], payload: Any, *, reserved: bool = False + ) -> bool: + """Queue one delivery. ``False`` if it was refused. + + Refused means the worker has been cancelled, or the queue is at its + ceiling. Either way the return value matters: a caller reporting a + failure needs to know the report was accepted, or it has to fall back + to something else rather than let the failure disappear. + + ``reserved`` marks a delivery that may use the report headroom above + the ceiling. Pass it for diagnostics, never for activity. + """ + if self._closed: + return False + ceiling = self._max_queued + (REPORT_RESERVE if reserved else 0) + if self._queue.qsize() >= ceiling: + self._dropped += 1 + return False + self._queue.put_nowait((list(handlers), payload)) + return True + + @property + def idle(self) -> bool: + return self._queue.empty() + + @property + def dropped(self) -> int: + """Deliveries refused because the queue was full. + + Surfaced through health rather than kept here: a drop that only this + object knows about is the silent data loss the whole readout exists to + make diagnosable. + """ + return self._dropped + + async def _run(self) -> None: + while True: + entry = await self._queue.get() + if entry is None: + self._queue.task_done() + return + handlers, payload = entry + for handler in handlers: + try: + result = handler(payload) + if inspect.isawaitable(result): + await result + except asyncio.CancelledError: + raise + except Exception as exc: + try: + outcome = self._on_handler_error(exc) + if inspect.isawaitable(outcome): + await outcome + except Exception: # pragma: no cover - reporting must not throw + pass + self._queue.task_done() + + async def drain(self, timeout: float) -> bool: + """Wait up to ``timeout`` for the queue to empty. ``True`` if it did. + + Bounded on purpose. The caller's own teardown runs whether or not this + succeeds — otherwise one parked handler would hold a seat for the life + of the process, and would also hold up the very timeout meant to + rescue a stalled session. + + Draining does **not** stop accepting work; only :meth:`cancel` does. + Teardown itself submits here — the ``end`` event, and any error raised + by the departure call — and those arrive while the drain is in flight. + Refusing them here would silently drop exactly the reports that explain + why the session went away. The submissions during teardown are bounded, + and ingestion has already stopped at the session level, so the queue + still empties. + """ + if timeout <= 0: + return self._queue.empty() + try: + await asyncio.wait_for(self._queue.join(), timeout=timeout) + return True + except (asyncio.TimeoutError, asyncio.CancelledError): + return False + except Exception: # pragma: no cover + return False + + def cancel(self) -> None: + self._closed = True + worker = self._worker + self._worker = None + if worker is not None and not worker.done(): + worker.cancel() + + +__all__ = ["MAX_QUEUED_DELIVERIES", "REPORT_RESERVE", "SerialDelivery"] diff --git a/lark_channel/channel/meeting/session.py b/lark_channel/channel/meeting/session.py new file mode 100644 index 0000000..c6f53e0 --- /dev/null +++ b/lark_channel/channel/meeting/session.py @@ -0,0 +1,472 @@ +"""One meeting, as the application sees it. + +The same class serves both entry points; what differs is the injected source and +the ``mode`` string. That is the point of the design: moving from following a +meeting to joining one changes the entry-point line and nothing else. + +``dispose()`` versus ``leave()`` is the one distinction callers must internalize: + +* ``leave()`` departs the meeting, then tears down. +* ``dispose()`` tears down **without** departing — because a reconnect must not + make the bot vanish from every meeting it is in. + +Which means: before the process exits, ``leave()`` every live session, or the +bot sits in those meetings until the server ends them. ``leave()`` therefore +keeps working after ``dispose()``, so that instruction is usable rather than a +trap. +""" + +import asyncio +import inspect +import json +import uuid as _uuid +from typing import Any, Callable, Dict, List, Optional + +from lark_channel.api.vc.bot import build_bot_message_request +from lark_channel.core.log import logger + +from ..errors import FeishuChannelError, FeishuChannelErrorCode +from .dedup import activity_key +from .errors import log_error_fields, sanitize_for_log +from .loop_affinity import await_on, run_on +from .normalize import activity_type_of, items_of, unpack +from .rate_limit import SendRateLimiter +from .serial_queue import SerialDelivery +from .stabilizer import TranscriptStabilizer +from .types import ( + ActivityTypeStats, + MEETING_EVENT_NAMES, + MeetingEndEvent, + MeetingOptions, +) + +Unsubscribe = Callable[[], None] + + +class MeetingSession: + """A live meeting: an event stream, a way to speak, and a way to leave.""" + + def __init__( + self, + *, + meeting_id: str, + meeting_no: str, + mode: str, + topic: Optional[str] = None, + options: MeetingOptions, + api: Any, + config: Any, + seen: Any, + bot_open_id_getter: Callable[[], Optional[str]], + on_teardown: Callable[["MeetingSession", str], Any], + depart: Callable[..., Any], + ) -> None: + self.meeting_id = meeting_id + self.meeting_no = meeting_no + self.mode = mode + #: untrusted — whatever the meeting's creator typed. + self.topic = topic + self._options = options + self._api = api + self._config = config + self._seen = seen + self._bot_open_id_getter = bot_open_id_getter + self._on_teardown = on_teardown + self._depart_call = depart + + self._handlers: Dict[str, List[Callable]] = {} + self._stats: Dict[str, ActivityTypeStats] = {} + self._source: Any = None + self._loop: Optional[Any] = None + self._closed = False + self._ended = False + self._left = False + self._drain_task: Optional[Any] = None + self._limiter = SendRateLimiter(config.send_rate_limit_per_minute) + self._delivery = SerialDelivery( + on_handler_error=self._report_from_worker + ) + window = options.stabilize_seconds + self._stabilizer = TranscriptStabilizer( + window_seconds=window, emit=self._enqueue_transcript + ) + + def __repr__(self) -> str: + # Field-by-field would print whatever else ends up on the instance; a + # meeting session is one hop away from a user ticket. + return "MeetingSession(meeting_id=%r, meeting_no=%r, mode=%r)" % ( + self.meeting_id, + self.meeting_no, + self.mode, + ) + + # -- wiring ---------------------------------------------------------- + def attach(self, source: Any) -> None: + """Bind the source and remember the loop that now owns this session. + + Everything the session owns — the delivery queue, the source's tasks, + the debounce timers — belongs to this loop. ``dispose()`` and + ``leave()`` are public and can be called from anywhere, so they route + the loop-bound parts back here: cancelling a task or waiting on a queue + from a different loop either raises or silently never completes. + """ + self._source = source + self._loop = asyncio.get_event_loop() + self._delivery.start() + source.start() + + + # -- subscriptions --------------------------------------------------- + def on(self, name: str, handler: Callable) -> Unsubscribe: + """Subscribe to a session event. Multicast; returns an unsubscribe. + + An unknown name is a warning rather than an error, matching + ``FeishuChannel.on`` — but it is worth surfacing, because the failure it + produces otherwise is a handler that is simply never called. + """ + if name not in MEETING_EVENT_NAMES: + logger.warning( + "meeting: unknown session event %r; known events are %s", + name, + ", ".join(MEETING_EVENT_NAMES), + ) + self._handlers.setdefault(name, []).append(handler) + + def unsubscribe() -> None: + handlers = self._handlers.get(name) + if not handlers: + return + try: + handlers.remove(handler) + except ValueError: + return + if not handlers: + self._handlers.pop(name, None) + + return unsubscribe + + @property + def dropped_deliveries(self) -> int: + """Events refused by this session's delivery queue.""" + return self._delivery.dropped + + def get_stats(self) -> Dict[str, ActivityTypeStats]: + """Per activity type parse accounting for this session.""" + return dict(self._stats) + + # -- ingestion ------------------------------------------------------- + async def ingest(self, activities: List[Dict[str, Any]]) -> None: + """Unpack and deliver activity objects, in the order given. + + Order is the meaning of some of these: a document swap arrives as + ended-then-started. Both nesting levels are walked in array order and + delivered serially. + """ + if self._closed: + return + if self._source is not None: + self._source.touch() + for activity in activities: + await self._ingest_one(activity) + + async def _ingest_one(self, activity: Dict[str, Any]) -> None: + activity_event_type = activity_type_of(activity) + if activity_event_type is None: + unpack( + activity, + meeting_id=self.meeting_id, + include_raw=self._options.include_raw, + stats=self._stats, + ) + return + items = items_of(activity, activity_event_type) + key = activity_key( + activity, + items, + meeting_id=self.meeting_id, + activity_event_type=activity_event_type, + event_id=activity.get("event_id"), + ) + if self._seen.has_sync(key): + return + self._seen.add_sync(key) + events = unpack( + activity, + meeting_id=self.meeting_id, + include_raw=self._options.include_raw, + stats=self._stats, + ) + for name, event in events: + event.self_echo = self._is_own(event) + if name == "transcript": + self._stabilizer.offer(event) + else: + self._emit(name, event) + + def _is_own(self, event: Any) -> bool: + """Whether this item came from our own bot. + + ``uat`` mode cannot produce an echo: the bot is not in the meeting. + + In ``tat`` mode, before the bot's own id is resolved the honest answer + is "maybe", and "maybe" has to read as ``True`` — ``False`` means + "definitely not me" and lets a reply loop close. + """ + if self.mode != "tat": + return False + own = self._bot_open_id_getter() + if not own: + return True + actor = getattr(event, "actor", None) + return bool(actor is not None and actor.id and actor.id == own) + + def _enqueue_transcript(self, event: Any) -> None: + self._emit("transcript", event) + + def _emit(self, name: str, payload: Any) -> None: + handlers = self._handlers.get(name) + if handlers: + self._delivery.submit(handlers, payload) + + # -- outbound -------------------------------------------------------- + async def send_message(self, text: str) -> None: + """Send a message into the meeting. ``tat`` only.""" + if self.mode != "tat": + raise FeishuChannelError( + FeishuChannelErrorCode.NOT_SUPPORTED, + "send_message needs the bot to be a participant; this session " + "follows a meeting without joining it", + ) + if not self._limiter.try_acquire(): + raise FeishuChannelError( + FeishuChannelErrorCode.RATE_LIMITED, + "in-meeting send budget exhausted for this session", + ) + request = build_bot_message_request( + meeting_id=self.meeting_id, + msg_type="text", + content=json.dumps({"text": text}, ensure_ascii=False), + uuid=str(_uuid.uuid4()), + ) + result = await self._api.call(request, what="in-meeting message") + if not result.ok: + raise self._api.error_for(result, what="in-meeting message") + + # -- teardown -------------------------------------------------------- + def dispose(self, *, reason: str = "disposed") -> None: + """Stop local work. Does **not** depart the meeting. Idempotent. + + The delivery queue is drained afterwards, on the owning loop, because + there is no caller here to await it. + + Synchronous and unconditional: cancelling timers, flushing settled + transcripts, unregistering, and handing the seat back must all happen + whether or not a handler is currently parked. Waiting on the delivery + queue here would let one wedged handler hold a seat forever — and would + hang the timeout meant to rescue that very situation. + """ + if self._closed: + return + # Set here, synchronously, so this is idempotent and stops accepting + # new activity the moment the caller asks — even if the rest has to + # hop to another loop. + self._closed = True + run_on(self._loop, lambda: self._dispose_on_owner_loop(reason, drain=True)) + + def _dispose_on_owner_loop(self, reason: str, *, drain: bool) -> None: + if self._source is not None: + self._source.stop() + # Flushed before the queue stops accepting work, or the last thing + # anybody said in the meeting dies with the debounce timer. + self._stabilizer.flush_all() + self._end(reason) + self._on_teardown(self, reason) + if drain: + # Nobody is awaiting this teardown, so the drain is scheduled. + # A teardown that *is* awaited drains inline instead — draining + # from both places races the departure call, and the loser is the + # error report explaining why the session went away. + asyncio.ensure_future(self.drain()) + + async def leave(self) -> None: + """Depart the meeting, then tear down. Idempotent, and still valid + after ``dispose()``. + + A failed departure does not abort teardown: the moment a meeting ends + is exactly when this call is most likely to 404, and aborting there + would leak a session for every normally-ended meeting. The failure goes + to the ``error`` event instead, and accounting decides separately + whether the seat can be released. + """ + return await await_on(self._loop, self._leave_on_owner_loop) + + async def retire(self, *, reason: str, depart: bool) -> None: + """Internal teardown with an explicit reason. Awaited by its caller. + + The one path that both names a reason other than ``left`` and needs the + meeting departed — the meeting ending, an idle deadline expiring. + """ + return await await_on( + self._loop, lambda: self._retire_on_owner_loop(reason, depart) + ) + + async def _retire_on_owner_loop(self, reason: str, depart: bool) -> None: + if not self._closed: + self._closed = True + self._dispose_on_owner_loop(reason, drain=False) + if depart and not self._left and self.mode == "tat": + self._left = True + await self._depart() + await self.drain() + + async def _leave_on_owner_loop(self) -> None: + if not self._closed: + self._closed = True + self._dispose_on_owner_loop("left", drain=False) + if not self._left and self.mode == "tat": + self._left = True + await self._depart() + # Explicit teardown waits out the bounded drain, so a caller that + # awaited leave() knows the handlers are finished or have been given up + # on. dispose() cannot: it is synchronous by design. + await self.drain() + + async def _depart(self) -> Any: + # Identifies the caller so a superseded handle cannot eject the bot + # from a meeting another session is now serving. + result = await self._depart_call(self.meeting_id, requester=self) + if result is not None and not getattr(result, "ok", False): + await self.report(self._api.error_for(result, what="meeting departure")) + return result + + async def drain(self) -> None: + """Give queued handlers a bounded chance to finish, then stop them. + + Single-flight rather than merely idempotent. Teardown is reachable from + a ``leave()`` the caller awaits *and* from the disposal path at the same + time; a plain "already done" guard would let ``leave()`` return before + the drain it is supposed to have waited for. Both callers await the same + operation, and the timeout is reported once. + """ + if self._drain_task is None: + self._drain_task = asyncio.ensure_future(self._drain_once()) + try: + await asyncio.shield(self._drain_task) + except RuntimeError: + # The drain was started on a loop that has since stopped, so + # awaiting its future from here is not possible. Teardown has + # already done everything that matters — the timers are cancelled, + # the seat is back, the session is unregistered — and this method + # promises not to raise, so the unfinished wait is dropped. + logger.debug( + "meeting: could not await the drain for meeting %s across loops", + sanitize_for_log(self.meeting_id), + ) + except asyncio.CancelledError: + raise + + async def _drain_once(self) -> None: + timeout = self._config.dispose_drain_timeout_seconds + if self._source is not None: + # Bounded: one of these loops may be parked in a call that cannot be + # cancelled promptly, and teardown must not wait on it indefinitely. + try: + await asyncio.wait_for( + self._source.wait_closed(), timeout=max(0.05, timeout) + ) + except (asyncio.TimeoutError, asyncio.CancelledError): + pass + drained = await self._delivery.drain(timeout) + if not drained: + logger.warning( + "meeting: could not drain the delivery queue for meeting %s " + "within %ss; a handler is still running and is being cancelled", + sanitize_for_log(self.meeting_id), + timeout, + ) + self._delivery.cancel() + + def _end(self, reason: str) -> None: + if self._ended: + return + self._ended = True + self._emit("end", MeetingEndEvent(meeting_id=self.meeting_id, reason=reason)) + + async def _report_from_worker(self, error: BaseException) -> None: + """Report a failure raised *inside* the delivery worker. + + Deliberately does not go through the queue. ``_report`` submits to the + error handlers, and this is called when a handler raised — so + submitting would hand the same handlers the same kind of work, they + would raise again, and the queue would feed itself forever while every + real event queued behind it. + """ + resolved = self._as_channel_error(error) + handlers = list(self._handlers.get("error") or ()) + if not handlers: + self._log_error(resolved) + return + await self._invoke_error_handlers(handlers, resolved) + + async def _invoke_error_handlers( + self, handlers: List[Callable], error: FeishuChannelError + ) -> None: + for handler in handlers: + try: + result = handler(error) + if inspect.isawaitable(result): + await result + except Exception: + # An error handler that itself fails is logged, never + # re-dispatched: that is the loop this method exists to avoid. + logger.exception("meeting: an error handler raised") + + async def report(self, error: BaseException) -> None: + """Route a failure to this session's ``error`` subscribers. + + Awaitable on purpose. The fallback path below has to call handlers + directly, and a version of this that *returned* that work left every + caller silently dropping a coroutine — which is how "a failed departure + after a dispose is reported" turned back into "it is swallowed", with + only a never-awaited warning to show for it. + """ + resolved = self._as_channel_error(error) + handlers = list(self._handlers.get("error") or ()) + if not handlers: + self._log_error(resolved) + return + if self._delivery.submit(handlers, resolved, reserved=True): + return + # The queue is already shut down. That is the ordinary case for a + # `leave()` after a `dispose()`: disposal drained and cancelled the + # queue, and the departure call then failed. Dropping the report here + # would silence exactly the failure that explains the state. + await self._invoke_error_handlers(handlers, resolved) + + + @staticmethod + def _as_channel_error(error: BaseException) -> FeishuChannelError: + if isinstance(error, FeishuChannelError): + return error + return FeishuChannelError( + FeishuChannelErrorCode.UNKNOWN, + "%s in a meeting handler" % type(error).__name__, + ) + + def _log_error(self, error: FeishuChannelError) -> None: + """The fallback when nobody is subscribed. + + ``context`` is left out on purpose: it can hold a console link, which is + a signed one-click grant. + """ + fields = log_error_fields(self.meeting_id, error) + logger.error( + "meeting: %s (meeting_id=%s code=%s feishu_code=%s)", + fields["message"], + fields["meeting_id"], + fields["code"], + fields["feishu_code"], + ) + + +__all__ = ["MeetingSession"] diff --git a/lark_channel/channel/meeting/sources/__init__.py b/lark_channel/channel/meeting/sources/__init__.py new file mode 100644 index 0000000..afb7a74 --- /dev/null +++ b/lark_channel/channel/meeting/sources/__init__.py @@ -0,0 +1,6 @@ +"""Event sources: activity pushed while joined, and activity polled while following.""" + +from .poll_source import PollSource +from .push_source import PushSource + +__all__ = ["PollSource", "PushSource"] diff --git a/lark_channel/channel/meeting/sources/poll_source.py b/lark_channel/channel/meeting/sources/poll_source.py new file mode 100644 index 0000000..d922920 --- /dev/null +++ b/lark_channel/channel/meeting/sources/poll_source.py @@ -0,0 +1,257 @@ +"""Following as the user: two loops over REST, under the user's own token. + +The activity loop reads ``bots/events``; the end-detection loop re-checks +``user_active_meeting``, because this path gets no meeting-ended event. + +Three things here are load-bearing: + +**Ticket lookup is non-interactive.** The loop runs every few seconds for the +whole meeting, so anything it does per round it does hundreds of times. The +interactive resolver starts a device flow whenever the stored scopes do not +contain the requested one verbatim — an ordinary state, since ticket scopes are +whatever the platform granted the app — which in a loop means hundreds of +authorization cards or a silent stall inside ``poll``. + +**Failure backoff is counted separately from empty-poll backoff.** A failing +credential must not keep retrying on the three-second cadence: that turns one +leak risk into thousands, and puts sustained invalid-auth traffic in front of +the platform's risk controls, where it can take the application's message path +down with it. + +**The two loops fail asymmetrically, on purpose.** A credential failure stops +both — the ticket is shared, so neither can work. A retryable failure in +end-detection does *not* end the session: those failures are correlated across +every follow session (same endpoint, same cadence, often the same user), so +ending on them would end all of them at once while their transcripts were +flowing fine. The cost is losing meeting-end detection in that window, which is +lighter than killing healthy sessions. +""" + +import asyncio +from typing import Any, Awaitable, Callable, List, Optional + +from lark_channel.api.vc.bot import ( + build_bot_events_request_as_user, + build_user_active_meeting_request, +) +from lark_channel.core.log import logger + +from ..errors import sanitize_for_log + +#: Feishu codes that mean the ticket will not start working again by itself. +CREDENTIAL_FEISHU_CODES = frozenset( + {99991400, 99991401, 99991663, 99991664, 99991665, 99991666, 99991668, 99991672} +) +CREDENTIAL_HTTP_STATUSES = frozenset({401, 403}) + + +def _is_credential_failure(result: Any) -> bool: + if getattr(result, "feishu_code", None) in CREDENTIAL_FEISHU_CODES: + return True + return getattr(result, "status", None) in CREDENTIAL_HTTP_STATUSES + + +class PollSource: + """Polls in-meeting events as the user, and watches for the meeting ending.""" + + mode = "uat" + + def __init__( + self, + *, + meeting_id: str, + api: Any, + config: Any, + resolve_ticket: Callable[[], Awaitable[str]], + deliver: Callable[[List[dict]], Any], + on_ended: Callable[[str], Awaitable[Any]], + on_terminated: Callable[[Any], Awaitable[Any]], + on_error: Callable[[Any], Awaitable[Any]], + ) -> None: + self._meeting_id = meeting_id + self._api = api + self._config = config + self._resolve_ticket = resolve_ticket + self._deliver = deliver + self._on_ended = on_ended + self._on_terminated = on_terminated + self._on_error = on_error + self._tasks: List[asyncio.Task] = [] + self._closing: List[asyncio.Task] = [] + self._page_token: Optional[str] = None + self._running = False + self._terminating = False + self._last_error: Any = None + + def start(self) -> None: + self._running = True + self._tasks.append(asyncio.ensure_future(self._activity_loop())) + if self._config.active_meeting_check_interval_seconds > 0: + self._tasks.append(asyncio.ensure_future(self._end_detection_loop())) + + def stop(self) -> None: + """Stop the loops. Never cancels the caller's own task. + + Same reason as :meth:`PushSource.stop`: teardown is reached *from* one + of these loops, and cancelling the running task would raise inside the + teardown it triggered. + """ + self._running = False + try: + current = asyncio.current_task() + except RuntimeError: # pragma: no cover - no running loop + current = None + for task in self._tasks: + if task is not current and not task.done(): + task.cancel() + # Kept so teardown can wait for the cancellations to land. A cancelled + # task still needs one turn of the loop to unwind, and a loop that stops + # before that turn reports it as destroyed-while-pending — which is both + # noise and a real leak in a long-lived process. + self._closing = [t for t in self._tasks if t is not current] + self._tasks = [] + + async def wait_closed(self) -> None: + """Wait for the cancelled loops to unwind. Safe to call repeatedly.""" + pending = [t for t in self._closing if not t.done()] + self._closing = [] + if pending: + await asyncio.gather(*pending, return_exceptions=True) + + def touch(self) -> None: + """No idle deadline applies to this mode; kept for source parity.""" + + # -- loops ----------------------------------------------------------- + async def _activity_loop(self) -> None: + idle_rounds = 0 + failures = 0 + try: + while self._running: + outcome = await self._poll_once() + if outcome is None: + return + got_events, failed = outcome + if failed: + failures += 1 + if failures >= self._config.poll_max_consecutive_failures: + await self._terminate(self._last_error) + return + await asyncio.sleep(self._failure_delay(failures)) + continue + failures = 0 + if got_events: + idle_rounds = 0 + delay = self._empty_delay(0) + else: + # The ladder is read before the counter moves, so the first + # quiet round waits the floor rather than double it. + delay = self._empty_delay(idle_rounds) + idle_rounds += 1 + await asyncio.sleep(delay) + except asyncio.CancelledError: + raise + + async def _poll_once(self): + """``(got_events, failed)``, or ``None`` when the source terminated.""" + try: + token = await self._resolve_ticket() + except Exception as exc: + await self._terminate(exc) + return None + request = build_bot_events_request_as_user( + meeting_id=self._meeting_id, page_token=self._page_token + ) + result = await self._api.call( + request, user_access_token=token, what="meeting event poll" + ) + if result.ok: + self._page_token = result.data.get("page_token") or self._page_token + events = result.data.get("events") + events = [e for e in events if isinstance(e, dict)] if isinstance(events, list) else [] + if events: + await self._deliver(events) + return bool(events), False + error = self._api.error_for(result, what="meeting event poll") + if _is_credential_failure(result): + await self._terminate(error) + return None + # Reported even though it will be retried: a caller watching `error` + # is watching for exactly this, and staying quiet until the source + # gives up entirely hides a meeting whose transcript has stalled. + self._last_error = error + await self._on_error(error) + return False, True + + async def _end_detection_loop(self) -> None: + interval = self._config.active_meeting_check_interval_seconds + failures = 0 + try: + while self._running: + await asyncio.sleep(interval) + if not self._running: + return + try: + token = await self._resolve_ticket() + except Exception as exc: + await self._terminate(exc) + return + result = await self._api.call( + build_user_active_meeting_request(), + user_access_token=token, + what="active meeting lookup", + ) + if result.ok: + failures = 0 + if not self._still_listed(result): + await self._on_ended("no_longer_active") + return + continue + if _is_credential_failure(result): + # Shared ticket: neither loop can work, so both stop. + await self._terminate( + self._api.error_for(result, what="active meeting lookup") + ) + return + # Retryable: keep checking, never end the session on it. + failures += 1 + await asyncio.sleep(self._failure_delay(failures)) + except asyncio.CancelledError: + raise + + def _still_listed(self, result: Any) -> bool: + from ..coerce import meeting_id_str + + meetings = result.data.get("meetings") + if not isinstance(meetings, list): + return False + for meeting in meetings: + if not isinstance(meeting, dict): + continue + if meeting_id_str(meeting.get("meeting_id")) == self._meeting_id: + return True + return False + + # -- helpers --------------------------------------------------------- + def _empty_delay(self, idle_rounds: int) -> float: + floor = self._config.poll_min_interval_seconds + ceiling = self._config.poll_max_interval_seconds + return min(floor * (2 ** idle_rounds), ceiling) + + def _failure_delay(self, failures: int) -> float: + floor = self._config.poll_min_interval_seconds + ceiling = self._config.poll_failure_max_interval_seconds + return min(floor * (2 ** failures), ceiling) + + async def _terminate(self, error: Any) -> None: + if self._terminating: + return + self._terminating = True + self._running = False + logger.debug( + "meeting: follow source for %s terminating", + sanitize_for_log(self._meeting_id), + ) + await self._on_terminated(error) + + +__all__ = ["CREDENTIAL_FEISHU_CODES", "CREDENTIAL_HTTP_STATUSES", "PollSource"] diff --git a/lark_channel/channel/meeting/sources/push_source.py b/lark_channel/channel/meeting/sources/push_source.py new file mode 100644 index 0000000..b0e0162 --- /dev/null +++ b/lark_channel/channel/meeting/sources/push_source.py @@ -0,0 +1,138 @@ +"""Joined as the bot: activity arrives on the socket, the probe fills the gaps. + +Nothing to poll — the channel's dispatcher routes activity here by meeting id. +What this source owns is the two watchdogs a joined session needs: the liveness +probe (which also backfills) and the optional idle deadline. +""" + +import asyncio +from typing import Any, Callable, List, Optional + +from lark_channel.core.log import logger + +from ..liveness import IN_MEETING, NOT_IN_MEETING + + +class PushSource: + """Watchdogs for a joined session. Activity itself is pushed in.""" + + mode = "tat" + + def __init__( + self, + *, + meeting_id: str, + probe: Any, + probe_interval_seconds: float, + idle_timeout_seconds: float, + deliver: Callable[[List[dict]], Any], + on_absent: Callable[[], Any], + on_idle: Callable[[], Any], + confirm_membership: Callable[[], None], + ) -> None: + self._meeting_id = meeting_id + self._probe = probe + self._probe_interval = probe_interval_seconds + self._idle_timeout = idle_timeout_seconds + self._deliver = deliver + self._on_absent = on_absent + self._on_idle = on_idle + self._confirm_membership = confirm_membership + self._tasks: List[asyncio.Task] = [] + self._closing: List[asyncio.Task] = [] + self._page_token: Optional[str] = None + self._last_activity = 0.0 + self._running = False + + def start(self) -> None: + self._running = True + self.touch() + if self._probe_interval > 0: + self._tasks.append(asyncio.ensure_future(self._probe_loop())) + if self._idle_timeout > 0: + self._tasks.append(asyncio.ensure_future(self._idle_loop())) + + def touch(self) -> None: + """Note that something happened, for the idle deadline.""" + self._last_activity = asyncio.get_event_loop().time() + + def stop(self) -> None: + """Stop the loops. Never cancels the caller's own task. + + Teardown is often triggered *from* one of these loops — an idle + deadline expiring, a probe proving the bot is gone. Cancelling the + running task there would raise at its next ``await``, which is inside + the teardown itself, so the departure call would never be made and the + seat would never come back. The clear flag lets that task finish on its + own instead. + """ + self._running = False + try: + current = asyncio.current_task() + except RuntimeError: # pragma: no cover - no running loop + current = None + for task in self._tasks: + if task is not current and not task.done(): + task.cancel() + # Kept so teardown can wait for the cancellations to land. A cancelled + # task still needs one turn of the loop to unwind, and a loop that stops + # before that turn reports it as destroyed-while-pending — which is both + # noise and a real leak in a long-lived process. + self._closing = [t for t in self._tasks if t is not current] + self._tasks = [] + + async def wait_closed(self) -> None: + """Wait for the cancelled loops to unwind. Safe to call repeatedly.""" + pending = [t for t in self._closing if not t.done()] + self._closing = [] + if pending: + await asyncio.gather(*pending, return_exceptions=True) + + async def _probe_loop(self) -> None: + try: + while self._running: + await asyncio.sleep(self._probe_interval) + if not self._running: + return + verdict, events, next_token = await self._probe.probe( + meeting_id=self._meeting_id, page_token=self._page_token + ) + self._page_token = next_token + if verdict == IN_MEETING: + # Two different clocks. `touch()` is the idle deadline; + # `confirm_membership()` is the accounting deadline, which + # is a backstop for mis-accounting and must be refreshed by + # anything that proves the meeting is still ours. A long + # meeting with no pushed activity would otherwise have its + # seat released while the bot is demonstrably still in it. + self._confirm_membership() + if events: + # Backfill counts as activity: the push path being quiet + # while the probe keeps finding events is exactly the case + # the idle deadline must not reclaim. + self.touch() + self._confirm_membership() + await self._deliver(events) + if verdict == NOT_IN_MEETING: + await self._on_absent() + return + except asyncio.CancelledError: + raise + except Exception as exc: # pragma: no cover - defensive + logger.warning("meeting: liveness loop stopped: %s", type(exc).__name__) + + async def _idle_loop(self) -> None: + try: + while self._running: + await asyncio.sleep(max(0.01, self._idle_timeout / 4.0)) + if not self._running: + return + idle_for = asyncio.get_event_loop().time() - self._last_activity + if idle_for >= self._idle_timeout: + await self._on_idle() + return + except asyncio.CancelledError: + raise + + +__all__ = ["PushSource"] diff --git a/lark_channel/channel/meeting/stabilizer.py b/lark_channel/channel/meeting/stabilizer.py new file mode 100644 index 0000000..46320b8 --- /dev/null +++ b/lark_channel/channel/meeting/stabilizer.py @@ -0,0 +1,81 @@ +"""Transcript settling. + +The protocol has no "this is the final text" marker, so a later item with the +same ``sentence_id`` supersedes the earlier one. With a zero window every +revision is delivered and consumers watch a sentence grow; with a positive +window a sentence is delivered once, after it stops changing for that long. + +Whatever is still buffered has to be flushed when the session goes away. The +debounce timer dies with the session, and if the sentence dies with it the last +thing anybody said in the meeting is silently lost. +""" + +import asyncio +from typing import Any, Callable, Dict, Optional, Tuple + +#: Buffered sentences allowed per session. A meeting with many speakers can +#: hold a lot of sentences open at once; past this the oldest is delivered +#: early rather than dropped, because dropping loses content while delivering +#: early only loses the settling guarantee for one sentence. +MAX_PENDING_TRANSCRIPTS = 256 + + +class TranscriptStabilizer: + """Debounces transcripts by ``sentence_id``.""" + + def __init__( + self, + *, + window_seconds: float, + emit: Callable[[Any], None], + ) -> None: + self._window = window_seconds + self._emit = emit + self._pending: "Dict[str, Tuple[Any, Optional[asyncio.TimerHandle]]]" = {} + + @property + def enabled(self) -> bool: + return self._window > 0 + + def offer(self, event: Any) -> None: + """Deliver ``event`` now, or hold it until its sentence settles.""" + sentence_id = getattr(event, "sentence_id", None) + if not self.enabled or not sentence_id: + self._emit(event) + return + self._cancel(sentence_id) + if len(self._pending) >= MAX_PENDING_TRANSCRIPTS: + self._flush_oldest() + loop = asyncio.get_event_loop() + handle = loop.call_later(self._window, self._settle, sentence_id) + self._pending[sentence_id] = (event, handle) + + def _settle(self, sentence_id: str) -> None: + entry = self._pending.pop(sentence_id, None) + if entry is not None: + self._emit(entry[0]) + + def _cancel(self, sentence_id: str) -> None: + entry = self._pending.pop(sentence_id, None) + if entry is not None and entry[1] is not None: + entry[1].cancel() + + def _flush_oldest(self) -> None: + for sentence_id in list(self._pending): + entry = self._pending.pop(sentence_id) + if entry[1] is not None: + entry[1].cancel() + self._emit(entry[0]) + return + + def flush_all(self) -> None: + """Deliver everything still buffered. Safe to call more than once.""" + pending = list(self._pending.items()) + self._pending.clear() + for _sentence_id, (event, handle) in pending: + if handle is not None: + handle.cancel() + self._emit(event) + + +__all__ = ["MAX_PENDING_TRANSCRIPTS", "TranscriptStabilizer"] diff --git a/lark_channel/channel/meeting/tests/__init__.py b/lark_channel/channel/meeting/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/lark_channel/channel/meeting/tests/conftest.py b/lark_channel/channel/meeting/tests/conftest.py new file mode 100644 index 0000000..73d83c8 --- /dev/null +++ b/lark_channel/channel/meeting/tests/conftest.py @@ -0,0 +1,75 @@ +"""Shared fixtures for the meeting-channel test suite.""" + +import pytest + +from . import fixtures as fx + + +@pytest.fixture +def vc(): + """Fake VC transport, installed for the whole test.""" + fake = fx.FakeVC() + with fake.patched(): + yield fake + + +@pytest.fixture +def make_ch(): + """Channel factory that tears every channel down afterwards. + + Channels own a background thread and an event loop; leaving them running + would let one test's polling loop bleed into the next one's assertions. + """ + created = [] + + def _make(**kwargs): + channel = fx.make_channel(**kwargs) + created.append(channel) + return channel + + yield _make + for channel in created: + try: + channel.stop(join_timeout=1.0) + except Exception: + pass + + +@pytest.fixture +def tat_channel(vc, make_ch): + """Factory for a connected channel, ready for ``join_meeting``.""" + + def _make(**meeting_overrides): + channel = make_ch(meeting=fx.meeting_config(**meeting_overrides)) + fx.mark_connected(channel) + return channel + + return _make + + +@pytest.fixture +def uat_channel(vc, make_ch): + """Factory for an unconnected channel holding a ticket, for ``follow_my_meeting``. + + Returns ``(channel, token_store, device_flow)``. Deliberately does not + connect: the follow path is REST-only and must work without a WebSocket. + """ + + def _make(*, scopes=None, access_token="u-REAL", **meeting_overrides): + store = fx.FakeTokenStore() + store.put( + fx.USER_OPEN_ID, + fx.make_uat( + access_token, + scopes=list(scopes) if scopes is not None else [fx.MEETING_EVENT_SCOPE], + ), + ) + flow = fx.FakeDeviceFlow() + channel = make_ch( + meeting=fx.meeting_config(**meeting_overrides), + token_store=store, + device_flow=flow, + ) + return channel, store, flow + + return _make diff --git a/lark_channel/channel/meeting/tests/fixtures.py b/lark_channel/channel/meeting/tests/fixtures.py new file mode 100644 index 0000000..253af9e --- /dev/null +++ b/lark_channel/channel/meeting/tests/fixtures.py @@ -0,0 +1,1007 @@ +"""Wire-shape fixtures and fakes for the meeting-channel tests. + +Read this before touching any test in this package. + +``push_activity()`` and ``poll_events()`` are deliberately **asymmetric**, +because the platform itself is: + +=============================== ===================================== +push (``vc.bot.meeting_activity_v1``) poll (``GET /vc/v1/bots/events``) +=============================== ===================================== +``*_items`` flattened onto the ``*_items`` nested under +activity object ``events[].payload`` +``meeting.id`` is an **int** ``meeting_id`` is a **str** +actor ``id`` is a **nested actor ``id`` is a **plain string** +dict** ``{open_id, union_id, +user_id}`` +activity items carry **no** every event carries ``event_id`` +``event_id`` +=============================== ===================================== + +Never reshape either builder to match the other, and never rebuild them from +a generated model's type hints. The generated vc models declare +``actor.id: str`` and ``meeting.id: int``; each of those disagrees with what +the platform really sends on at least one of the two transports. A fixture +built from the hints makes ``self_echo`` compare a real open_id against +``""`` forever, and makes session routing drop every pushed event — with the +whole suite still green. ``test_fixtures.py`` pins both shapes so a later +"cleanup" cannot quietly flip them. +""" + +import asyncio +import inspect +import json +import time +from contextlib import contextmanager +from typing import Any, Dict, List, Optional +from unittest.mock import patch + +import httpx + +from lark_channel.channel.auth.device_flow import DeviceFlowInit +from lark_channel.channel.errors import UATAuthError +from lark_channel.channel.types import UAT +from lark_channel.core.http.transport import Transport, _build_header +from lark_channel.core.model import RawResponse +from lark_channel.core.token.manager import TokenManager + +# Captured before anything patches ``asyncio.sleep``, so helpers in this +# module keep working inside tests that fast-forward the SDK's own sleeps. +_REAL_SLEEP = asyncio.sleep + + +# --------------------------------------------------------------------------- +# Well-known identifiers +# --------------------------------------------------------------------------- + +MEETING_NO = "123456789" +#: Long meeting id as the join response and the poll transport spell it. +MEETING_ID_STR = "7654321" +#: The very same meeting, as the push transport spells it. +MEETING_ID_INT = 7654321 + +OTHER_MEETING_NO = "987654321" +OTHER_MEETING_ID_STR = "1234567" +OTHER_MEETING_ID_INT = 1234567 + +BOT_OPEN_ID = "ou_bot_self" +USER_OPEN_ID = "ou_ticket_owner" + +ACTIVITY_EVENT_TYPE = "vc.bot.meeting_activity_v1" +INVITED_EVENT_TYPE = "vc.bot.meeting_invited_v1" +ENDED_EVENT_TYPE = "vc.bot.meeting_ended_v1" + +URI_JOIN = "/open-apis/vc/v1/bots/join" +URI_LEAVE = "/open-apis/vc/v1/bots/leave" +URI_MESSAGE = "/open-apis/vc/v1/bots/message" +URI_EVENTS = "/open-apis/vc/v1/bots/events" +URI_ACTIVE_MEETING = "/open-apis/vc/v1/bots/user_active_meeting" + +MEETING_EVENT_SCOPE = "vc:meeting.meetingevent:read" + +ALL_ACTIVITY_TYPES = ( + "transcript_received", + "chat_received", + "participant_joined", + "participant_left", + "magic_share_started", + "magic_share_ended", + "document_context_changed", +) + + +# --------------------------------------------------------------------------- +# Actors — the single most expensive shape difference between the transports +# --------------------------------------------------------------------------- + + +def actor( + open_id: str = "ou_speaker", + *, + shape: str, + name: str = "Alice", + union_id: Optional[str] = None, + user_id: Optional[str] = None, + user_type: int = 1, + user_role: int = 2, +) -> Dict[str, Any]: + """One participant, spelled the way ``shape`` spells it. + + ``shape="push"`` nests three id namespaces under ``id`` because the push + transport takes no ``user_id_type`` query parameter and therefore hands + back all of them. ``shape="poll"`` returns a bare string because the poll + request pins ``user_id_type=open_id``. + """ + if shape == "push": + ident: Any = { + "open_id": open_id, + "union_id": union_id or open_id.replace("ou_", "on_"), + "user_id": user_id or open_id.replace("ou_", "u_"), + } + elif shape == "poll": + ident = open_id + else: + raise ValueError("shape must be 'push' or 'poll'") + return { + "id": ident, + "name": name, + "user_type": user_type, + "user_role": user_role, + } + + +def actor_without_id(open_id: str = "ou_fallback", *, name: str = "Bob") -> Dict[str, Any]: + """An actor that omits ``id`` entirely, forcing the sibling-field fallback.""" + return {"name": name, "open_id": open_id, "user_id": open_id.replace("ou_", "u_")} + + +# --------------------------------------------------------------------------- +# Activity items (the inner ``*_items`` entries) +# --------------------------------------------------------------------------- + + +def transcript_item( + *, + shape: str, + speaker: Optional[Dict[str, Any]] = None, + text: str = "hello there", + sentence_id: Optional[str] = "sent-1", + language: Optional[str] = "zh_cn", + start_time_ms: Optional[Any] = "1730000000123", + end_time_ms: Optional[Any] = "1730000000456", +) -> Dict[str, Any]: + item: Dict[str, Any] = { + "speaker": speaker if speaker is not None else actor(shape=shape), + "text": text, + } + if sentence_id is not None: + item["sentence_id"] = sentence_id + if language is not None: + item["language"] = language + if start_time_ms is not None: + item["start_time_ms"] = start_time_ms + if end_time_ms is not None: + item["end_time_ms"] = end_time_ms + return item + + +def chat_item( + *, + shape: str, + operator: Optional[Dict[str, Any]] = None, + content: str = "hi from a participant", + message_id: Optional[str] = "om_chat_1", + message_type: Optional[int] = 1, + send_time: Optional[Any] = "1730000000500", +) -> Dict[str, Any]: + item: Dict[str, Any] = { + "operator": operator if operator is not None else actor("ou_chatter", shape=shape), + "content": content, + } + if message_id is not None: + item["message_id"] = message_id + if message_type is not None: + item["message_type"] = message_type + if send_time is not None: + item["send_time"] = send_time + return item + + +def participant_joined_item( + *, + shape: str, + participant: Optional[Dict[str, Any]] = None, + join_time: Optional[Any] = "1730000000600", +) -> Dict[str, Any]: + return { + "participant": participant + if participant is not None + else actor("ou_joiner", shape=shape), + "join_time": join_time, + } + + +def participant_left_item( + *, + shape: str, + participant: Optional[Dict[str, Any]] = None, + leave_time: Optional[Any] = "1730000000700", + leave_reason: Optional[int] = 1, +) -> Dict[str, Any]: + return { + "participant": participant + if participant is not None + else actor("ou_leaver", shape=shape), + "leave_time": leave_time, + "leave_reason": leave_reason, + } + + +def share_doc(url: str = "https://example.test/docx/doc_1", title: str = "Design doc"): + return {"url": url, "title": title} + + +def share_started_item( + *, + shape: str, + operator: Optional[Dict[str, Any]] = None, + share_id: Optional[str] = "share-1", + doc: Optional[Dict[str, Any]] = None, + time_: Optional[Any] = "1730000000800", +) -> Dict[str, Any]: + item: Dict[str, Any] = { + "operator": operator if operator is not None else actor("ou_sharer", shape=shape), + "share_id": share_id, + "time": time_, + } + # The field is `share_doc`, not `doc`. Reading `doc` yields None forever. + item["share_doc"] = doc if doc is not None else share_doc() + return item + + +def share_ended_item(**kwargs) -> Dict[str, Any]: + item = share_started_item(**kwargs) + item["time"] = "1730000000900" + return item + + +def document_context_item( + *, + shape: str, + kind: str, + operator: Optional[Dict[str, Any]] = None, + share_id: Optional[str] = "share-1", + context_type: Optional[str] = None, + section_title: str = "Chapter 2", +) -> Dict[str, Any]: + """``kind`` is one of ``comment_focus`` / ``section_location`` / + ``element_preview`` / ``none``. + + ``kind="none"`` models the platform shipping a fourth kind of context we + have never seen: all three known sub-objects absent. ``context_type`` + models the platform starting to send an explicit discriminator that the + generated model does not have a field for. + """ + item: Dict[str, Any] = { + "operator": operator if operator is not None else actor("ou_editor", shape=shape), + "share_id": share_id, + "share_doc": share_doc(), + "time": "1730000001000", + } + if context_type is not None: + item["context_type"] = context_type + if kind == "comment_focus": + item["comment_focus"] = {"comment_id": "cmt_1", "focused": True} + elif kind == "section_location": + item["section_location"] = { + "title": section_title, + "level": 2, + "parent_titles": ["Chapter 1"], + } + elif kind == "element_preview": + item["element_preview"] = { + "action": "preview", + "element_type": "image", + "element_token": "img_token_1", + "block_id": "blk_1", + } + elif kind != "none": + raise ValueError("unknown document context kind: %s" % kind) + return item + + +_ITEM_BUILDERS = { + "transcript_received": transcript_item, + "chat_received": chat_item, + "participant_joined": participant_joined_item, + "participant_left": participant_left_item, + "magic_share_started": share_started_item, + "magic_share_ended": share_ended_item, +} + + +def default_items(activity_event_type: str, *, shape: str) -> List[Dict[str, Any]]: + """One canonical inner item for the given activity type.""" + if activity_event_type == "document_context_changed": + return [document_context_item(shape=shape, kind="comment_focus")] + builder = _ITEM_BUILDERS.get(activity_event_type) + if builder is None: + # Unknown types carry no items we know how to read. + return [] + return [builder(shape=shape)] + + +# --------------------------------------------------------------------------- +# Activity objects (the outer ``meeting_activity_items`` / ``events`` entries) +# --------------------------------------------------------------------------- + + +def push_item( + activity_event_type: str, + items: Optional[List[Dict[str, Any]]] = None, +) -> Dict[str, Any]: + """One ``meeting_activity_items[]`` entry: ``*_items`` sits flat on it.""" + if items is None: + items = default_items(activity_event_type, shape="push") + return { + "activity_event_type": activity_event_type, + "%s_items" % activity_event_type: list(items), + } + + +def poll_item( + activity_event_type: str, + items: Optional[List[Dict[str, Any]]] = None, + *, + event_id: str = "evt-poll-1", + meeting_id: str = MEETING_ID_STR, +) -> Dict[str, Any]: + """One ``data.events[]`` entry: everything interesting hides in ``payload``.""" + if items is None: + items = default_items(activity_event_type, shape="poll") + return { + "event_id": event_id, + "meeting_id": meeting_id, + "payload": { + "activity_event_type": activity_event_type, + "%s_items" % activity_event_type: list(items), + }, + } + + +def push_activity( + activities: List[Dict[str, Any]], + *, + meeting_id: int = MEETING_ID_INT, + meeting_no: str = MEETING_NO, + topic: str = "Weekly sync", + envelope_event_id: str = "env-1", +) -> Dict[str, Any]: + """A full ``vc.bot.meeting_activity_v1`` p2 envelope. + + ``meeting.id`` stays an ``int`` on purpose. ``envelope_event_id`` is the + dispatcher-level id in the p2 header — it is *not* a per-activity id, so + two pushes carrying byte-identical activity items still arrive with + different envelope ids. + """ + return { + "schema": "2.0", + "header": { + "event_id": envelope_event_id, + "event_type": ACTIVITY_EVENT_TYPE, + "create_time": "1730000000000", + "token": "", + "app_id": "cli_x", + "tenant_key": "tk_1", + }, + "event": { + "meeting": { + "id": meeting_id, + "topic": topic, + "meeting_no": meeting_no, + "start_time": 1730000000, + }, + "meeting_activity_items": list(activities), + }, + } + + +def poll_events( + activities: List[Dict[str, Any]], + *, + page_token: Optional[str] = "page-token-1", + has_more: bool = False, +) -> Dict[str, Any]: + """A full ``GET /vc/v1/bots/events`` response body.""" + return { + "code": 0, + "msg": "success", + "data": { + "events": list(activities), + "page_token": page_token, + "has_more": has_more, + }, + } + + +def push_meeting_invited( + *, + meeting_no: str = MEETING_NO, + meeting_id: int = MEETING_ID_INT, + inviter_open_id: str = "ou_inviter", + topic: str = "Weekly sync", + call_id: str = "call-1", +) -> Dict[str, Any]: + return { + "schema": "2.0", + "header": { + "event_id": "env-invited-1", + "event_type": INVITED_EVENT_TYPE, + "create_time": "1730000000000", + "token": "", + "app_id": "cli_x", + "tenant_key": "tk_1", + }, + "event": { + "meeting": {"id": meeting_id, "meeting_no": meeting_no, "topic": topic}, + "inviter": actor(inviter_open_id, shape="push", name="Inviter"), + "bot": actor(BOT_OPEN_ID, shape="push", name="Helper"), + "call_id": call_id, + "invite_time": 1730000000, + }, + } + + +def push_meeting_ended(*, meeting_id: int = MEETING_ID_INT) -> Dict[str, Any]: + return { + "schema": "2.0", + "header": { + "event_id": "env-ended-1", + "event_type": ENDED_EVENT_TYPE, + "create_time": "1730000000000", + "token": "", + "app_id": "cli_x", + "tenant_key": "tk_1", + }, + "event": { + "meeting": {"id": meeting_id, "meeting_no": MEETING_NO}, + }, + } + + +def active_meeting_body( + meetings: Optional[List[Dict[str, Any]]] = None, +) -> Dict[str, Any]: + if meetings is None: + meetings = [ + { + "meeting_id": MEETING_ID_STR, + "meeting_no": MEETING_NO, + "topic": "Weekly sync", + } + ] + return {"code": 0, "msg": "success", "data": {"meetings": list(meetings)}} + + +def join_body(meeting_id: str = MEETING_ID_STR, *, topic: str = "Weekly sync"): + return { + "code": 0, + "msg": "success", + "data": { + "meeting": { + "id": meeting_id, + "meeting_no": MEETING_NO, + "topic": topic, + } + }, + } + + +# --------------------------------------------------------------------------- +# Fake transport — records the bytes that would go on the wire +# --------------------------------------------------------------------------- + + +class ApiCall: + """One captured outbound request. + + ``headers`` is what the real ``Transport`` would have put on the wire: it + comes out of the transport's own header assembler, so an assertion on + ``headers["Authorization"]`` is an assertion about the request's identity + rather than about some field on ``RequestOption`` that may still be + discarded downstream. + """ + + def __init__(self, conf, request, option, headers): + self.conf = conf + self.request = request + self.option = option + self.headers = headers + # Snapshotted at send time. The SDK scrubs credentials off the request + # object once the call returns, and it does so in place — so reading + # `request.body` afterwards cannot tell "sent, then cleaned up" apart + # from "never sent at all". + body_at_send = getattr(request, "body", None) + self.sent_body = dict(body_at_send) if isinstance(body_at_send, dict) else body_at_send + self.uri = getattr(request, "uri", None) + method = getattr(request, "http_method", None) + self.method = getattr(method, "name", None) + self.queries = list(getattr(request, "queries", []) or []) + self.body = getattr(request, "body", None) + self.token_types = set(getattr(request, "token_types", set()) or set()) + + @property + def authorization(self) -> Optional[str]: + for key, value in self.headers.items(): + if key.lower() == "authorization": + return value + return None + + def query(self, name: str) -> Optional[str]: + for key, value in self.queries: + if key == name: + return value + return None + + def __repr__(self) -> str: # pragma: no cover - debugging aid + return "" % (self.method, self.uri) + + +class FakeVC: + """Routes VC endpoints to canned responses and records every call.""" + + tenant_token = "t-MINTED-TENANT" + + def __init__(self) -> None: + self.calls: List[ApiCall] = [] + self._routes: Dict[str, Any] = {} + self.json(URI_JOIN, join_body()) + self.json(URI_LEAVE, {"code": 0, "msg": "success"}) + self.json(URI_MESSAGE, {"code": 0, "msg": "success"}) + self.json(URI_EVENTS, poll_events([])) + self.json(URI_ACTIVE_MEETING, active_meeting_body()) + + # -- routing --------------------------------------------------------- + def route(self, uri: str, responder) -> None: + """``responder(call) -> (status, body_dict)``, or raises to simulate + a transport-level failure. + + A responder may also return an awaitable, which lets a test hold a + request open — the only way to assert on two calls actually overlapping + rather than on whichever order the scheduler happened to pick. + """ + self._routes[uri] = responder + + def json(self, uri: str, body: Dict[str, Any], *, status: int = 200) -> None: + self.route(uri, lambda call: (status, body)) + + def sequence(self, uri: str, responses: List[Any]) -> None: + """Reply with ``responses`` in order; the last entry repeats. + + Each entry is a body dict, a ``(status, body)`` pair, or an exception + instance to raise. + """ + box = {"i": 0} + + def responder(call): + i = min(box["i"], len(responses) - 1) + box["i"] += 1 + entry = responses[i] + if isinstance(entry, BaseException): + raise entry + if isinstance(entry, tuple): + return entry + return (200, entry) + + self.route(uri, responder) + + # -- inspection ------------------------------------------------------ + def count(self, uri: str) -> int: + return sum(1 for c in self.calls if c.uri == uri) + + def for_uri(self, uri: str) -> List[ApiCall]: + return [c for c in self.calls if c.uri == uri] + + def last(self, uri: str) -> ApiCall: + calls = self.for_uri(uri) + assert calls, "no call recorded for %s" % uri + return calls[-1] + + # -- transport ------------------------------------------------------- + async def aexecute(self, conf, req, option=None): + from lark_channel.core.model import RequestOption + + if option is None: + option = RequestOption() + headers = dict(_build_header(req, option, conf)) + call = ApiCall(conf, req, option, headers) + self.calls.append(call) + responder = self._routes.get(req.uri) + if responder is None: + raise AssertionError("unrouted request: %s %s" % (call.method, req.uri)) + result = responder(call) + if inspect.isawaitable(result): + result = await result + status, body = result + resp = RawResponse() + resp.status_code = status + resp.headers = {"Content-Type": "application/json; charset=utf-8"} + resp.content = json.dumps(body).encode("utf-8") + return resp + + @contextmanager + def patched(self): + """Patch the transport and tenant-token minting for the whole block. + + ``core.token.auth.verify`` is left *unpatched* on purpose: it is the + function that decides which credential a request goes out with, so + stubbing it would erase the very behaviour the identity tests exist + to pin down. Only the network calls underneath it are faked. + """ + with patch.object(Transport, "aexecute", new=self.aexecute), patch.object( + TokenManager, "get_self_tenant_token", new=lambda conf: self.tenant_token + ), patch.object( + TokenManager, "get_self_app_token", new=lambda conf: "a-MINTED-APP" + ): + yield self + + +def error_body(code: int, msg: str = "denied", **extra) -> Dict[str, Any]: + body: Dict[str, Any] = {"code": code, "msg": msg} + body.update(extra) + return body + + +# --------------------------------------------------------------------------- +# Credential fakes +# --------------------------------------------------------------------------- + + +def make_uat( + access_token: str = "u-REAL", + *, + scopes: Optional[List[str]] = None, + refresh_token: Optional[str] = "r-1", + expires_in: Optional[float] = 3600.0, + open_id: str = USER_OPEN_ID, +) -> UAT: + return UAT( + access_token=access_token, + refresh_token=refresh_token, + expires_at=None if expires_in is None else time.time() + expires_in, + refresh_expires_at=time.time() + 30 * 24 * 3600, + scopes=list(scopes) if scopes is not None else [MEETING_EVENT_SCOPE], + open_id=open_id, + ) + + +class FakeTokenStore: + """TokenStore that counts every access and can hand out a rotation.""" + + def __init__(self, initial: Optional[Dict[str, UAT]] = None) -> None: + self.data: Dict[str, UAT] = dict(initial or {}) + self.get_calls: List[str] = [] + self.set_calls: List[Any] = [] + self.delete_calls: List[str] = [] + self._rotation: Dict[str, List[UAT]] = {} + self._rotation_pos: Dict[str, int] = {} + + def put(self, user_id: str, token: UAT) -> None: + self.data[user_id] = token + + def rotate(self, user_id: str, tokens: List[UAT]) -> None: + """Serve ``tokens`` one per ``get``; the last one repeats.""" + self._rotation[user_id] = list(tokens) + self._rotation_pos[user_id] = 0 + + async def get(self, user_id: str) -> Optional[UAT]: + self.get_calls.append(user_id) + rotation = self._rotation.get(user_id) + if rotation: + pos = min(self._rotation_pos[user_id], len(rotation) - 1) + self._rotation_pos[user_id] += 1 + return rotation[pos] + return self.data.get(user_id) + + async def set(self, user_id: str, token: UAT) -> None: + self.set_calls.append((user_id, token)) + self.data[user_id] = token + + async def delete(self, user_id: str) -> None: + self.delete_calls.append(user_id) + self.data.pop(user_id, None) + + +class FakeDeviceFlow: + """DeviceFlowClient stand-in with counters and controllable refresh.""" + + def __init__( + self, + *, + refresh_results: Optional[List[Any]] = None, + poll_result: Optional[UAT] = None, + ) -> None: + self.start_calls: List[Any] = [] + self.poll_calls: List[Any] = [] + self.refresh_calls: List[str] = [] + self._refresh_results = list(refresh_results or []) + self._poll_result = poll_result + + async def start(self, scopes) -> DeviceFlowInit: + self.start_calls.append(list(scopes or [])) + return DeviceFlowInit( + verification_uri="https://example.test/device", + verification_uri_complete="https://example.test/device?code=ABC", + user_code="ABC-123", + device_code="dev-1", + expires_in=600, + interval=1, + ) + + async def poll(self, device_code, interval=None, timeout_seconds=None) -> UAT: + self.poll_calls.append(device_code) + if self._poll_result is None: + raise UATAuthError("device flow was not authorized") + return self._poll_result + + async def refresh(self, refresh_token: str) -> UAT: + self.refresh_calls.append(refresh_token) + if not self._refresh_results: + raise UATAuthError("refresh not configured") + idx = min(len(self.refresh_calls) - 1, len(self._refresh_results) - 1) + outcome = self._refresh_results[idx] + if isinstance(outcome, BaseException): + raise outcome + return outcome + + async def close(self) -> None: + return None + + +def httpx_connect_error(token: str = "u-secret") -> httpx.ConnectError: + """A transport failure carrying a bare credential on its request. + + ``httpx`` — not ``requests`` — is what ``Transport.aexecute`` uses, and an + ``httpx`` exception keeps the outgoing request (headers included) reachable + through attribute walks. ``repr()`` of this object is clean, so a redaction + check that only inspects ``repr()`` proves nothing. + """ + request = httpx.Request( + "GET", + "https://open.feishu.cn" + URI_EVENTS, + headers={"Authorization": "Bearer %s" % token}, + ) + return httpx.ConnectError("connection refused", request=request) + + +# --------------------------------------------------------------------------- +# Channel harness +# --------------------------------------------------------------------------- + + +def meeting_config(**overrides): + from lark_channel.channel.config import MeetingChannelConfig + + return MeetingChannelConfig(**overrides) + + +def make_channel( + *, + meeting=None, + policy=None, + inbound=None, + token_store=None, + device_flow=None, + app_id: str = "cli_x", + app_secret: str = "secret", +): + """Build a channel wired for meeting tests. Does not touch the network.""" + from lark_channel.channel import FeishuChannel + from lark_channel.channel.config import ChannelConfig + + cfg = ChannelConfig() + if meeting is not None: + cfg.meeting = meeting + if policy is not None: + cfg.policy = policy + if inbound is not None: + cfg.inbound = inbound + channel = FeishuChannel( + app_id=app_id, + app_secret=app_secret, + config=cfg, + token_store=token_store, + ) + if device_flow is not None: + channel._device_flow = device_flow + return channel + + +def mark_connected(channel, *, bot_open_id: Optional[str] = BOT_OPEN_ID): + """Put the channel into the state ``connect()`` would leave it in. + + Real ``connect()`` opens a WebSocket, so tests fake the post-connect + state instead: transport started, readiness flipped, dispatcher built + (which is also what registers the internal ``vc.bot.*`` processors), and + bot identity resolved so ``self_echo`` has something to compare against. + """ + from lark_channel.channel.bot_identity import BotIdentity + + channel._ensure_bg_loop() + channel._started = True + if bot_open_id is not None: + identity = BotIdentity(open_id=bot_open_id, name="Helper") + channel._store_bot_identity(identity) + channel._dispatcher = channel._build_dispatcher() + channel._mark_ready() + return channel + + +def deliver(channel, payload: Dict[str, Any]): + """Feed one p2 envelope through the channel's real dispatcher.""" + return channel.dispatcher._do_without_validation( + json.dumps(payload).encode("utf-8") + ) + + +async def wait_for(predicate, *, timeout: float = 3.0, what: str = "condition"): + """Poll ``predicate`` until true. Uses the pre-patch ``asyncio.sleep`` so + it still works in tests that fast-forward the SDK's own sleeps.""" + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if predicate(): + return + await _REAL_SLEEP(0.005) + raise AssertionError("timed out after %ss waiting for %s" % (timeout, what)) + + +async def settle(rounds: int = 6): + """Let queued work on this loop (and the channel's bg loop) drain.""" + for _ in range(rounds): + await _REAL_SLEEP(0.01) + + +class SleepRecorder: + """Records every ``asyncio.sleep`` duration and returns immediately. + + After ``max_sleeps`` the caller is parked on a bounded real sleep so a + polling loop stops spinning; a session teardown cancels it. The park is + bounded rather than infinite so a missing cancel shows up as a slow test + instead of a hung suite. + """ + + park_seconds = 5.0 + + def __init__(self, *, max_sleeps: Optional[int] = None) -> None: + self.durations: List[float] = [] + self._max = max_sleeps + + def between(self, low: float, high: float) -> List[float]: + return [d for d in self.durations if low <= d <= high] + + async def __call__(self, delay=0, *args, **kwargs): + self.durations.append(delay) + if self._max is not None and len(self.durations) > self._max: + await _REAL_SLEEP(self.park_seconds) + return None + await _REAL_SLEEP(0) + return None + + +@contextmanager +def fast_sleep(*, max_sleeps: Optional[int] = None): + """Collapse ``asyncio.sleep`` for the duration of the block.""" + recorder = SleepRecorder(max_sleeps=max_sleeps) + with patch("asyncio.sleep", new=recorder): + yield recorder + + +# --------------------------------------------------------------------------- +# Deep inspection helpers for the credential-hygiene checks +# --------------------------------------------------------------------------- + +_FOLLOWED_ATTRS = ( + "__cause__", + "__context__", + # A raised error carries the frames it unwound through, and every frame + # exposes its locals. That is how a credential stays reachable from an + # error whose repr is spotless — see `_frame_locals` below. + "__traceback__", + "tb_frame", + "tb_next", + "f_locals", + "args", + "request", + "response", + "headers", + "cause", + "context", +) + + +def deep_strings(root: Any, *, max_depth: int = 10, exclude=()) -> List[str]: + """Every string reachable from ``root`` within ``max_depth`` hops. + + Walks exception chains, ``__dict__``, mappings, sequences, and the + attribute names a transport exception hangs its request on — which is + how a credential survives "the repr looks fine". + + ``exclude`` prunes specific objects by identity. Use it when an object + graph loops back to a place a credential is legitimately allowed to live + (the ticket store), so the walk answers "is it stored *here*" rather than + "is it stored anywhere in the process". + """ + found: List[str] = [] + seen = set(id(item) for item in exclude) + + def walk(obj: Any, depth: int) -> None: + if depth > max_depth or obj is None: + return + oid = id(obj) + if oid in seen: + return + seen.add(oid) + if isinstance(obj, str): + found.append(obj) + return + if isinstance(obj, (bytes, bytearray)): + found.append(bytes(obj).decode("utf-8", "replace")) + return + if isinstance(obj, (int, float, bool)): + return + items = getattr(obj, "items", None) + if callable(items): + try: + pairs = list(items()) + except Exception: + pairs = [] + for key, value in pairs: + walk(key, depth + 1) + walk(value, depth + 1) + return + if isinstance(obj, (list, tuple, set, frozenset)): + for item in obj: + walk(item, depth + 1) + return + for attr in _FOLLOWED_ATTRS: + if hasattr(obj, attr): + try: + value = getattr(obj, attr) + except Exception: + continue + if attr == "f_locals": + # Snapshotted: a live frame's mapping mutates while walked. + try: + value = dict(value) + except Exception: + continue + walk(value, depth + 1) + state = getattr(obj, "__dict__", None) + if isinstance(state, dict): + for key, value in state.items(): + walk(key, depth + 1) + walk(value, depth + 1) + + walk(root, 0) + return found + + +def json_dump_all(*values: Any) -> str: + return json.dumps(values, default=str, ensure_ascii=False) + + +def record_text(record) -> str: + """Everything a log record can put in front of a human or a log file.""" + parts = [str(record.msg), record.getMessage()] + args = record.args + if isinstance(args, dict): + parts.append(json_dump_all(args)) + elif args: + parts.append(json_dump_all(*args)) + return "\n".join(parts) + + +CONTROL_CHARS = tuple( + [chr(c) for c in range(0x00, 0x20)] + [chr(c) for c in range(0x7F, 0xA0)] +) + + +def follow_ready_channel(*, meeting=None, scopes=None, access_token: str = "u-REAL"): + """A channel that already holds a usable ticket for ``USER_OPEN_ID``. + + Returns ``(channel, token_store, device_flow)``. The device flow is a fake + with counters and no configured outcome, so any accidental interactive + authorization shows up as a recorded call (and then fails loudly) instead + of silently working. + """ + store = FakeTokenStore() + store.put( + USER_OPEN_ID, + make_uat( + access_token, + scopes=list(scopes) if scopes is not None else [MEETING_EVENT_SCOPE], + ), + ) + flow = FakeDeviceFlow() + channel = make_channel(meeting=meeting, token_store=store, device_flow=flow) + return channel, store, flow diff --git a/lark_channel/channel/meeting/tests/test_coerce.py b/lark_channel/channel/meeting/tests/test_coerce.py new file mode 100644 index 0000000..0d35ed9 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_coerce.py @@ -0,0 +1,69 @@ +"""Identifier coercion: actor ids arrive in two shapes, meeting ids in two types.""" + +from lark_channel.channel.meeting.coerce import actor_id, meeting_id_str, to_ms + +from . import fixtures as fx + + +def test_actor_id_prefers_open_id_when_the_id_field_is_a_nested_object(): + resolved = actor_id( + {"id": {"open_id": "ou_x", "union_id": "on_y", "user_id": "u_z"}} + ) + assert resolved == "ou_x" + + +def test_actor_id_falls_back_through_union_id_then_user_id(): + assert actor_id({"id": {"union_id": "on_y", "user_id": "u_z"}}) == "on_y" + assert actor_id({"id": {"user_id": "u_z"}}) == "u_z" + + +def test_actor_id_accepts_a_plain_string_id(): + assert actor_id({"id": "ou_x"}) == "ou_x" + + +def test_actor_id_falls_back_to_sibling_open_id_when_id_is_absent(): + assert actor_id(fx.actor_without_id("ou_fallback")) == "ou_fallback" + + +def test_actor_id_never_returns_a_non_string(): + # An empty result is allowed, a dict leaking through is not: downstream + # compares this value against the bot's own open_id. + for candidate in ({}, {"id": {}}, {"id": None}, None): + resolved = actor_id(candidate) + assert isinstance(resolved, str) or resolved is None + + +def test_meeting_id_normalizes_int_and_str_to_the_same_string(): + assert meeting_id_str(fx.MEETING_ID_INT) == fx.MEETING_ID_STR + assert meeting_id_str(fx.MEETING_ID_STR) == fx.MEETING_ID_STR + + +def test_to_ms_parses_string_digits_and_never_raises_on_garbage(): + assert to_ms("1730000000123") == 1730000000123 + assert to_ms(1730000000123) == 1730000000123 + assert to_ms("abc") is None + assert to_ms(None) is None + assert to_ms("") is None + + +async def test_int_meeting_id_in_a_push_routes_to_a_session_keyed_by_string( + vc, tat_channel +): + """The join response spells the meeting id ``"7654321"``; the push spells + the same meeting ``7654321``. Without normalization the session lookup + misses and every pushed activity is dropped in silence.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + assert session.meeting_id == fx.MEETING_ID_STR + + seen = [] + session.on("transcript", lambda event: seen.append(event)) + fx.deliver( + channel, + fx.push_activity( + [fx.push_item("transcript_received")], meeting_id=fx.MEETING_ID_INT + ), + ) + + await fx.wait_for(lambda: seen, what="a transcript routed from an int meeting id") + assert seen[0].meeting_id == fx.MEETING_ID_STR diff --git a/lark_channel/channel/meeting/tests/test_credential_hygiene.py b/lark_channel/channel/meeting/tests/test_credential_hygiene.py new file mode 100644 index 0000000..811bd14 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_credential_hygiene.py @@ -0,0 +1,502 @@ +"""Credentials, capability links, and untrusted meeting content. + +This is the first path in the channel that handles a user access token, and it +does so on a loop, so every leak here repeats for the life of the meeting. Two +things make "I printed it and it looked clean" worthless as evidence: + +* the transport exception's ``repr()`` hides the outgoing headers, while the + token stays reachable by walking attributes — which is exactly what crash + reporters do to an exception chain; +* the redaction layer moves a secret out of the message template but does not + strip control characters, so a forged log line still lands in the file. + +So the assertions walk object graphs and compare formatted output, not reprs. +""" + +import json +import logging + +import pytest + +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode +from lark_channel.channel.meeting import MeetingOptions +from lark_channel.channel.meeting.errors import safe_console_url, sanitize_for_log +from lark_channel.core.log import redact_for_log + +from . import fixtures as fx + +_CONSOLE_URL = "https://open.feishu.cn/app/cli_x/auth?q=vc%3Ameeting&sig=SECRETSIG" +_CONTROL_SAMPLES = [ + "hello\nInfo: fake log line", + "hello\rcarriage", + "hello\x1b[31mred", + "hello\x00null", + "hello\x85nel", +] + + +def _has_control_chars(text): + return any(char in text for char in fx.CONTROL_CHARS if char != "\t") + + +# --------------------------------------------------------------------------- +# Tokens +# --------------------------------------------------------------------------- + + +async def test_a_transport_failure_never_logs_the_token(vc, uat_channel, caplog): + channel, _store, _flow = uat_channel( + access_token="u-secret", active_meeting_check_interval_seconds=300.0 + ) + with fx.fast_sleep(max_sleeps=8): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + errors = [] + session.on("error", lambda err: errors.append(err)) + with caplog.at_level(logging.DEBUG, logger="Lark"): + vc.route( + fx.URI_EVENTS, + lambda call: (_ for _ in ()).throw(fx.httpx_connect_error("u-secret")), + ) + await fx.wait_for(lambda: errors, what="the poll failure") + session.dispose() + + for record in caplog.records: + assert "u-secret" not in fx.record_text(record), record.getMessage() + + +async def test_the_error_handed_to_the_business_carries_no_token_anywhere( + vc, uat_channel +): + """This object goes straight to whatever the application reports errors + with, and those tools walk the cause chain.""" + channel, _store, _flow = uat_channel( + access_token="u-secret", active_meeting_check_interval_seconds=300.0 + ) + errors = [] + with fx.fast_sleep(max_sleeps=8): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("error", lambda err: errors.append(err)) + vc.route( + fx.URI_EVENTS, + lambda call: (_ for _ in ()).throw(fx.httpx_connect_error("u-secret")), + ) + await fx.wait_for(lambda: errors, what="the poll failure") + session.dispose() + + error = errors[0] + assert isinstance(error, FeishuChannelError) + assert "u-secret" not in repr(error) + assert "u-secret" not in json.dumps(error, default=str) + reachable = fx.deep_strings(error, max_depth=10) + assert not any("u-secret" in text for text in reachable), reachable + + +async def test_a_session_does_not_keep_a_copy_of_the_token(vc, uat_channel): + """The ticket store is the one place a token is allowed to live; a token + parked on a session outlives the request it was minted for.""" + channel, store, _flow = uat_channel( + access_token="u-secret", active_meeting_check_interval_seconds=300.0 + ) + with fx.fast_sleep(max_sleeps=6): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 1, what="at least one poll" + ) + session.dispose() + + assert "u-secret" not in repr(session) + reachable = fx.deep_strings(session, max_depth=10, exclude=(channel, store)) + assert not any("u-secret" in text for text in reachable), reachable + + +async def test_meeting_passwords_stay_out_of_logs_events_and_the_session( + vc, tat_channel, caplog +): + """Both directions count as a credential: the one handed to join, and the + one some meeting queries hand back in their response body.""" + response = fx.join_body() + response["data"]["meeting"]["password"] = "resp-s3cr3t" + vc.json(fx.URI_JOIN, response) + + channel = tat_channel() + with caplog.at_level(logging.DEBUG, logger="Lark"): + session = await channel.join_meeting( + fx.MEETING_NO, + password="given-s3cr3t", + options=MeetingOptions(include_raw=True), + ) + got = [] + session.on("transcript", lambda event: got.append(event)) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: got, what="the transcript") + + for secret in ("given-s3cr3t", "resp-s3cr3t"): + for record in caplog.records: + assert secret not in fx.record_text(record), record.getMessage() + assert secret not in repr(session) + assert secret not in json.dumps(got[0].raw, default=str) + reachable = fx.deep_strings(session, max_depth=10, exclude=(channel,)) + assert not any(secret in text for text in reachable), reachable + + +async def test_a_poll_failure_with_no_error_handler_stays_contained( + vc, uat_channel, caplog +): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + with fx.fast_sleep(max_sleeps=8): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + with caplog.at_level(logging.DEBUG, logger="Lark"): + vc.route( + fx.URI_EVENTS, + lambda call: (_ for _ in ()).throw(fx.httpx_connect_error()), + ) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 2, what="a couple of poll attempts" + ) + session.dispose() + + for needle in ("Task exception was never retrieved", "background task raised"): + assert needle not in caplog.text + + +async def test_the_fallback_error_log_stays_minimal(vc, tat_channel, caplog): + """With no error handler registered the failure is logged instead, and that + log must not become the leak the error object was cleaned up to avoid.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + vc.route( + fx.URI_LEAVE, + lambda call: ( + 403, + { + "code": 99991672, + "msg": "no permission", + "console_url": _CONSOLE_URL, + }, + ), + ) + + with caplog.at_level(logging.DEBUG, logger="Lark"): + await session.leave() + await fx.settle() + + logged = "\n".join(fx.record_text(record) for record in caplog.records) + assert "SECRETSIG" not in logged + assert _CONSOLE_URL not in logged + assert fx.MEETING_ID_STR in logged + assert "99991672" in logged + + +# --------------------------------------------------------------------------- +# The console link is itself a credential +# --------------------------------------------------------------------------- + + +def test_the_redaction_layer_masks_console_links(): + assert redact_for_log({"console_url": _CONSOLE_URL}) == {"console_url": "***"} + assert redact_for_log({"consoleUrl": _CONSOLE_URL}) == {"consoleUrl": "***"} + + +def test_a_legitimate_console_link_survives_byte_for_byte(): + """It is a signed one-click link whose contents are opaque; re-encoding or + reassembling any part of it makes it stop working.""" + assert safe_console_url(_CONSOLE_URL) == _CONSOLE_URL + + +@pytest.mark.parametrize( + "candidate", + [ + "javascript:alert(1)", + "data:text/html;base64,PHNjcmlwdD4=", + "\tjava\nscript:alert(1)", + "http://open.feishu.cn/app", + ], +) +def test_a_non_https_console_link_is_dropped(candidate): + """The domain this arrives from is configurable, so the field is not a + trusted source, and downstream renders it as a link.""" + assert safe_console_url(candidate) is None + + +def test_a_console_link_with_embedded_userinfo_is_dropped(): + """The whole point of the field is that an administrator clicks it, and a + link whose prefix reads like the official domain is a ready-made lure.""" + assert safe_console_url("https://open.feishu.cn@elsewhere.example/x") is None + + +def test_an_unparsable_console_link_is_dropped_without_raising(): + """This validation runs while an error object is being constructed, so + raising here replaces the real API failure with a parsing failure.""" + assert safe_console_url("https://[::1") is None + + +async def test_a_valid_console_link_reaches_the_error_context_unchanged( + vc, tat_channel +): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + vc.route( + fx.URI_MESSAGE, + lambda call: ( + 403, + { + "code": 99991672, + "msg": "no permission", + "console_url": _CONSOLE_URL, + "error": {"console_url": _CONSOLE_URL}, + }, + ), + ) + + with pytest.raises(FeishuChannelError) as excinfo: + await session.send_message("hello") + + assert excinfo.value.context["console_url"] == _CONSOLE_URL + + +async def test_an_unparsable_console_link_does_not_hide_the_api_failure( + vc, tat_channel +): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + vc.route( + fx.URI_MESSAGE, + lambda call: ( + 403, + {"code": 99991672, "msg": "no permission", "console_url": "https://[::1"}, + ), + ) + + with pytest.raises(FeishuChannelError) as excinfo: + await session.send_message("hello") + + assert "console_url" not in (excinfo.value.context or {}) + assert "99991672" in str(excinfo.value) or excinfo.value.context.get("feishu_code") == 99991672 + + +# --------------------------------------------------------------------------- +# Meeting content is untrusted input +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("sample", _CONTROL_SAMPLES) +def test_sanitizing_removes_every_control_character(sample): + cleaned = sanitize_for_log(sample) + assert not _has_control_chars(cleaned) + assert "hello" in cleaned + + +async def test_a_forged_meeting_title_cannot_forge_a_log_line( + vc, uat_channel, caplog +): + """Passing untrusted text as a formatting argument only moves it out of the + message template; the logging layer still formats it into the same output + line, so the assertion has to be on the formatted result.""" + hostile = "Standup\nInfo: [Lark] all clear\x1b[31m" + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + vc.json( + fx.URI_ACTIVE_MEETING, + fx.active_meeting_body( + [ + { + "meeting_id": fx.MEETING_ID_STR, + "meeting_no": fx.MEETING_NO, + "topic": "First", + }, + { + "meeting_id": fx.OTHER_MEETING_ID_STR, + "meeting_no": fx.OTHER_MEETING_NO, + "topic": hostile, + }, + ] + ), + ) + + with caplog.at_level(logging.DEBUG, logger="Lark"): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.settle() + session.dispose() + + for record in caplog.records: + assert hostile not in str(record.msg) + assert not _has_control_chars(record.getMessage()) + + +async def test_a_join_failure_leaves_no_credential_in_its_traceback(vc, tat_channel): + """A raised error carries the frames it unwound through, and each frame + exposes its locals. So an error whose ``repr`` is spotless can still hand a + crash reporter the password that caused the failure — and a wrong password + is the most ordinary way for this call to fail.""" + channel = tat_channel() + vc.json(fx.URI_JOIN, fx.error_body(120002, "wrong password"), status=400) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.MEETING_NO, password="given-s3cr3t") + + error = excinfo.value + assert "given-s3cr3t" not in repr(error) + # The outermost frame is this test's own, so the walk would otherwise reach + # the recording transport's send-time snapshot. Excluding those handles keeps + # the question "is it reachable through anything the SDK owns" — note that + # `exclude` only skips those objects themselves, so any subtree still + # reachable from SDK state is still walked. + reachable = fx.deep_strings(error, max_depth=12, exclude=(channel, vc)) + assert not any("given-s3cr3t" in text for text in reachable), [ + text for text in reachable if "given-s3cr3t" in text + ] + + +async def test_a_follow_failure_leaves_no_ticket_in_its_traceback(vc, uat_channel): + """"No active meeting" is the everyday failure on the follow path, and the + frames it unwinds through are the ones holding the user's ticket.""" + channel, store, _flow = uat_channel(access_token="u-secret") + vc.json(fx.URI_ACTIVE_MEETING, fx.active_meeting_body([])) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + + # The outermost frame in the traceback is this test's own, so the walk would + # otherwise reach the recording transport and the ticket store — the one + # place a token is supposed to live. Excluding them keeps the question + # "is it reachable through anything the SDK owns". + reachable = fx.deep_strings( + excinfo.value, max_depth=12, exclude=(channel, store, vc) + ) + assert not any("u-secret" in text for text in reachable), [ + text for text in reachable if "u-secret" in text + ] + + +async def test_the_password_is_actually_sent_and_only_then_cleaned_up( + vc, tat_channel +): + """Two halves of one property, and the second is worthless without the + first: cleanup happens in place on the request object, so a scrub that ran + too early would look identical to a correct one — while password-protected + meetings silently failed to join.""" + channel = tat_channel() + + session = await channel.join_meeting(fx.MEETING_NO, password="given-s3cr3t") + assert session is not None + + call = vc.last(fx.URI_JOIN) + # It really went out. + assert call.sent_body["password"] == "given-s3cr3t" + # And it is gone from the request object afterwards, which is what keeps it + # out of any error raised on this path. + assert (call.request.body or {}).get("password") is None + assert getattr(call.option, "tenant_access_token", None) is None + + +def test_the_reachability_walk_can_actually_find_things(): + """A positive control. The two checks above prove "not found", and a walk + that silently stopped finding anything — a depth limit, a pruned attribute — + would keep proving it forever.""" + sentinel = "sentinel-value-42" + error = FeishuChannelError( + FeishuChannelErrorCode.UNKNOWN, "carrier", context={"probe": sentinel} + ) + + assert any(sentinel in text for text in fx.deep_strings(error, max_depth=10)) + + +async def test_a_departure_failure_after_a_dispose_still_reaches_the_handler( + vc, tat_channel +): + """The documented shutdown order is dispose-then-leave, and disposal + cancels the delivery queue — so this report has nowhere to be queued. An + implementation that hands the fallback work back to its caller instead of + doing it swallows the failure and leaves only a never-awaited warning.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + errors = [] + session.on("error", lambda err: errors.append(err)) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + session.dispose() + await fx.settle() + await session.leave() + + await fx.wait_for(lambda: errors, what="the departure failure") + assert isinstance(errors[0], FeishuChannelError) + + +async def test_an_async_error_handler_actually_runs(vc, tat_channel): + """Every error handler in this suite used to be a synchronous lambda, which + is why a version that returned the first coroutine instead of awaiting it + looked correct.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + seen = [] + + async def on_error(err): + seen.append(err) + + session.on("error", on_error) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + await session.leave() + + await fx.wait_for(lambda: seen, what="the async error handler to run") + + +async def test_an_error_handler_that_raises_does_not_wedge_the_queue( + vc, tat_channel +): + """Reporting a handler failure by handing it to the error handlers means the + error handlers can produce more of the same. Routing that back through the + queue feeds it forever, and every real event queues behind it.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + def explode(_event): + raise RuntimeError("handler bug") + + session.on("transcript", explode) + session.on("error", explode) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.settle() + + # The queue still works afterwards. + chats = [] + session.on("chat", lambda event: chats.append(event)) + fx.deliver( + channel, + fx.push_activity([fx.push_item("chat_received")], envelope_event_id="env-2"), + ) + await fx.wait_for(lambda: chats, what="a later event to still be delivered") + + +def _join_reply_missing_id(*, echoed_password: str): + """A join reply that echoes the password back but carries no meeting id.""" + body = fx.join_body() + body["data"]["meeting"].pop("id") + body["data"]["meeting"]["password"] = echoed_password + return body + + +async def test_a_join_that_answers_without_an_id_leaks_no_echoed_password( + vc, tat_channel +): + """The password comes back in the response, so it can leak outbound too. + + `docs/security.md` promises meeting passwords stay out of error objects in + both directions. The inbound direction has its own failure shape: a reply + that carries a meeting object — password echoed — but no id, which the join + path rejects. That raise unwinds through the frame holding the decoded + response, so an implementation that keeps a reference to it hands the + password to anything reading frame locals. + """ + channel = tat_channel() + # Built in a helper, not a local: the outermost frame in the traceback is + # this test's own, so a reply held here would answer the question with the + # test's own bookkeeping rather than with anything the SDK kept. + vc.json(fx.URI_JOIN, _join_reply_missing_id(echoed_password="echoed-s3cr3t")) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.MEETING_NO, password="given-s3cr3t") + + reachable = fx.deep_strings(excinfo.value, max_depth=12, exclude=(channel, vc)) + assert not any("echoed-s3cr3t" in text for text in reachable), [ + text for text in reachable if "echoed-s3cr3t" in text + ] diff --git a/lark_channel/channel/meeting/tests/test_dedup.py b/lark_channel/channel/meeting/tests/test_dedup.py new file mode 100644 index 0000000..d03425f --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_dedup.py @@ -0,0 +1,127 @@ +"""Suppressing repeats without suppressing distinct content.""" + +from . import fixtures as fx + + +async def test_repeated_event_id_is_delivered_once(vc, uat_channel): + channel, _store, _flow = uat_channel() + got = [] + body = fx.poll_events([fx.poll_item("transcript_received", event_id="evt-1")]) + with fx.fast_sleep(): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("transcript", lambda event: got.append(event)) + vc.sequence(fx.URI_EVENTS, [body, body, fx.poll_events([])]) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 3, what="three polling rounds" + ) + await fx.settle() + session.dispose() + + assert len(got) == 1 + + +async def test_identical_pushed_item_without_event_id_is_delivered_once( + vc, tat_channel +): + """Pushed activity items carry no id of their own, so the only thing that + can catch a redelivery is a key synthesized from the content.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event)) + + for envelope_id in ("env-a", "env-b"): + fx.deliver( + channel, + fx.push_activity( + [fx.push_item("transcript_received")], envelope_event_id=envelope_id + ), + ) + await fx.wait_for(lambda: got, what="the first transcript") + await fx.settle() + + assert len(got) == 1 + + +async def test_growing_sentence_is_delivered_every_time_and_keeps_its_sentence_id( + vc, tat_channel +): + """A sentence id is an upsert handle, not a dedup key: the platform resends + the same sentence as the speaker keeps talking and the text grows.""" + channel = tat_channel(stabilize_seconds=0.0) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event)) + + for text in ("今天", "今天来讨论"): + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item( + "transcript_received", + [fx.transcript_item(shape="push", text=text, sentence_id="s-1")], + ) + ] + ), + ) + await fx.wait_for(lambda: len(got) >= 2, what="both revisions of the sentence") + + assert [event.text for event in got] == ["今天", "今天来讨论"] + assert all(event.sentence_id == "s-1" for event in got) + + +async def test_same_sentence_in_two_parallel_meetings_is_not_cross_suppressed( + vc, tat_channel +): + """Two meetings running at once will produce byte-identical greetings from + the same person seconds apart. A dedup key without the meeting in it makes + one meeting's transcript vanish from the other.""" + channel = tat_channel() + vc.sequence( + fx.URI_JOIN, + [fx.join_body(fx.MEETING_ID_STR), fx.join_body(fx.OTHER_MEETING_ID_STR)], + ) + first = await channel.join_meeting(fx.MEETING_NO) + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + + got = [] + first.on("transcript", lambda event: got.append(("first", event))) + second.on("transcript", lambda event: got.append(("second", event))) + + identical = [fx.transcript_item(shape="push", text="能听见吗", sentence_id="s-1")] + for meeting_id in (fx.MEETING_ID_INT, fx.OTHER_MEETING_ID_INT): + fx.deliver( + channel, + fx.push_activity( + [fx.push_item("transcript_received", identical)], + meeting_id=meeting_id, + envelope_event_id="env-%s" % meeting_id, + ), + ) + + await fx.wait_for(lambda: len(got) >= 2, what="one transcript per meeting") + assert sorted(label for label, _ in got) == ["first", "second"] + + +async def test_message_layer_dedup_does_not_suppress_a_meeting_event( + vc, uat_channel +): + """The two dedup layers share an implementation but must not share a key + space: platform event ids are global, so one layer marking an id would + silently swallow the other layer's event.""" + channel, _store, _flow = uat_channel() + channel._ensure_bg_loop() + channel.safety.seen.add_sync("evt-collide") + + got = [] + body = fx.poll_events([fx.poll_item("transcript_received", event_id="evt-collide")]) + with fx.fast_sleep(): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("transcript", lambda event: got.append(event)) + vc.sequence(fx.URI_EVENTS, [body, fx.poll_events([])]) + await fx.wait_for(lambda: got, what="the transcript despite the marked id") + session.dispose() + + assert channel.safety.seen.has_sync("evt-collide") is True + assert channel.safety.seen.has_sync("evt-meeting-only") is False diff --git a/lark_channel/channel/meeting/tests/test_fixtures.py b/lark_channel/channel/meeting/tests/test_fixtures.py new file mode 100644 index 0000000..03f7fbe --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_fixtures.py @@ -0,0 +1,95 @@ +"""Locks the wire shapes the rest of this suite is built on. + +These are not tests of the SDK — they are a guard rail on ``fixtures.py``. +The reason they exist: a sibling port of this feature shipped 542 unit tests +and eight review rounds without noticing that echo suppression never fired, +because every fixture had been built from the generated model's type hints +(``actor.id: str``) instead of from the bytes the platform actually sends +(``actor.id`` is a nested object on the push transport). Fixtures shaped like +the type hints make a broken implementation pass. + +If one of these fails, fix the fixture back — do not adjust the expectation +to match whatever the implementation happens to want. +""" + +from . import fixtures as fx + + +def test_push_transport_carries_meeting_id_as_int(): + payload = fx.push_activity([fx.push_item("transcript_received")]) + meeting_id = payload["event"]["meeting"]["id"] + assert isinstance(meeting_id, int) + assert meeting_id == fx.MEETING_ID_INT + + +def test_poll_transport_carries_meeting_id_as_str(): + body = fx.poll_events([fx.poll_item("transcript_received")]) + meeting_id = body["data"]["events"][0]["meeting_id"] + assert isinstance(meeting_id, str) + assert meeting_id == fx.MEETING_ID_STR + + +def test_push_and_poll_meeting_ids_denote_the_same_meeting(): + assert str(fx.MEETING_ID_INT) == fx.MEETING_ID_STR + + +def test_push_actor_id_is_a_nested_object_with_three_namespaces(): + payload = fx.push_activity([fx.push_item("transcript_received")]) + item = payload["event"]["meeting_activity_items"][0]["transcript_received_items"][0] + identifier = item["speaker"]["id"] + assert isinstance(identifier, dict) + assert set(identifier) == {"open_id", "union_id", "user_id"} + assert identifier["open_id"].startswith("ou_") + + +def test_poll_actor_id_is_a_bare_open_id_string(): + body = fx.poll_events([fx.poll_item("transcript_received")]) + item = body["data"]["events"][0]["payload"]["transcript_received_items"][0] + identifier = item["speaker"]["id"] + assert isinstance(identifier, str) + assert identifier.startswith("ou_") + + +def test_push_activity_items_are_flat_and_carry_no_event_id(): + payload = fx.push_activity([fx.push_item("chat_received")]) + item = payload["event"]["meeting_activity_items"][0] + assert "chat_received_items" in item + assert "payload" not in item + assert "event_id" not in item + + +def test_poll_events_nest_items_under_payload_and_carry_an_event_id(): + body = fx.poll_events([fx.poll_item("chat_received")]) + event = body["data"]["events"][0] + assert "chat_received_items" not in event + assert "chat_received_items" in event["payload"] + assert "activity_event_type" in event["payload"] + assert event["event_id"] + + +def test_shared_document_lives_under_share_doc_not_doc(): + item = fx.share_started_item(shape="push") + assert "share_doc" in item + assert "doc" not in item + + +def test_item_level_timestamps_are_strings_on_both_transports(): + for shape in ("push", "poll"): + transcript = fx.transcript_item(shape=shape) + assert isinstance(transcript["start_time_ms"], str) + assert isinstance(fx.chat_item(shape=shape)["send_time"], str) + assert isinstance( + fx.participant_joined_item(shape=shape)["join_time"], str + ) + + +def test_document_context_items_never_carry_a_context_type_by_default(): + for kind in ("comment_focus", "section_location", "element_preview", "none"): + item = fx.document_context_item(shape="push", kind=kind) + assert "context_type" not in item + + +def test_connect_error_hides_the_token_from_repr_but_not_from_a_deep_walk(): + error = fx.httpx_connect_error("u-secret") + assert "u-secret" not in repr(error) + assert any("u-secret" in text for text in fx.deep_strings(error)) diff --git a/lark_channel/channel/meeting/tests/test_health.py b/lark_channel/channel/meeting/tests/test_health.py new file mode 100644 index 0000000..e6c1db9 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_health.py @@ -0,0 +1,292 @@ +"""Health readout and the liveness probe. + +Failures on this path are silent by nature: a missing permission, an +undeclared subscription, or a renamed field all look identical from the +outside — nothing happens. Splitting "arrived" from "arrived but unpacked to +nothing" is what separates "the platform never sent it" from "we can no +longer read what it sends". +""" + +import asyncio +import logging + +import httpx +import pytest + +from lark_channel.event.dispatcher_handler import EventDispatcherHandlerBuilder + +from . import fixtures as fx + + +def _probe_calls(vc): + return vc.for_uri(fx.URI_EVENTS) + + +async def test_a_quiet_channel_reports_zero_received_and_a_live_registration( + vc, tat_channel +): + channel = tat_channel() + health = channel.get_meeting_event_health() + assert health.received == 0 + assert health.last_at is None + assert health.registered is True + assert health.stats == {} + + +async def test_a_failed_internal_registration_is_reported_as_such( + vc, tat_channel, monkeypatch +): + channel = tat_channel() + real_register = EventDispatcherHandlerBuilder.register_p2_customized_event + + def _refuse(self, event_type, handler): + if event_type.startswith("vc.bot."): + raise RuntimeError("subscription unavailable") + return real_register(self, event_type, handler) + + monkeypatch.setattr( + EventDispatcherHandlerBuilder, "register_p2_customized_event", _refuse + ) + channel._dispatcher = channel._build_dispatcher() + + health = channel.get_meeting_event_health() + assert health.registered is False + assert health.reason + + +async def test_received_and_per_type_counters_move_with_traffic(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event)) + + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: got, what="the transcript") + + health = channel.get_meeting_event_health() + assert health.received >= 1 + assert health.last_at is not None + stats = health.stats["transcript_received"] + assert stats.received == 1 + assert stats.empty == 0 + + fx.deliver( + channel, + fx.push_activity( + [fx.push_item("transcript_received", [])], envelope_event_id="env-empty" + ), + ) + await fx.wait_for( + lambda: channel.get_meeting_event_health().stats["transcript_received"].empty + == 1, + what="the empty unpack to be counted", + ) + + +async def test_the_stat_key_space_is_bounded_by_count(vc, tat_channel): + """These keys are chosen by the server, are required to exist for types we + do not know, and are never reset for the life of the process.""" + channel = tat_channel() + await channel.join_meeting(fx.MEETING_NO) + + types = ["t_%04d" % index for index in range(5001)] + fx.deliver( + channel, fx.push_activity([fx.push_item(name) for name in types]) + ) + + await fx.wait_for( + lambda: len(channel.get_meeting_event_health().stats) >= 5000, + what="the key space to fill up", + timeout=20.0, + ) + await fx.settle() + + stats = channel.get_meeting_event_health().stats + assert sum(1 for key in stats if key.startswith("t_")) == 5000 + assert "__other__" in stats + + +@pytest.mark.parametrize( + "activity_type", + [ + "x" * 200, + "has\nnewline", + "has\x1b[31mansi", + "HasUpperCase", + "has spaces", + ], +) +async def test_a_malformed_activity_type_never_becomes_a_key( + vc, tat_channel, activity_type +): + """A bounded key count does not stop one 200-character key, and it does not + stop a key with a newline in it from reshaping a log line.""" + channel = tat_channel() + await channel.join_meeting(fx.MEETING_NO) + + fx.deliver(channel, fx.push_activity([fx.push_item(activity_type)])) + + await fx.wait_for( + lambda: channel.get_meeting_event_health().stats, + what="the activity to be accounted for", + ) + stats = channel.get_meeting_event_health().stats + assert activity_type not in stats + assert "__other__" in stats + + +async def test_probe_verdicts_and_the_unknown_streak_are_both_visible( + vc, tat_channel +): + """If the probe's permission assumption does not hold in some tenant it + returns "unknown" forever, and reclamation silently degrades to a feature + that is off by default. This counter is the only sign of that.""" + channel = tat_channel(liveness_probe_interval_seconds=0.02) + await channel.join_meeting(fx.MEETING_NO) + + await fx.wait_for( + lambda: channel.get_meeting_event_health().liveness.consecutive_unknown >= 2, + what="two inconclusive probes", + ) + liveness = channel.get_meeting_event_health().liveness + assert liveness.last_probe_at is not None + assert liveness.last_verdict == "unknown" + + vc.json(fx.URI_EVENTS, fx.poll_events([fx.poll_item("transcript_received")])) + await fx.wait_for( + lambda: channel.get_meeting_event_health().liveness.last_verdict == "in_meeting", + what="a conclusive probe", + ) + assert channel.get_meeting_event_health().liveness.consecutive_unknown == 0 + + +async def test_a_probe_proving_the_bot_left_ends_the_session_without_leaving( + vc, tat_channel +): + """Being removed by a host produces no meeting-ended event at all, and + calling depart for a meeting the bot is not in is a guaranteed failure.""" + channel = tat_channel(liveness_probe_interval_seconds=0.02) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json( + fx.URI_EVENTS, + fx.error_body(120004, "bot is not in the meeting"), + status=403, + ) + + await fx.wait_for(lambda: ended, what="the session to end") + assert [event.reason for event in ended] == ["no_longer_active"] + await fx.settle() + assert vc.count(fx.URI_LEAVE) == 0 + + +async def test_a_user_scoped_absence_code_does_not_end_the_session(vc, tat_channel): + """``120003`` is about a user, ``120004`` about the bot; both are 403. + Treating the user-scoped one as the bot's departure kills live sessions.""" + channel = tat_channel(liveness_probe_interval_seconds=0.02) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json( + fx.URI_EVENTS, + fx.error_body(120003, "user is not in the meeting"), + status=403, + ) + + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 3, what="several probes" + ) + assert ended == [] + + +@pytest.mark.parametrize("failure", ["network", "permission", "empty"]) +async def test_an_inconclusive_probe_keeps_the_session_alive( + vc, tat_channel, failure +): + """Probes run on the same schedule for every session, so their failures are + correlated: ending sessions on an inconclusive probe takes them all out in + one tick.""" + channel = tat_channel(liveness_probe_interval_seconds=0.02) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + if failure == "network": + request = httpx.Request("GET", "https://open.feishu.cn" + fx.URI_EVENTS) + vc.route( + fx.URI_EVENTS, + lambda call: (_ for _ in ()).throw( + httpx.ConnectError("unreachable", request=request) + ), + ) + elif failure == "permission": + vc.json(fx.URI_EVENTS, fx.error_body(99991672, "no permission"), status=403) + else: + vc.json(fx.URI_EVENTS, fx.poll_events([])) + + await fx.wait_for(lambda: vc.count(fx.URI_EVENTS) >= 3, what="several probes") + await fx.settle() + assert ended == [] + assert channel.get_meeting_event_health().liveness.last_verdict == "unknown" + + +async def test_the_probe_asks_for_a_page_the_endpoint_will_accept(vc, tat_channel): + """This endpoint rejects a page size below twenty at validation time, so a + probe asking for one item never gets an answer about anything.""" + channel = tat_channel(liveness_probe_interval_seconds=0.02) + await channel.join_meeting(fx.MEETING_NO) + + await fx.wait_for(lambda: _probe_calls(vc), what="the first probe") + page_size = _probe_calls(vc)[0].query("page_size") + assert page_size is not None + assert int(page_size) >= 20 + + +async def test_probe_backfill_is_delivered_and_resets_the_idle_clock( + vc, tat_channel, caplog +): + """One call does two jobs: it proves the bot is still a participant and it + picks up whatever the push transport missed.""" + channel = tat_channel( + liveness_probe_interval_seconds=0.02, idle_timeout_seconds=0.5 + ) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + ended = [] + session.on("transcript", lambda event: got.append(event)) + session.on("end", lambda event: ended.append(event)) + + # Every probe has to bring back something genuinely new, or dedup + # suppresses it and the idle clock is never touched. + served = {"n": 0} + + def _fresh_backfill(call): + served["n"] += 1 + index = served["n"] + return ( + 200, + fx.poll_events( + [ + fx.poll_item( + "transcript_received", + [ + fx.transcript_item( + shape="poll", + text="line-%d" % index, + sentence_id="s-%d" % index, + ) + ], + event_id="evt-backfill-%d" % index, + ) + ] + ), + ) + + vc.route(fx.URI_EVENTS, _fresh_backfill) + + with caplog.at_level(logging.WARNING, logger="Lark"): + await fx.wait_for(lambda: got, what="the backfilled transcript") + await asyncio.sleep(0.7) + + assert ended == [] diff --git a/lark_channel/channel/meeting/tests/test_identity.py b/lark_channel/channel/meeting/tests/test_identity.py new file mode 100644 index 0000000..0a1eb09 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_identity.py @@ -0,0 +1,178 @@ +"""Which credential each meeting request actually goes out with. + +Every assertion here is on the ``Authorization`` header the transport would +put on the wire, never on ``RequestOption.user_access_token``. That field can +be set correctly and then thrown away further down: a request declaring both +tenant and user token types resolves to a freshly minted *tenant* token and +has its declaration rewritten in place, so the user's authorization becomes +decoration. Only the header tells you whose identity the read happened under. +""" + +import inspect +import json + +from lark_channel.api.vc.bot import ( + build_bot_events_request_as_app, + build_bot_events_request_as_user, + build_bot_join_request, + build_bot_leave_request, + build_bot_message_request, + build_user_active_meeting_request, +) +from lark_channel.core.enum import AccessTokenType +from lark_channel.core.model import BaseRequest, RequestOption +from lark_channel.core.token.auth import verify + +from . import fixtures as fx + +_ALL_BUILDERS = { + "build_bot_join_request": (build_bot_join_request, {"meeting_no": fx.MEETING_NO}), + "build_bot_leave_request": ( + build_bot_leave_request, + {"meeting_id": fx.MEETING_ID_STR}, + ), + "build_bot_message_request": ( + build_bot_message_request, + { + "meeting_id": fx.MEETING_ID_STR, + "msg_type": "text", + "content": '{"text":"hi"}', + "uuid": "uuid-1", + }, + ), + "build_bot_events_request_as_user": ( + build_bot_events_request_as_user, + {"meeting_id": fx.MEETING_ID_STR}, + ), + "build_bot_events_request_as_app": ( + build_bot_events_request_as_app, + {"meeting_id": fx.MEETING_ID_STR}, + ), + "build_user_active_meeting_request": (build_user_active_meeting_request, {}), +} + + +async def test_meeting_event_polling_goes_out_as_the_user(vc, uat_channel): + channel, _store, _flow = uat_channel() + with fx.fast_sleep(max_sleeps=3): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 1, what="the first event poll" + ) + session.dispose() + + call = vc.last(fx.URI_EVENTS) + assert call.authorization == "Bearer u-REAL" + + +async def test_active_meeting_lookup_goes_out_as_the_user(vc, uat_channel): + channel, _store, _flow = uat_channel() + with fx.fast_sleep(max_sleeps=3): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_ACTIVE_MEETING) >= 1, + what="the active-meeting lookup", + ) + session.dispose() + + assert vc.last(fx.URI_ACTIVE_MEETING).authorization == "Bearer u-REAL" + + +async def test_join_leave_and_in_meeting_message_go_out_as_the_app(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + await session.send_message("hello meeting") + await session.leave() + + expected = "Bearer %s" % fx.FakeVC.tenant_token + for uri in (fx.URI_JOIN, fx.URI_MESSAGE, fx.URI_LEAVE): + assert vc.last(uri).authorization == expected, uri + + +def test_every_vc_builder_declares_exactly_one_token_type(): + """Two declared token types make the transport overwrite the header once + per type, and the winner depends on set iteration order — which varies + with the process hash seed. Same code, different identity per run.""" + for name, (builder, kwargs) in _ALL_BUILDERS.items(): + request = builder(**kwargs) + assert len(request.token_types) == 1, "%s declares %r" % ( + name, + request.token_types, + ) + + assert build_bot_events_request_as_user(meeting_id=fx.MEETING_ID_STR).token_types == { + AccessTokenType.USER + } + assert build_user_active_meeting_request().token_types == {AccessTokenType.USER} + assert build_bot_events_request_as_app(meeting_id=fx.MEETING_ID_STR).token_types == { + AccessTokenType.TENANT + } + for name in ("build_bot_join_request", "build_bot_leave_request", "build_bot_message_request"): + builder, kwargs = _ALL_BUILDERS[name] + assert builder(**kwargs).token_types == {AccessTokenType.TENANT}, name + + +def test_channel_client_allows_manually_supplied_tokens(vc, make_ch): + """Without this switch the transport layer skips manual tokens entirely + and a user-scoped request either falls back to the app identity or fails + with "need enable set token".""" + channel = make_ch() + assert channel.client.config.enable_set_token is True + + +def test_tenant_requests_still_mint_from_app_credentials(vc, make_ch): + """Regression: flipping the manual-token switch on must not change how a + request that carries no manual token gets its credential.""" + channel = make_ch() + request = BaseRequest() + request.token_types = {AccessTokenType.TENANT} + option = RequestOption() + + verify(channel.client.config, request, option) + + assert option.tenant_access_token == fx.FakeVC.tenant_token + assert option.user_access_token is None + assert request.token_types == {AccessTokenType.TENANT} + + +async def test_each_polling_round_builds_a_fresh_request_and_option(vc, uat_channel): + """The transport writes ``Authorization`` back onto the request object + itself, so a reused request carries the previous round's credential into + the next one.""" + channel, store, _flow = uat_channel() + store.rotate( + fx.USER_OPEN_ID, + [ + fx.make_uat("u-CREATE"), + fx.make_uat("u-ROUND1"), + fx.make_uat("u-ROUND2"), + fx.make_uat("u-ROUND3"), + ], + ) + with fx.fast_sleep(max_sleeps=6): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 2, what="two polling rounds" + ) + session.dispose() + + calls = vc.for_uri(fx.URI_EVENTS) + first, second = calls[0], calls[1] + assert first.request is not second.request + assert first.option is not second.option + assert first.authorization != second.authorization + first_token = first.authorization.split(None, 1)[1] + assert first_token not in json.dumps(dict(second.request.headers)) + + +def test_user_id_type_is_pinned_by_the_builders_not_exposed_as_a_parameter(): + """The actor ids in the event stream have to live in the same namespace as + the bot's own open_id, or echo detection compares two unrelated random + strings and never matches.""" + for name in ("build_bot_events_request_as_user", "build_bot_events_request_as_app", + "build_user_active_meeting_request"): + builder, kwargs = _ALL_BUILDERS[name] + params = inspect.signature(builder).parameters + assert "user_id_type" not in params, name + request = builder(**kwargs) + assert ("user_id_type", "open_id") in request.queries, name diff --git a/lark_channel/channel/meeting/tests/test_meeting_public_api.py b/lark_channel/channel/meeting/tests/test_meeting_public_api.py new file mode 100644 index 0000000..9b8e883 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_meeting_public_api.py @@ -0,0 +1,200 @@ +"""What ships, what is exported, and what the source is allowed to use. + +The package deliberately keeps a trimmed copy of the API layer with a +file-by-file closure test, so adding a service is a packaging decision that +has to be made explicitly rather than by accident. +""" + +import ast +import dataclasses +from pathlib import Path + +import pytest + +import lark_channel +import lark_channel.api as api_root +from lark_channel import channel as channel_pkg +from lark_channel.channel.auth import uat_runner +from lark_channel.channel.config import ChannelConfig, MeetingChannelConfig + +ROOT = Path(lark_channel.__file__).resolve().parents[1] +NEW_SOURCE_DIRS = ("lark_channel/api/vc", "lark_channel/channel/meeting") +#: New files that live outside those directories and would otherwise escape the +#: syntax guard below. +NEW_SOURCE_FILES = ("lark_channel/channel/raw_events.py",) + +MEETING_PUBLIC_NAMES = [ + "ActivityTypeStats", + "DocumentContextEvent", + "LivenessHealth", + "MeetingActor", + "MeetingChannelConfig", + "MeetingChatEvent", + "MeetingEndEvent", + "MeetingEventHealth", + "MeetingInvitedEvent", + "MeetingOptions", + "MembershipHealth", + "MeetingSession", + "ParticipantEvent", + "ShareEvent", + "TranscriptEvent", +] + + +def _allowlisted_api_files(): + path = ROOT / "tests/runtime/api_file_allowlist.txt" + return set( + line.strip() + for line in path.read_text(encoding="utf-8").splitlines() + if line.strip() + ) + + +def _new_source_files(): + files = [] + for directory in NEW_SOURCE_DIRS: + base = ROOT / directory + if not base.exists(): + continue + for path in base.rglob("*.py"): + if "/tests/" in path.as_posix(): + continue + files.append(path) + for name in NEW_SOURCE_FILES: + path = ROOT / name + if path.exists(): + files.append(path) + return files + + +def _declared_api_roots(): + """Read the packaging closure's root set without importing its module — + the runtime test directory is not an importable package.""" + source = (ROOT / "tests/runtime/test_api_allowlist.py").read_text(encoding="utf-8") + tree = ast.parse(source) + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + targets = [t.id for t in node.targets if isinstance(t, ast.Name)] + if "ALLOWED_API_ROOTS" in targets: + return set(ast.literal_eval(node.value)) + raise AssertionError("ALLOWED_API_ROOTS not found") + + +def test_the_video_conference_service_is_now_part_of_the_packaged_api(): + assert "vc" in _declared_api_roots() + assert (Path(api_root.__file__).resolve().parent / "vc").exists() + + +def test_the_other_unpackaged_services_stay_unpackaged(): + api_path = Path(api_root.__file__).resolve().parent + for name in ("calendar", "bitable", "drive_full", "docx", "admin"): + assert not (api_path / name).exists(), name + + +def test_the_new_api_files_are_in_the_packaging_closure(): + allowlisted = _allowlisted_api_files() + vc_files = set( + path.relative_to(ROOT).as_posix() + for path in (ROOT / "lark_channel/api/vc").rglob("*.py") + ) + assert vc_files + assert vc_files <= allowlisted + + +def test_the_video_conference_package_stays_a_thin_builder_layer(): + """Copying a generated model tree in would multiply the packaged surface + and add a second source of truth for field shapes.""" + files = set( + path.relative_to(ROOT / "lark_channel/api/vc").as_posix() + for path in (ROOT / "lark_channel/api/vc").rglob("*.py") + ) + assert files == {"__init__.py", "bot.py"} + + +def test_the_dependency_list_is_untouched(): + source = (ROOT / "setup.py").read_text(encoding="utf-8") + tree = ast.parse(source) + requires = None + for node in ast.walk(tree): + if isinstance(node, ast.keyword) and node.arg == "install_requires": + requires = [ + element.value + for element in node.value.elts + if isinstance(element, ast.Constant) + ] + assert requires == [ + "requests>=2.25", + "requests_toolbelt>=0.9", + "pycryptodome>=3.9", + "websockets>=11,<16", + "httpx>=0.24,<1.0", + ] + + +def test_the_non_interactive_ticket_helper_is_not_part_of_the_public_surface(): + """It returns a bare token; exporting it invites callers to route around + the store's lifecycle management.""" + assert hasattr(uat_runner, "resolve_user_auth_non_interactive") + assert "resolve_user_auth_non_interactive" not in getattr(uat_runner, "__all__", []) + for module in (lark_channel, channel_pkg): + assert "resolve_user_auth_non_interactive" not in getattr(module, "__all__", []) + + +@pytest.mark.parametrize("name", MEETING_PUBLIC_NAMES) +def test_meeting_types_are_exported_from_both_public_entry_points(name): + assert name in channel_pkg.__all__, name + assert hasattr(channel_pkg, name), name + assert hasattr(lark_channel, name), name + + +def test_the_meeting_config_field_is_appended_at_the_end(): + """Field order on this dataclass is part of the public contract for + positional callers.""" + names = [field.name for field in dataclasses.fields(ChannelConfig)] + assert names[-1] == "meeting" + assert isinstance(ChannelConfig().meeting, MeetingChannelConfig) + + +def test_the_new_sources_stay_within_the_oldest_supported_python(): + """The support matrix starts two releases before union syntax in + annotations, slot-enabled dataclasses, and the newer asyncio helpers.""" + banned_attributes = {"to_thread", "timeout", "TaskGroup"} + sources = _new_source_files() + # Without this the check would quietly pass on an empty file list. + assert sources + offences = [] + for path in sources: + tree = ast.parse(path.read_text(encoding="utf-8")) + for node in ast.walk(tree): + annotations = [] + if isinstance(node, ast.AnnAssign): + annotations.append(node.annotation) + elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + annotations.append(node.returns) + for arg in list(node.args.args) + list(node.args.kwonlyargs): + annotations.append(arg.annotation) + for annotation in annotations: + if annotation is None: + continue + for inner in ast.walk(annotation): + if isinstance(inner, ast.BinOp) and isinstance(inner.op, ast.BitOr): + offences.append("%s: union operator in annotation" % path.name) + if isinstance(node, ast.Attribute) and node.attr in banned_attributes: + if isinstance(node.value, ast.Name) and node.value.id == "asyncio": + offences.append("%s: asyncio.%s" % (path.name, node.attr)) + if isinstance(node, ast.keyword) and node.arg == "slots": + offences.append("%s: dataclass slots" % path.name) + assert offences == [] + + +def test_the_follow_entry_point_states_its_trust_boundary_up_front(): + """The SDK cannot tell whether the supplied open_id belongs to the caller, + and a cached ticket resolves without notifying its owner. Anybody reading + the signature has to meet that before they meet the parameters.""" + from lark_channel.channel import FeishuChannel + + doc = FeishuChannel.follow_my_meeting.__doc__ or "" + first_paragraph = doc.strip().split("\n\n")[0] + assert "user_open_id" in first_paragraph + assert "prompt_context" in first_paragraph diff --git a/lark_channel/channel/meeting/tests/test_membership.py b/lark_channel/channel/meeting/tests/test_membership.py new file mode 100644 index 0000000..8818a5c --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_membership.py @@ -0,0 +1,420 @@ +"""Server-side participation accounting — the thing the concurrency gate reads. + +The gate cannot count live sessions: disposal stops local work but leaves the +bot a participant, and a failed departure removes the session while the seat +is still taken. So the seat is released on *evidence that the bot is no longer +a participant*, which is a different thing from "the leave call returned 200". +Both directions have to hold: never releasing a seat on a normal meeting end +bricks both entry points for the life of the process, and always releasing one +turns the gate off. +""" + +import asyncio +import logging + +import httpx +import pytest + +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode + +from . import fixtures as fx + + +def _sequential_joins(vc, start=8000001): + """Give every join a distinct long meeting id, and report them back.""" + minted = [] + + def responder(call): + meeting_id = str(start + len(minted)) + minted.append(meeting_id) + return (200, fx.join_body(meeting_id)) + + vc.route(fx.URI_JOIN, responder) + return minted + + +async def test_a_normally_ended_meeting_gives_its_seat_back(vc, tat_channel): + """Ending is exactly when a departure call is most likely to 404, so a seat + that only comes back on a clean departure leaks once per normal meeting.""" + channel = tat_channel(max_concurrent_sessions=1) + minted = _sequential_joins(vc) + + for _ in range(3): + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + fx.deliver(channel, fx.push_meeting_ended(meeting_id=int(minted[-1]))) + await fx.wait_for(lambda: ended, what="the meeting-ended signal") + await fx.wait_for( + lambda: channel.get_meeting_event_health().membership.held == 0, + what="the gate reading to fall back", + ) + + assert len(minted) == 3 + + +@pytest.mark.parametrize( + "outcome,evidence", + [ + ((404, fx.error_body(404, "not found")), "404"), + ((400, fx.error_body(121105, "meeting not exist")), "121105"), + ((403, fx.error_body(120004, "bot is not in the meeting")), "120004"), + ], +) +async def test_departure_errors_that_prove_absence_release_the_seat( + vc, tat_channel, outcome, evidence +): + channel = tat_channel(max_concurrent_sessions=1) + _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: outcome) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + + membership = channel.get_meeting_event_health().membership + assert membership.held == 0 + assert membership.released_by_evidence.get(evidence) == 1 + + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert second is not None + + +@pytest.mark.parametrize( + "failure", + [ + "server_error", + "timeout", + ], +) +async def test_departure_failures_of_unknown_outcome_keep_the_seat( + vc, tat_channel, failure +): + channel = tat_channel( + max_concurrent_sessions=1, membership_reconcile_interval_seconds=0.0 + ) + _sequential_joins(vc) + if failure == "server_error": + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + else: + request = httpx.Request("POST", "https://open.feishu.cn" + fx.URI_LEAVE) + vc.route( + fx.URI_LEAVE, + lambda call: (_ for _ in ()).throw( + httpx.TimeoutException("timed out", request=request) + ), + ) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + + membership = channel.get_meeting_event_health().membership + assert membership.held == 1 + assert membership.retained_without_session == 1 + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.OTHER_MEETING_NO) + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + + +async def test_meeting_ended_releases_a_seat_whose_session_is_already_gone( + vc, tat_channel +): + """Three rules stack into a dead end: a failed departure keeps the seat, + still removes the session, and routing drops events for meetings it has no + session for. Accounting has to keep listening after delivery stops.""" + channel = tat_channel( + max_concurrent_sessions=1, membership_reconcile_interval_seconds=0.0 + ) + minted = _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + with pytest.raises(FeishuChannelError): + await channel.join_meeting(fx.OTHER_MEETING_NO) + + fx.deliver(channel, fx.push_meeting_ended(meeting_id=int(minted[0]))) + await fx.settle() + + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert second is not None + + +async def test_a_stale_seat_is_reclaimed_by_the_next_admission(vc, uat_channel): + """Nothing periodic reclaims this seat, and nothing should: a loop that + outlived ``disconnect()`` would have to sit on its own thread, and a thread + still running after the call whose whole job is releasing resources is a + dangling resource. So reclamation hangs off admission instead — the moment + a seat is actually wanted. After a disconnect, joining is unavailable + anyway; following is the entry point a stale seat can really block, and + following goes through admission.""" + channel, _store, _flow = uat_channel( + max_concurrent_sessions=1, + membership_reconcile_interval_seconds=60.0, + membership_max_age_seconds=3600.0, + active_meeting_check_interval_seconds=300.0, + ) + fx.mark_connected(channel) + _sequential_joins(vc) + vc.sequence( + fx.URI_LEAVE, + [(500, fx.error_body(500, "boom")), (200, {"code": 0, "msg": "success"})], + ) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + assert channel.get_meeting_event_health().membership.retained_without_session == 1 + before = vc.count(fx.URI_LEAVE) + vc.count(fx.URI_EVENTS) + + await channel.disconnect() + followed = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + + assert followed.mode == "uat" + membership = channel.get_meeting_event_health().membership + assert membership.reconcile_attempts >= 1 + assert vc.count(fx.URI_LEAVE) + vc.count(fx.URI_EVENTS) > before + assert membership.retained_without_session == 0 + followed.dispose() + + +async def test_reconciliation_is_throttled_per_entry(vc, uat_channel): + """Admission is on the latency path of two public calls, so a stale entry + must not be retried once per attempt.""" + channel, _store, _flow = uat_channel( + max_concurrent_sessions=1, + membership_reconcile_interval_seconds=60.0, + membership_max_age_seconds=3600.0, + active_meeting_check_interval_seconds=300.0, + ) + fx.mark_connected(channel) + _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + before = vc.count(fx.URI_LEAVE) + vc.count(fx.URI_EVENTS) + + for _ in range(2): + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + + assert channel.get_meeting_event_health().membership.reconcile_attempts == 1 + assert vc.count(fx.URI_LEAVE) + vc.count(fx.URI_EVENTS) == before + 1 + + +async def test_releasing_on_meeting_not_exist_is_reported_not_just_absorbed( + vc, tat_channel, caplog +): + """That code also fires when we have been sending the wrong meeting id all + along. Absorbing it silently turns the gate off and keeps the suite green.""" + channel = tat_channel(max_concurrent_sessions=1) + _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (400, fx.error_body(121105, "meeting not exist"))) + + session = await channel.join_meeting(fx.MEETING_NO) + with caplog.at_level(logging.WARNING, logger="Lark"): + await session.leave() + await fx.settle() + + assert any( + record.levelno >= logging.WARNING and "121105" in record.getMessage() + for record in caplog.records + ), caplog.text + membership = channel.get_meeting_event_health().membership + assert membership.released_by_evidence["121105"] == 1 + + +async def test_a_seat_past_its_deadline_is_force_released_with_a_warning( + vc, tat_channel, caplog +): + channel = tat_channel( + max_concurrent_sessions=1, + membership_reconcile_interval_seconds=0.0, + membership_max_age_seconds=0.1, + ) + _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + session = await channel.join_meeting(fx.MEETING_NO) + with caplog.at_level(logging.WARNING, logger="Lark"): + await session.leave() + await asyncio.sleep(0.25) + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + + assert second is not None + assert any(record.levelno >= logging.WARNING for record in caplog.records) + assert ( + channel.get_meeting_event_health().membership.released_by_evidence["ttl"] == 1 + ) + + +async def test_repeated_failed_departures_never_brick_either_entry_point( + vc, uat_channel +): + """One tenant member looping "invite the bot, end the meeting" is enough to + exhaust the gate if a 5xx departure strands the seat. It takes both entry + points down together, because they share the gate.""" + channel, _store, _flow = uat_channel( + max_concurrent_sessions=2, + membership_reconcile_interval_seconds=0.0, + active_meeting_check_interval_seconds=300.0, + ) + fx.mark_connected(channel) + minted = _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + for _ in range(3): + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + fx.deliver(channel, fx.push_meeting_ended(meeting_id=int(minted[-1]))) + await fx.settle() + + survivor = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert survivor is not None + followed = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + assert followed.mode == "uat" + + +async def test_a_probe_proving_absence_hands_the_seat_back(vc, tat_channel): + """The one reclamation path with no meeting-ended event behind it. Nothing + else ever produces evidence for it — lazy reconciliation only looks at + entries whose *departure* was inconclusive — so a seat left here waits out + the accounting deadline, and repeating the removal exhausts the ceiling.""" + channel = tat_channel( + max_concurrent_sessions=1, liveness_probe_interval_seconds=0.02 + ) + _sequential_joins(vc) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json( + fx.URI_EVENTS, + fx.error_body(120004, "bot is not in the meeting"), + status=403, + ) + + await fx.wait_for(lambda: ended, what="the session to end") + await fx.wait_for( + lambda: channel.get_meeting_event_health().membership.held == 0, + what="the seat to come back", + ) + + membership = channel.get_meeting_event_health().membership + assert membership.released_by_evidence.get("120004") == 1 + # No departure call: the bot is already out, and the endpoint rejects a + # departure for a meeting it is not in. + assert vc.count(fx.URI_LEAVE) == 0 + # And the ceiling really is free again. + assert await channel.join_meeting(fx.OTHER_MEETING_NO) is not None + + +async def test_a_live_session_is_never_expired_by_the_accounting_deadline( + vc, tat_channel +): + """Two hours of a meeting where nobody speaks produces no activity and no + conclusive probe. The deadline is a backstop for mis-accounting, and a seat + with a live session is not mis-accounted.""" + channel = tat_channel( + max_concurrent_sessions=1, + membership_max_age_seconds=0.05, + liveness_probe_interval_seconds=0.0, + ) + _sequential_joins(vc) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + await asyncio.sleep(0.2) + # Reading health is what runs the expiry sweep. + membership = channel.get_meeting_event_health().membership + assert membership.held == 1 + assert "ttl" not in membership.released_by_evidence + assert ended == [] + + +async def test_rejoining_a_meeting_resets_its_accounting_entry(vc, tat_channel): + """Carrying the old flags over would count a meeting with a live session as + one whose departure is unresolved, and send reconciliation after it every + interval — each attempt refused, each one logging the opposite of the + truth.""" + channel = tat_channel(membership_reconcile_interval_seconds=0.0) + vc.route(fx.URI_JOIN, lambda call: (200, fx.join_body(fx.MEETING_ID_STR))) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + first = await channel.join_meeting(fx.MEETING_NO) + await first.leave() + await fx.settle() + assert channel.get_meeting_event_health().membership.retained_without_session == 1 + + await channel.join_meeting(fx.MEETING_NO) + await fx.settle() + + membership = channel.get_meeting_event_health().membership + assert membership.held == 1 + assert membership.retained_without_session == 0 + + +async def test_two_joins_of_one_meeting_leave_the_live_seat_alone(vc, tat_channel): + """The supersede path accounts for the new session, then the old one's + teardown runs, then the new one is registered. A stored "has a live + session" flag ends up ``False`` for a meeting that is very much live, and + the deadline then releases its seat.""" + channel = tat_channel( + membership_max_age_seconds=0.05, + liveness_probe_interval_seconds=0.0, + membership_reconcile_interval_seconds=0.0, + ) + vc.route(fx.URI_JOIN, lambda call: (200, fx.join_body(fx.MEETING_ID_STR))) + + first = await channel.join_meeting(fx.MEETING_NO) + second = await channel.join_meeting(fx.MEETING_NO) + # The two calls are sequential, so the in-flight table is already clear and + # the second really is a new session superseding the first. Asserted rather + # than assumed: if `join_meeting` ever starts handing back an existing + # session for an already-joined meeting, this guard would stop exercising + # the supersede path while staying green. + assert second is not first + await asyncio.sleep(0.2) + + membership = channel.get_meeting_event_health().membership + assert membership.held == 1 + assert "ttl" not in membership.released_by_evidence + + +async def test_becoming_ready_reconciles_stranded_seats(vc, tat_channel): + """The third reconciliation point. It was documented before it existed, so + it gets an assertion of its own.""" + channel = tat_channel( + membership_reconcile_interval_seconds=0.0, membership_max_age_seconds=3600.0 + ) + _sequential_joins(vc) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "boom"))) + + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle() + assert channel.get_meeting_event_health().membership.retained_without_session == 1 + + attempts_before = channel.get_meeting_event_health().membership.reconcile_attempts + channel._meeting._membership._interval = 60.0 + vc.route(fx.URI_LEAVE, lambda call: (200, {"code": 0, "msg": "success"})) + + # What a reconnect does. + fx.mark_connected(channel) + + await fx.wait_for( + lambda: channel.get_meeting_event_health().membership.reconcile_attempts + > attempts_before, + what="reconciliation on becoming ready", + ) + await fx.wait_for( + lambda: channel.get_meeting_event_health().membership.retained_without_session + == 0, + what="the stranded seat to come back", + ) diff --git a/lark_channel/channel/meeting/tests/test_normalize.py b/lark_channel/channel/meeting/tests/test_normalize.py new file mode 100644 index 0000000..e4649a4 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_normalize.py @@ -0,0 +1,297 @@ +"""Unpacking one wire event into a stream of session events. + +Every case here drives a real session — the push cases through the channel's +dispatcher, the poll cases through the polling transport — because the two +transports nest the same data differently and only an end-to-end path proves +both nestings are read. +""" + +import asyncio +from dataclasses import asdict + +from . import fixtures as fx + + +async def _tat_session(channel): + return await channel.join_meeting(fx.MEETING_NO) + + +async def _poll_delivery(vc, uat_channel, body, event_name, *, expected=1, **meeting_kw): + """Start a follow session, then make the poll transport serve ``body``. + + The response is swapped in *after* the handler is registered so the very + first poll round cannot deliver before anybody is listening. + """ + channel, _store, _flow = uat_channel(**meeting_kw) + got = [] + with fx.fast_sleep(): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on(event_name, lambda event: got.append(event)) + vc.sequence(fx.URI_EVENTS, [body, fx.poll_events([])]) + await fx.wait_for( + lambda: len(got) >= expected, what="%s events from polling" % expected + ) + session.dispose() + return session, got + + +async def test_every_item_of_every_activity_is_delivered_in_array_order( + vc, tat_channel +): + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("transcript", lambda event: got.append(event)) + + activities = [] + for outer in range(2): + items = [ + fx.transcript_item( + shape="push", + text="line-%d-%d" % (outer, inner), + sentence_id="sent-%d-%d" % (outer, inner), + ) + for inner in range(3) + ] + activities.append(fx.push_item("transcript_received", items)) + fx.deliver(channel, fx.push_activity(activities)) + + await fx.wait_for(lambda: len(got) >= 6, what="six transcripts") + assert [event.text for event in got] == [ + "line-0-0", + "line-0-1", + "line-0-2", + "line-1-0", + "line-1-1", + "line-1-2", + ] + + +async def test_push_and_poll_shapes_produce_identical_session_events( + vc, tat_channel, uat_channel +): + """The same logical transcript, sent over both transports, must come out + the other side field-for-field identical.""" + channel = tat_channel() + session = await _tat_session(channel) + pushed = [] + session.on("transcript", lambda event: pushed.append(event)) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: pushed, what="the pushed transcript") + + _polled_session, polled = await _poll_delivery( + vc, uat_channel, fx.poll_events([fx.poll_item("transcript_received")]), "transcript" + ) + + assert asdict(pushed[0]) == asdict(polled[0]) + + +async def test_share_ended_reaches_the_handler_before_share_started(vc, tat_channel): + """Order is the whole meaning of these two: a document swap arrives as + ended-then-started, and reordering makes the business reconstruct the + wrong shared document.""" + channel = tat_channel() + session = await _tat_session(channel) + order = [] + + async def slow_handler(event): + if event.action == "ended": + await asyncio.sleep(0.01) + order.append(event.action) + + session.on("share", slow_handler) + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item("magic_share_ended"), + fx.push_item("magic_share_started"), + ] + ), + ) + + await fx.wait_for(lambda: len(order) >= 2, what="both share events") + assert order == ["ended", "started"] + + +async def test_each_activity_type_maps_to_its_event_and_its_originator_field( + vc, tat_channel +): + channel = tat_channel() + session = await _tat_session(channel) + names = ("transcript", "chat", "participant", "share", "document_context") + got = dict((name, []) for name in names) + for name in names: + session.on(name, lambda event, name=name: got[name].append(event)) + + fx.deliver( + channel, + fx.push_activity([fx.push_item(t) for t in fx.ALL_ACTIVITY_TYPES]), + ) + await fx.wait_for( + lambda: sum(len(v) for v in got.values()) >= 7, what="all seven activities" + ) + + assert got["transcript"][0].actor.id == "ou_speaker" + assert got["chat"][0].actor.id == "ou_chatter" + assert [e.action for e in got["participant"]] == ["joined", "left"] + assert [e.actor.id for e in got["participant"]] == ["ou_joiner", "ou_leaver"] + assert [e.action for e in got["share"]] == ["started", "ended"] + assert all(e.actor.id == "ou_sharer" for e in got["share"]) + assert got["document_context"][0].actor.id == "ou_editor" + + +async def test_unknown_activity_type_is_counted_as_empty_without_raising( + vc, tat_channel +): + channel = tat_channel() + session = await _tat_session(channel) + fx.deliver(channel, fx.push_activity([fx.push_item("brand_new_type")])) + + await fx.wait_for( + lambda: "brand_new_type" in session.get_stats(), + what="the unknown type to be accounted for", + ) + stats = session.get_stats()["brand_new_type"] + assert stats.received == 1 + assert stats.empty == 1 + + +async def test_document_context_with_no_known_sub_object_is_skipped_not_counted_empty( + vc, tat_channel +): + """A fourth kind of document context is forward compatibility, not a + parse failure — counting it as empty would make the health readout claim + a field-shape regression that never happened.""" + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("document_context", lambda event: got.append(event)) + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item( + "document_context_changed", + [fx.document_context_item(shape="push", kind="none")], + ) + ] + ), + ) + + await fx.wait_for( + lambda: "document_context_changed" in session.get_stats(), + what="the document context item to be accounted for", + ) + await fx.settle() + stats = session.get_stats()["document_context_changed"] + assert stats.received == 1 + assert stats.empty == 0 + assert got == [] + + +async def test_document_context_type_is_derived_from_the_present_sub_object( + vc, tat_channel +): + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("document_context", lambda event: got.append(event)) + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item( + "document_context_changed", + [fx.document_context_item(shape="push", kind=kind)], + ) + for kind in ("comment_focus", "section_location", "element_preview") + ] + ), + ) + + await fx.wait_for(lambda: len(got) >= 3, what="three document context events") + assert [event.context_type for event in got] == [ + "comment_focus", + "section_location", + "element_preview", + ] + + +async def test_explicit_context_type_wins_over_the_sub_object_it_disagrees_with( + vc, tat_channel +): + """The generated model has no ``context_type`` field at all, so a payload + that carries one is the platform having moved ahead of it — believe the + payload.""" + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("document_context", lambda event: got.append(event)) + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item( + "document_context_changed", + [ + fx.document_context_item( + shape="push", + kind="comment_focus", + context_type="section_location", + ) + ], + ) + ] + ), + ) + + await fx.wait_for(lambda: got, what="the document context event") + assert got[0].context_type == "section_location" + + +async def test_string_timestamps_become_ints_and_unparsable_ones_become_none( + vc, tat_channel +): + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("transcript", lambda event: got.append(event)) + fx.deliver( + channel, + fx.push_activity( + [ + fx.push_item( + "transcript_received", + [ + fx.transcript_item( + shape="push", + sentence_id="parsable", + start_time_ms="1730000000123", + ), + fx.transcript_item( + shape="push", sentence_id="garbage", start_time_ms="abc" + ), + fx.transcript_item( + shape="push", sentence_id="absent", start_time_ms=None + ), + ], + ) + ] + ), + ) + + await fx.wait_for(lambda: len(got) >= 3, what="three transcripts") + assert [event.start_ms for event in got] == [1730000000123, None, None] + + +async def test_shared_document_is_read_from_share_doc(vc, tat_channel): + channel = tat_channel() + session = await _tat_session(channel) + got = [] + session.on("share", lambda event: got.append(event)) + fx.deliver(channel, fx.push_activity([fx.push_item("magic_share_started")])) + + await fx.wait_for(lambda: got, what="the share event") + assert got[0].doc.url == "https://example.test/docx/doc_1" + assert got[0].doc.title == "Design doc" diff --git a/lark_channel/channel/meeting/tests/test_poll_source.py b/lark_channel/channel/meeting/tests/test_poll_source.py new file mode 100644 index 0000000..cbd4d32 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_poll_source.py @@ -0,0 +1,493 @@ +"""The follow-mode polling loop and the non-interactive ticket lookup it uses. + +The loop runs every few seconds for the whole meeting, so anything it does +per round it does hundreds of times: an interactive authorization in there is +hundreds of authorization cards, and a refresh that is not written back +invalidates the ticket for everybody else holding it. +""" + +import asyncio +import logging + +import pytest + +from lark_channel.channel.auth import uat_runner +from lark_channel.channel.config import UATConfig +from lark_channel.channel.errors import ( + FeishuChannelError, + FeishuChannelErrorCode, + UATAuthError, +) + +from . import fixtures as fx + +_CREDENTIAL_FAILURES = [ + (401, fx.error_body(0, "unauthorized")), + (403, fx.error_body(0, "forbidden")), + (200, fx.error_body(99991400, "invalid app_ticket")), + (200, fx.error_body(99991401, "invalid access token")), + (200, fx.error_body(99991668, "token expired")), +] + + +def _poll_sleeps(clock): + """The empty-poll ladder, separated from the slower end-detection loop.""" + return [d for d in clock.durations if 0 < d <= 10] + + +# --------------------------------------------------------------------------- +# Backoff +# --------------------------------------------------------------------------- + + +async def test_empty_polls_back_off_from_three_seconds_to_a_ten_second_ceiling( + vc, uat_channel +): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + with fx.fast_sleep(max_sleeps=8) as clock: + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: len(_poll_sleeps(clock)) >= 3, what="three backoff steps" + ) + session.dispose() + + assert _poll_sleeps(clock)[:3] == [3.0, 6.0, 10.0] + + +async def test_receiving_events_returns_the_backoff_to_its_floor(vc, uat_channel): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + vc.sequence( + fx.URI_EVENTS, + [ + fx.poll_events([]), + fx.poll_events([]), + fx.poll_events([fx.poll_item("transcript_received")]), + fx.poll_events([]), + ], + ) + with fx.fast_sleep(max_sleeps=8) as clock: + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 4, what="four polling rounds" + ) + session.dispose() + + sleeps = _poll_sleeps(clock) + assert sleeps[:2] == [3.0, 6.0] + assert 3.0 in sleeps[2:] + + +# --------------------------------------------------------------------------- +# Ticket handling inside the loop +# --------------------------------------------------------------------------- + + +async def test_every_round_reads_the_ticket_store_without_going_interactive( + vc, uat_channel, monkeypatch +): + channel, store, flow = uat_channel(active_meeting_check_interval_seconds=300.0) + with fx.fast_sleep(max_sleeps=10): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + reads_at_start = len(store.get_calls) + + interactive = [] + + async def _forbidden(**kwargs): + interactive.append(kwargs) + raise AssertionError("the polling loop must not go interactive") + + monkeypatch.setattr( + "lark_channel.channel.channel.require_user_auth", _forbidden + ) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 4, what="four polling rounds" + ) + session.dispose() + + assert len(store.get_calls) > reads_at_start + assert interactive == [] + assert flow.start_calls == [] + + +async def test_a_ticket_missing_the_meeting_scope_never_triggers_an_auth_card( + vc, uat_channel +): + """Ticket scopes are whatever the platform granted the app, so the request + scope failing to appear verbatim is an ordinary state, not an error. The + interactive helper answers that state by starting a device flow.""" + channel, store, flow = uat_channel(active_meeting_check_interval_seconds=300.0) + # A generous sleep budget: collapsing sleep lets the loop spin through its + # allowance in microseconds, so a tight budget can be exhausted — and the + # loop parked — before this test has even swapped the ticket. The budget is + # only here to stop a polling loop spinning forever; it is not the contract. + with fx.fast_sleep(max_sleeps=200): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + store.put( + fx.USER_OPEN_ID, fx.make_uat("u-NOSCOPE", scopes=["im:message:send_as_bot"]) + ) + # Guarded on the count first: `vc.last` raises when nothing has been + # recorded yet, and a predicate that raises escapes the wait instead of + # being retried. + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 1 + and vc.last(fx.URI_EVENTS).authorization == "Bearer u-NOSCOPE", + what="the loop to use the scope-less ticket", + ) + rounds = vc.count(fx.URI_EVENTS) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= rounds + 3, what="three more rounds" + ) + session.dispose() + + assert flow.start_calls == [] + + +async def test_non_interactive_lookup_refuses_rather_than_prompting(vc): + store = fx.FakeTokenStore() + flow = fx.FakeDeviceFlow() + + with pytest.raises(UATAuthError): + await uat_runner.resolve_user_auth_non_interactive( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + ) + + assert flow.start_calls == [] + # Deleting is the interactive path's prerogative: it can immediately ask + # for a new authorization, whereas a polling loop can only make the next + # unrelated call fail with a surprise card. + assert store.delete_calls == [] + + +async def test_a_rotated_refresh_token_is_written_back(vc): + """The old refresh token is dead the moment it is used. Not persisting the + new one makes every holder of this ticket invalidate it for the others.""" + store = fx.FakeTokenStore() + store.put( + fx.USER_OPEN_ID, + fx.make_uat("u-OLD", refresh_token="r-old", expires_in=10.0), + ) + rotated = fx.make_uat("u-NEW", refresh_token="r-new") + flow = fx.FakeDeviceFlow(refresh_results=[rotated]) + + resolved = await uat_runner.resolve_user_auth_non_interactive( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + ) + + assert flow.refresh_calls == ["r-old"] + assert resolved.access_token == "u-NEW" + stored = store.data[fx.USER_OPEN_ID] + assert stored.refresh_token == "r-new" + assert stored.access_token == "u-NEW" + + +async def test_a_failed_refresh_does_not_delete_the_ticket(vc): + store = fx.FakeTokenStore() + store.put( + fx.USER_OPEN_ID, + fx.make_uat("u-OLD", refresh_token="r-old", expires_in=10.0), + ) + flow = fx.FakeDeviceFlow(refresh_results=[UATAuthError("refresh rejected")]) + + with pytest.raises(UATAuthError): + await uat_runner.resolve_user_auth_non_interactive( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + ) + + assert store.delete_calls == [] + assert flow.start_calls == [] + + +async def test_concurrent_refresh_and_interactive_resolution_do_not_overlap(vc): + """Whichever of the two arrives second with a stale refresh token gets a + rejection — and the interactive path answers a rejection by deleting the + ticket, so a valid ticket disappears and its owner gets an unexpected card.""" + store = fx.FakeTokenStore() + store.put( + fx.USER_OPEN_ID, + fx.make_uat("u-OLD", refresh_token="r-old", expires_in=10.0), + ) + inflight = [] + refreshed = fx.make_uat("u-NEW", refresh_token="r-new") + + class _SerializationProbe(fx.FakeDeviceFlow): + async def refresh(self, refresh_token): + inflight.append(refresh_token) + assert len(inflight) == 1, "two refreshes were in flight at once" + await asyncio.sleep(0.02) + inflight.pop() + self.refresh_calls.append(refresh_token) + return refreshed + + flow = _SerializationProbe() + + await asyncio.gather( + uat_runner.resolve_user_auth_non_interactive( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + ), + uat_runner.require_user_auth( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + scopes=[fx.MEETING_EVENT_SCOPE], + context=None, + ), + ) + + assert store.data[fx.USER_OPEN_ID].access_token == "u-NEW" + assert store.delete_calls == [] + + +async def test_both_resolvers_serialize_on_the_same_per_user_lock(vc, monkeypatch): + handed_out = [] + real_lock = uat_runner._get_user_lock + + def _spy(user_open_id): + lock = real_lock(user_open_id) + handed_out.append(lock) + return lock + + monkeypatch.setattr(uat_runner, "_get_user_lock", _spy) + store = fx.FakeTokenStore() + store.put(fx.USER_OPEN_ID, fx.make_uat("u-REAL")) + flow = fx.FakeDeviceFlow() + + await uat_runner.resolve_user_auth_non_interactive( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + ) + await uat_runner.require_user_auth( + device_flow=flow, + token_store=store, + uat_config=UATConfig(), + user_open_id=fx.USER_OPEN_ID, + scopes=[fx.MEETING_EVENT_SCOPE], + context=None, + ) + + assert len(handed_out) >= 2 + assert handed_out[0] is handed_out[1] + + +async def test_a_cross_loop_lock_error_is_treated_as_a_credential_failure( + vc, uat_channel, monkeypatch +): + """These locks are memoized per user and bound to whichever loop created + them; a second loop touching one raises instead of merely not excluding. + Left uncaught it surfaces as an unhandled task exception.""" + + class _WrongLoopLock: + async def __aenter__(self): + raise RuntimeError("got Future attached to a different loop") + + async def __aexit__(self, *exc): + return False + + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + errors = [] + ended = [] + with fx.fast_sleep(max_sleeps=10): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("error", lambda err: errors.append(err)) + session.on("end", lambda event: ended.append(event)) + monkeypatch.setattr( + uat_runner, "_get_user_lock", lambda user_open_id: _WrongLoopLock() + ) + await fx.wait_for(lambda: ended, what="the session to terminate") + + assert [event.reason for event in ended] == ["error"] + assert isinstance(errors[0], FeishuChannelError) + + +# --------------------------------------------------------------------------- +# Picking the meeting to follow +# --------------------------------------------------------------------------- + + +async def test_no_active_meeting_is_reported_as_such(vc, uat_channel): + channel, _store, _flow = uat_channel() + vc.json(fx.URI_ACTIVE_MEETING, fx.active_meeting_body([])) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + assert excinfo.value.code is FeishuChannelErrorCode.MEETING_NOT_FOUND + + +async def test_several_active_meetings_pick_the_first_and_say_so( + vc, uat_channel, caplog +): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + vc.json( + fx.URI_ACTIVE_MEETING, + fx.active_meeting_body( + [ + {"meeting_id": fx.MEETING_ID_STR, "meeting_no": fx.MEETING_NO, "topic": "First"}, + { + "meeting_id": fx.OTHER_MEETING_ID_STR, + "meeting_no": fx.OTHER_MEETING_NO, + "topic": "Second", + }, + ] + ), + ) + + with caplog.at_level(logging.WARNING, logger="Lark"): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + + assert session.meeting_id == fx.MEETING_ID_STR + warnings = [r for r in caplog.records if r.levelno >= logging.WARNING] + assert warnings, caplog.text + # A meeting title is whatever its creator typed, so it may only travel as + # a formatting argument — never pre-interpolated into the message. + assert all("Second" not in str(record.msg) for record in warnings) + + +# --------------------------------------------------------------------------- +# Failure handling +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("status,body", _CREDENTIAL_FAILURES) +async def test_credential_failures_stop_the_event_source(vc, uat_channel, status, body): + channel, store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + with fx.fast_sleep(max_sleeps=12): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json(fx.URI_EVENTS, body, status=status) + await fx.wait_for(lambda: ended, what="the session to terminate") + polls = vc.count(fx.URI_EVENTS) + reads = len(store.get_calls) + await fx.settle() + + assert vc.count(fx.URI_EVENTS) == polls + assert len(store.get_calls) == reads + + +async def test_retryable_failures_use_their_own_ladder_and_give_up_eventually( + vc, uat_channel +): + channel, _store, _flow = uat_channel( + poll_max_consecutive_failures=4, active_meeting_check_interval_seconds=300.0 + ) + with fx.fast_sleep(max_sleeps=20) as clock: + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json(fx.URI_EVENTS, fx.error_body(500, "upstream boom"), status=500) + await fx.wait_for( + lambda: ended, what="the session to give up after repeated failures" + ) + + failure_sleeps = [d for d in clock.durations if d > 10.0 and d < 300.0] + assert failure_sleeps, clock.durations + assert max(failure_sleeps) <= 60.0 + assert [event.reason for event in ended] == ["error"] + + +async def test_termination_emits_error_then_end_then_unregisters_the_session( + vc, uat_channel +): + """Stopping the loop is not enough: neither the idle timeout nor the + liveness probe applies to a follow session, so a half-terminated one is + never collected by anything.""" + channel, _store, _flow = uat_channel( + max_concurrent_sessions=1, active_meeting_check_interval_seconds=300.0 + ) + fx.mark_connected(channel) + order = [] + with fx.fast_sleep(max_sleeps=12): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("error", lambda err: order.append("error")) + session.on("end", lambda event: order.append("end:%s" % event.reason)) + vc.json(fx.URI_EVENTS, fx.error_body(0, "forbidden"), status=403) + await fx.wait_for( + lambda: any(item.startswith("end") for item in order), + what="the session to terminate", + ) + + assert order[:2] == ["error", "end:error"] + # The seat came back, so the terminated session is really gone. + joined = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert joined is not None + + +async def test_a_credential_failure_stops_the_end_detection_loop_too(vc, uat_channel): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=0.02) + with fx.fast_sleep(max_sleeps=12): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json(fx.URI_ACTIVE_MEETING, fx.error_body(99991401, "invalid token"), status=200) + await fx.wait_for(lambda: ended, what="the session to terminate") + counts = (vc.count(fx.URI_EVENTS), vc.count(fx.URI_ACTIVE_MEETING)) + await fx.settle() + + assert (vc.count(fx.URI_EVENTS), vc.count(fx.URI_ACTIVE_MEETING)) == counts + + +async def test_retryable_end_detection_failures_never_end_the_session(vc, uat_channel): + """These failures are correlated across every follow session — same + endpoint, same cadence, often the same user — so ending the session on them + would end all of them together, while their transcripts were flowing fine.""" + # Real intervals rather than collapsed sleeps. Two loops run for the whole + # test here, and with sleep collapsed they spin as fast as the loop allows + # and burn any sleep budget before the assertions land. Small real intervals + # keep them at a sane rate and remove the timing dependence entirely; the + # failure ladder's ceiling is covered by + # `test_retryable_failures_use_their_own_ladder_and_give_up_eventually`. + channel, _store, _flow = uat_channel( + active_meeting_check_interval_seconds=0.02, + poll_min_interval_seconds=0.01, + poll_max_interval_seconds=0.02, + poll_failure_max_interval_seconds=0.05, + ) + got = [] + ended = [] + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("transcript", lambda event: got.append(event)) + session.on("end", lambda event: ended.append(event)) + vc.json(fx.URI_ACTIVE_MEETING, fx.error_body(500, "boom"), status=500) + await fx.wait_for( + lambda: vc.count(fx.URI_ACTIVE_MEETING) >= 6, + what="the end-detection loop to keep trying", + ) + vc.sequence( + fx.URI_EVENTS, + [fx.poll_events([fx.poll_item("transcript_received")]), fx.poll_events([])], + ) + await fx.wait_for(lambda: got, what="a transcript on the still-live session") + session.dispose() + + assert ended == [] + + +async def test_the_meeting_dropping_off_the_active_list_ends_the_session( + vc, uat_channel +): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=0.02) + with fx.fast_sleep(max_sleeps=20): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + ended = [] + session.on("end", lambda event: ended.append(event)) + vc.json(fx.URI_ACTIVE_MEETING, fx.active_meeting_body([])) + await fx.wait_for(lambda: ended, what="the end signal") + polls = vc.count(fx.URI_EVENTS) + await fx.settle() + + assert [event.reason for event in ended] == ["no_longer_active"] + assert vc.count(fx.URI_EVENTS) == polls diff --git a/lark_channel/channel/meeting/tests/test_raw_events.py b/lark_channel/channel/meeting/tests/test_raw_events.py new file mode 100644 index 0000000..6949f59 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_raw_events.py @@ -0,0 +1,301 @@ +"""Subscribing to event types the channel has not wrapped. + +Two traps shape every case here. The dispatcher keeps callback events (the +ones whose return value goes back to Feishu) in a different table from plain +events, and it consults the callback table first — so a callback type +registered into the plain table never fires and never complains. And the +dispatcher is rebuilt from scratch on every start, so a subscription that +only lives on the current dispatcher instance goes quiet after one restart, +also without complaining. +""" + +import json +import logging +import time + +import pytest + +from lark_channel.channel.config import InboundConfig, PolicyConfig +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode +from lark_channel.core.json import JSON +from lark_channel.core.exception import EventException +from lark_channel.channel.raw_events import RawEventRegistry + +from . import fixtures as fx + + +def _envelope(event_type, event=None, *, event_id="env-raw-1"): + return { + "schema": "2.0", + "header": { + "event_id": event_id, + "event_type": event_type, + "create_time": "1730000000000", + "token": "", + "app_id": "cli_x", + "tenant_key": "tk_1", + }, + "event": event if event is not None else {"marker": "payload"}, + } + + +def _im_message(text="hello", open_id="ou_sender", message_id="om_raw_1"): + # A current timestamp, because the built-in inbound path runs a staleness + # check: a fixed one from the past is rejected as stale before it reaches + # any handler, which would make these cases fail for a reason that has + # nothing to do with raw subscriptions. + now = str(int(time.time() * 1000)) + return _envelope( + "im.message.receive_v1", + { + "sender": { + "sender_id": {"open_id": open_id, "user_id": "u_sender"}, + "sender_type": "user", + }, + "message": { + "message_id": message_id, + "chat_id": "oc_p2p", + "chat_type": "p2p", + "message_type": "text", + "create_time": now, + "update_time": now, + "content": json.dumps({"text": text}), + "mentions": [], + }, + }, + event_id="env-im-%s" % message_id, + ) + + +def _card_action(): + return _envelope( + "card.action.trigger", + { + "operator": {"open_id": "ou_clicker", "tenant_key": "tk_1"}, + "action": {"tag": "button", "value": {"kind": "raw-test"}}, + "context": {"open_message_id": "om_card_1", "open_chat_id": "oc_1"}, + "token": "card-token", + }, + event_id="env-card-1", + ) + + +async def test_an_unwrapped_event_type_reaches_its_handler(vc, tat_channel): + channel = tat_channel() + first, second = [], [] + off_first = channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: first.append(p)) + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: second.append(p)) + + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1")) + await fx.wait_for(lambda: first and second, what="both raw handlers") + assert isinstance(first[0], dict) + + off_first() + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1", event_id="env-raw-2")) + await fx.wait_for(lambda: len(second) >= 2, what="the surviving raw handler") + assert len(first) == 1 + + +async def test_a_wrapped_event_type_keeps_its_builtin_handler(vc, tat_channel): + """Same-key registration raises in this dispatcher rather than silently + overwriting, so the two handlers have to be combined explicitly.""" + channel = tat_channel() + messages, raw = [], [] + channel.on("message", lambda event: messages.append(event)) + channel.on_raw_event("im.message.receive_v1", lambda p: raw.append(p)) + + fx.deliver(channel, _im_message("hi there")) + + await fx.wait_for(lambda: raw, what="the raw handler") + await fx.wait_for(lambda: messages, what="the wrapped message handler") + + +async def test_a_callback_event_type_fires_and_keeps_returning_the_builtin_result( + vc, tat_channel +): + """Registering this type as a plain event puts it in the table the + dispatcher never reaches for it.""" + channel = tat_channel() + raw = [] + channel.on_raw_event("card.action.trigger", lambda p: raw.append(p)) + + result = fx.deliver(channel, _card_action()) + + await fx.wait_for(lambda: raw, what="the raw handler on a callback event") + assert result is not None + assert JSON.marshal(result) is not None + + +async def test_a_callback_type_with_no_builtin_handler_still_answers_feishu( + vc, tat_channel +): + """A callback with no valid return value leaves the card's button dead in + the user's client.""" + channel = tat_channel() + raw = [] + channel.on_raw_event("url.preview.get", lambda p: raw.append(p)) + + result = fx.deliver(channel, _envelope("url.preview.get", {"url": "https://x.test"})) + + await fx.wait_for(lambda: raw, what="the raw handler") + assert result is not None + assert JSON.marshal(result) is not None + + +@pytest.mark.parametrize( + "event_type", ["p2.im.message.receive_v1", "p1.drive.notice.comment_add_v1"] +) +def test_a_prefixed_event_type_is_rejected(vc, tat_channel, event_type): + """The prefix is added internally; passing one produces a key like + ``p2.p2.x`` that can never match an incoming event.""" + channel = tat_channel() + with pytest.raises(FeishuChannelError) as excinfo: + channel.on_raw_event(event_type, lambda p: None) + assert excinfo.value.code is FeishuChannelErrorCode.FORMAT_ERROR + + +async def test_unsubscribing_the_last_handler_does_not_uninstall_the_processor( + vc, tat_channel +): + """An unregistered type raises inside the dispatcher, which on the socket + path prints a full traceback for every single event of that type.""" + channel = tat_channel() + off = channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: None) + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1")) + off() + + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1", event_id="env-raw-3")) + + +async def test_a_raw_handler_that_raises_leaves_the_builtin_path_alone( + vc, tat_channel +): + channel = tat_channel() + messages, errors = [], [] + channel.on("message", lambda event: messages.append(event)) + channel.on("error", lambda err: errors.append(err)) + + def _explode(payload): + raise RuntimeError("handler bug") + + channel.on_raw_event("im.message.receive_v1", _explode) + fx.deliver(channel, _im_message("still delivered")) + + await fx.wait_for(lambda: messages, what="the wrapped message handler") + await fx.wait_for(lambda: errors, what="the raw handler failure") + + +async def test_raw_subscriptions_ignore_the_raw_payload_mirror_switch(vc, make_ch): + """``on_raw_event`` and the ``raw`` event are different features: one + subscribes to unwrapped event types, the other mirrors already-wrapped + events, and only the latter is what that switch controls.""" + channel = make_ch( + meeting=fx.meeting_config(), inbound=InboundConfig(emit_raw_events=False) + ) + fx.mark_connected(channel) + raw = [] + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: raw.append(p)) + + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1")) + await fx.wait_for(lambda: raw, what="the raw handler") + + +async def test_a_policy_that_blocks_every_dm_does_not_block_a_raw_subscription( + vc, make_ch +): + """This escape hatch sits outside the policy gate, the dedup cache, and the + loop guard by construction. Subscribing to a type the channel already + handles therefore opens an unpoliced path into that type — deliberate, and + pinned here so it stays a documented property rather than a later surprise.""" + channel = make_ch( + meeting=fx.meeting_config(), + policy=PolicyConfig(dm_policy="allowlist", allow_from=[]), + ) + fx.mark_connected(channel) + messages, raw = [], [] + channel.on("message", lambda event: messages.append(event)) + channel.on_raw_event("im.message.receive_v1", lambda p: raw.append(p)) + + fx.deliver(channel, _im_message("from a stranger", open_id="ou_stranger")) + + await fx.wait_for(lambda: raw, what="the raw handler") + await fx.settle() + assert messages == [] + + +async def test_subscriptions_survive_a_dispatcher_rebuild(vc, tat_channel): + """Every start rebuilds the whole processor table from scratch.""" + channel = tat_channel() + raw = [] + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: raw.append(p)) + + channel._dispatcher = channel._build_dispatcher() + + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1")) + await fx.wait_for(lambda: raw, what="the raw handler after a rebuild") + assert channel.get_meeting_event_health().registered is True + + +async def test_two_subscriptions_to_one_event_type_survive_a_rebuild_together( + vc, tat_channel +): + """Replaying handler by handler makes the second registration for a type + raise, and that exception escapes the rebuild and takes the whole start + down — the message path included. A single-handler check cannot see it.""" + channel = tat_channel() + first, second = [], [] + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: first.append(p)) + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: second.append(p)) + + channel._dispatcher = channel._build_dispatcher() + + fx.deliver(channel, _envelope("vc.bot.meeting_started_v1")) + await fx.wait_for(lambda: first and second, what="both handlers after a rebuild") + + +async def test_a_broken_replay_entry_cannot_take_the_rebuild_down( + vc, tat_channel, caplog, monkeypatch +): + channel = tat_channel() + real_install = RawEventRegistry._install + + def _poisoned(self, target, event_type): + if event_type == "poison.event_v1": + raise EventException("processor already registered, type: %s" % event_type) + return real_install(self, target, event_type) + + healthy = [] + channel.on_raw_event("poison.event_v1", lambda p: None) + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: healthy.append(p)) + monkeypatch.setattr(RawEventRegistry, "_install", _poisoned) + + with caplog.at_level(logging.WARNING, logger="Lark"): + channel._dispatcher = channel._build_dispatcher() + + assert any(record.levelno >= logging.WARNING for record in caplog.records), caplog.text + assert channel.get_meeting_event_health().reason + + +async def test_a_new_subscription_reaches_a_dispatcher_somebody_already_holds( + vc, tat_channel +): + """``on_raw_event`` installs onto the running dispatcher in place, so a + subscription takes effect immediately. Rebuilding and reassigning would only + help consumers that re-read the channel's attribute; a transport holds the + instance it was given and would not see the subscription until the next + start. + + Delivering through a reference captured *before* subscribing is what tells + those two apart — the suite's own helper re-reads the attribute at delivery + time, so every other case here passes either way. + """ + channel = tat_channel() + captured = channel.dispatcher + + seen = [] + channel.on_raw_event("vc.bot.meeting_started_v1", lambda p: seen.append(p)) + + captured._do_without_validation( + json.dumps(_envelope("vc.bot.meeting_started_v1")).encode("utf-8") + ) + await fx.wait_for(lambda: seen, what="the handler through the held dispatcher") diff --git a/lark_channel/channel/meeting/tests/test_registry.py b/lark_channel/channel/meeting/tests/test_registry.py new file mode 100644 index 0000000..295d47b --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_registry.py @@ -0,0 +1,334 @@ +"""The concurrency gate, session reuse, invite filtering, and follow filtering. + +Session creation is triggered from outside the process — anyone who can drag +the bot into a meeting, or send the bot a chat command — so both entry points +need a ceiling and an identity filter. +""" + +import asyncio +import threading + +import pytest + +from lark_channel.channel.config import MeetingChannelConfig, PolicyConfig +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode + +from . import fixtures as fx + + +def _two_active_meetings(): + return fx.active_meeting_body( + [ + { + "meeting_id": fx.MEETING_ID_STR, + "meeting_no": fx.MEETING_NO, + "topic": "First", + }, + { + "meeting_id": fx.OTHER_MEETING_ID_STR, + "meeting_no": fx.OTHER_MEETING_NO, + "topic": "Second", + }, + ] + ) + + +async def test_join_at_the_ceiling_refuses_without_calling_the_api(vc, tat_channel): + channel = tat_channel(max_concurrent_sessions=1) + await channel.join_meeting(fx.MEETING_NO) + joins = vc.count(fx.URI_JOIN) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.OTHER_MEETING_NO) + + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + assert vc.count(fx.URI_JOIN) == joins + + +async def test_follow_at_the_ceiling_refuses_without_sending_anything(vc, uat_channel): + channel, _store, _flow = uat_channel( + max_concurrent_sessions=1, active_meeting_check_interval_seconds=300.0 + ) + vc.json(fx.URI_ACTIVE_MEETING, _two_active_meetings()) + await channel.follow_my_meeting( + user_open_id=fx.USER_OPEN_ID, meeting_no=fx.MEETING_NO + ) + await fx.settle() + before = len(vc.calls) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting( + user_open_id=fx.USER_OPEN_ID, meeting_no=fx.OTHER_MEETING_NO + ) + + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + assert len(vc.calls) == before + + +async def test_the_ceiling_counts_followed_and_joined_meetings_together( + vc, uat_channel +): + """Following leaks harder than joining does: it needs no socket, no + liveness probe applies to it, and a permanently rate-limited follow session + is designed never to end itself. A gate that only counts joins is bolted + to the side that does not leak.""" + channel, _store, _flow = uat_channel( + max_concurrent_sessions=2, active_meeting_check_interval_seconds=300.0 + ) + fx.mark_connected(channel) + vc.sequence( + fx.URI_JOIN, + [fx.join_body(fx.OTHER_MEETING_ID_STR), fx.join_body("999999")], + ) + + await channel.follow_my_meeting( + user_open_id=fx.USER_OPEN_ID, meeting_no=fx.MEETING_NO + ) + await channel.join_meeting(fx.OTHER_MEETING_NO) + await fx.settle() + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting("555555555") + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + + +async def test_following_the_same_meeting_twice_reuses_one_session_and_one_loop( + vc, uat_channel +): + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + first = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.settle() + polls = vc.count(fx.URI_EVENTS) + + second = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.settle() + + assert second is first + # A second polling loop would fire its own first request immediately. + assert vc.count(fx.URI_EVENTS) == polls + + +async def test_a_permanently_rate_limited_follow_session_still_holds_its_seat( + vc, uat_channel +): + channel, _store, _flow = uat_channel( + max_concurrent_sessions=1, + poll_max_consecutive_failures=1000, + active_meeting_check_interval_seconds=300.0, + ) + fx.mark_connected(channel) + vc.json(fx.URI_EVENTS, fx.error_body(99991402, "too many request"), status=429) + + with fx.fast_sleep(max_sleeps=8): + await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 4, what="several failing poll rounds" + ) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.OTHER_MEETING_NO) + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + + +async def test_concurrent_joins_of_one_meeting_number_call_the_api_once( + vc, tat_channel +): + """The second caller has to join the in-flight attempt, not start its own. + + The overlap is arranged rather than hoped for: the first join is held open + at the transport until the second has entered. `asyncio.gather` alone does + not establish it — the fake transport returns without suspending, so the + first call can run start to finish, release its claim, and leave the second + with nothing in flight to find. That made this assertion pass or fail on + which Python version happened to yield somewhere in the call chain. + """ + channel = tat_channel() + # A `threading.Event`, not `asyncio.Event`: the entry point marshals onto the + # channel's background loop, so the responder runs on a different loop than + # this test and an asyncio primitive built here would not be awaitable there. + release = threading.Event() + + async def held(call): + while not release.is_set(): + await asyncio.sleep(0.001) + return (200, fx.join_body()) + + vc.route(fx.URI_JOIN, held) + + first_task = asyncio.ensure_future(channel.join_meeting(fx.MEETING_NO)) + await fx.wait_for( + lambda: vc.count(fx.URI_JOIN) == 1, what="the first join to reach the platform" + ) + second_task = asyncio.ensure_future(channel.join_meeting(fx.MEETING_NO)) + await asyncio.sleep(0.05) # let the second call get as far as it can + release.set() + + first, second = await asyncio.gather(first_task, second_task) + + assert vc.count(fx.URI_JOIN) == 1 + assert first is second + + +async def test_disposing_a_session_does_not_hand_the_seat_back(vc, tat_channel): + """Disposal stops local work but leaves the bot a participant server-side. + Reading the gate off live sessions makes it drop to zero while the bot is + still sitting in the meeting.""" + channel = tat_channel(max_concurrent_sessions=1) + session = await channel.join_meeting(fx.MEETING_NO) + session.dispose() + await fx.settle() + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.join_meeting(fx.OTHER_MEETING_NO) + assert excinfo.value.code is FeishuChannelErrorCode.TOO_MANY_SESSIONS + + +async def test_idle_timeout_leaves_the_meeting_and_hands_the_seat_back( + vc, tat_channel +): + channel = tat_channel( + max_concurrent_sessions=1, + idle_timeout_seconds=0.05, + liveness_probe_interval_seconds=0.0, + ) + vc.sequence( + fx.URI_JOIN, + [fx.join_body(fx.MEETING_ID_STR), fx.join_body(fx.OTHER_MEETING_ID_STR)], + ) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + await fx.wait_for(lambda: ended, what="the idle timeout") + assert ended[0].reason == "idle_timeout" + await fx.wait_for(lambda: vc.count(fx.URI_LEAVE) == 1, what="the leave call") + + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert second.meeting_id == fx.OTHER_MEETING_ID_STR + + +async def test_idle_reclamation_is_off_by_default(vc, tat_channel): + """Reclaiming an idle meeting means the bot visibly walks out of a meeting + where people simply were not talking.""" + assert MeetingChannelConfig().idle_timeout_seconds == 0.0 + + channel = tat_channel(liveness_probe_interval_seconds=0.0) + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + await asyncio.sleep(0.2) + assert ended == [] + assert vc.count(fx.URI_LEAVE) == 0 + + +async def test_repeated_join_and_reclaim_cycles_do_not_accumulate(vc, tat_channel): + channel = tat_channel(max_concurrent_sessions=2) + vc.route(fx.URI_JOIN, lambda call: (200, fx.join_body(fx.MEETING_ID_STR))) + + async def _tasks(): + counts = [len(asyncio.all_tasks())] + + async def _count_bg(): + return len(asyncio.all_tasks()) + + future = asyncio.run_coroutine_threadsafe(_count_bg(), channel._bg_loop) + counts.append(future.result(timeout=2.0)) + return sum(counts) + + baseline = None + for round_index in range(8): + session = await channel.join_meeting(fx.MEETING_NO) + await session.leave() + await fx.settle(2) + if round_index == 2: + baseline = await _tasks() + + assert baseline is not None + assert await _tasks() <= baseline + 2 + + +async def test_invite_from_outside_the_allowlist_is_dropped(vc, tat_channel): + channel = tat_channel(invite_allowlist=["ou_trusted"]) + invited = [] + channel.on("meetingInvited", lambda event: invited.append(event)) + + fx.deliver(channel, fx.push_meeting_invited(inviter_open_id="ou_stranger")) + await fx.settle() + + assert invited == [] + assert vc.count(fx.URI_JOIN) == 0 + + +async def test_invites_bypass_the_message_policy_by_default(vc, make_ch): + """The invite path is the only way into a joined meeting, and none of the + message-policy knobs apply to it. That bypass is deliberate; pinning it as + tested behaviour is what keeps it from being rediscovered as a bug.""" + channel = make_ch( + meeting=fx.meeting_config(invite_allowlist=None), + policy=PolicyConfig(dm_policy="allowlist", allow_from=[], group_policy="allowlist"), + ) + fx.mark_connected(channel) + invited = [] + channel.on("meetingInvited", lambda event: invited.append(event)) + + fx.deliver(channel, fx.push_meeting_invited(inviter_open_id="ou_anyone")) + + await fx.wait_for(lambda: invited, what="the invite to reach the handler") + assert invited[0].meeting_no == fx.MEETING_NO + assert invited[0].inviter.id == "ou_anyone" + + +async def test_follow_outside_the_allowlist_touches_neither_ticket_nor_network( + vc, uat_channel +): + channel, store, flow = uat_channel(follow_allowlist=["ou_trusted"]) + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + + assert excinfo.value.code is FeishuChannelErrorCode.PERMISSION_DENIED + assert store.get_calls == [] + assert flow.start_calls == [] + assert vc.calls == [] + + +async def test_follow_allowlist_is_checked_before_session_reuse(vc, uat_channel): + """Reuse is keyed on the meeting, so checking the allowlist afterwards lets + an unlisted caller inherit somebody else's live session — and with it that + person's transcript.""" + channel, store, _flow = uat_channel( + follow_allowlist=["ou_listed"], active_meeting_check_interval_seconds=300.0 + ) + store.put("ou_listed", fx.make_uat("u-LISTED", open_id="ou_listed")) + listed_session = await channel.follow_my_meeting(user_open_id="ou_listed") + assert listed_session.meeting_id == fx.MEETING_ID_STR + + with pytest.raises(FeishuChannelError) as excinfo: + await channel.follow_my_meeting(user_open_id="ou_not_on_the_list") + assert excinfo.value.code is FeishuChannelErrorCode.PERMISSION_DENIED + + +async def test_follow_accepts_any_open_id_by_default_and_never_pings_the_owner( + vc, uat_channel +): + """The SDK has no way to tell whether the supplied open_id belongs to the + caller, and a cache hit resolves silently. Both halves of that are pinned + here so the bypass stays a documented property.""" + assert MeetingChannelConfig().follow_allowlist is None + + channel, store, flow = uat_channel(active_meeting_check_interval_seconds=300.0) + prompts = [] + + class _PromptContext: + async def respond(self, card): + prompts.append(card) + + session = await channel.follow_my_meeting( + user_open_id=fx.USER_OPEN_ID, prompt_context=_PromptContext() + ) + await fx.settle() + + assert session.mode == "uat" + assert flow.start_calls == [] + assert prompts == [] diff --git a/lark_channel/channel/meeting/tests/test_self_echo.py b/lark_channel/channel/meeting/tests/test_self_echo.py new file mode 100644 index 0000000..3baf910 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_self_echo.py @@ -0,0 +1,80 @@ +"""Marking the bot's own contributions when they come back around.""" + +from . import fixtures as fx + + +def _chat_from(open_id): + return fx.push_activity( + [ + fx.push_item( + "chat_received", + [ + fx.chat_item( + shape="push", + operator=fx.actor(open_id, shape="push", name="Someone"), + ) + ], + ) + ] + ) + + +async def test_own_message_is_flagged_and_still_delivered(vc, tat_channel): + """Dropping it would break transcript-style consumers that need the bot's + own turns; the flag lets each consumer decide.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("chat", lambda event: got.append(event)) + + fx.deliver(channel, _chat_from(fx.BOT_OPEN_ID)) + + await fx.wait_for(lambda: got, what="the echoed chat message") + assert got[0].self_echo is True + + +async def test_follow_mode_never_flags_an_echo(vc, uat_channel): + """In follow mode the bot is not in the meeting at all, so nothing in the + stream can have come from it.""" + channel, _store, _flow = uat_channel() + got = [] + body = fx.poll_events( + [ + fx.poll_item( + "chat_received", + [ + fx.chat_item( + shape="poll", + operator=fx.actor(fx.BOT_OPEN_ID, shape="poll", name="Helper"), + ) + ], + ) + ] + ) + with fx.fast_sleep(): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + session.on("chat", lambda event: got.append(event)) + vc.sequence(fx.URI_EVENTS, [body, fx.poll_events([])]) + await fx.wait_for(lambda: got, what="the polled chat message") + session.dispose() + + assert got[0].self_echo is False + + +async def test_unresolved_bot_identity_flags_the_event_as_possibly_our_own( + vc, tat_channel, make_ch +): + """``False`` means "definitely not me" and lets the reply loop close. Until + the bot's own id is known, the honest answer is "maybe".""" + channel = make_ch(meeting=fx.meeting_config()) + fx.mark_connected(channel, bot_open_id=None) + assert channel.bot_identity is None + + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("chat", lambda event: got.append(event)) + + fx.deliver(channel, _chat_from("ou_someone_else")) + + await fx.wait_for(lambda: got, what="the chat message") + assert got[0].self_echo is True diff --git a/lark_channel/channel/meeting/tests/test_send_message.py b/lark_channel/channel/meeting/tests/test_send_message.py new file mode 100644 index 0000000..af564da --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_send_message.py @@ -0,0 +1,51 @@ +"""Speaking into the meeting.""" + +import json + +import pytest + +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode + +from . import fixtures as fx + + +async def test_text_is_wrapped_and_given_an_idempotency_key(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + await session.send_message("hello 会议") + + call = vc.last(fx.URI_MESSAGE) + assert call.body["msg_type"] == "text" + assert json.loads(call.body["content"]) == {"text": "hello 会议"} + assert call.body["uuid"] + + +async def test_follow_mode_cannot_speak_in_the_meeting(vc, uat_channel): + """In follow mode the bot is not a participant, so there is nowhere for a + message to appear.""" + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + + with pytest.raises(FeishuChannelError) as excinfo: + await session.send_message("hello") + assert excinfo.value.code is FeishuChannelErrorCode.NOT_SUPPORTED + assert vc.count(fx.URI_MESSAGE) == 0 + + +async def test_exceeding_the_per_minute_budget_refuses_before_calling_the_api( + vc, tat_channel +): + """The bot's own messages come back as meeting chat, so a handler that + replies without checking the echo flag self-amplifies at network speed.""" + channel = tat_channel(send_rate_limit_per_minute=2) + session = await channel.join_meeting(fx.MEETING_NO) + + await session.send_message("one") + await session.send_message("two") + sent = vc.count(fx.URI_MESSAGE) + + with pytest.raises(FeishuChannelError) as excinfo: + await session.send_message("three") + assert excinfo.value.code is FeishuChannelErrorCode.RATE_LIMITED + assert vc.count(fx.URI_MESSAGE) == sent diff --git a/lark_channel/channel/meeting/tests/test_serial_queue.py b/lark_channel/channel/meeting/tests/test_serial_queue.py new file mode 100644 index 0000000..c1191f8 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_serial_queue.py @@ -0,0 +1,100 @@ +"""The per-session delivery queue's ceiling. + +Serial, awaited delivery means a slow handler makes work pile up at whatever +rate the meeting produces activity. Bounding that pile is not just a memory +question: *which* end is dropped decides whether the application rebuilds the +meeting correctly or rebuilds something plausible and wrong. +""" + +import pytest + +from lark_channel.channel.meeting.health_view import MeetingHealthView +from lark_channel.channel.meeting.serial_queue import ( + REPORT_RESERVE, + SerialDelivery, +) + +from . import fixtures as fx + + +def _delivery(**kwargs): + return SerialDelivery(on_handler_error=lambda exc: None, **kwargs) + + +def test_a_full_queue_refuses_more_activity(): + """Unbounded growth is the failure this ceiling exists to prevent: nothing + consumes while a handler is parked, and the meeting keeps producing.""" + queue = _delivery(max_queued=3) + noop = lambda payload: None + + assert [queue.submit([noop], i) for i in range(3)] == [True, True, True] + assert queue.submit([noop], 3) is False + assert queue.dropped == 1 + + +async def test_overflow_drops_the_newest_so_what_is_queued_stays_in_order(): + """Order is the guarantee this queue exists for. A document swap arrives as + `magic_share_ended` then `magic_share_started`, so evicting from the front + to make room splits that pair and hands the application a queue that still + looks complete. An implementation that evicts the oldest must fail here. + """ + seen = [] + queue = _delivery(max_queued=2) + record = lambda payload: seen.append(payload) + + queue.submit([record], "magic_share_ended") + queue.submit([record], "magic_share_started") + queue.submit([record], "arrived_after_the_ceiling") + + queue.start() + try: + await fx.wait_for(lambda: len(seen) == 2, what="the two queued deliveries") + assert seen == ["magic_share_ended", "magic_share_started"] + finally: + # The worker outlives the test otherwise, and its coroutine is dropped + # when the loop closes — the same leak this suite errors on. + queue.cancel() + + +def test_an_error_report_gets_through_a_queue_full_of_activity(): + """Teardown submits here — the `end` event, and any error raised by the + departure call. Those are what explain why a session went away, so a + ceiling filled by transcripts must not be able to drop the explanation and + keep the noise.""" + queue = _delivery(max_queued=1) + noop = lambda payload: None + + queue.submit([noop], "activity") + assert queue.submit([noop], "more activity") is False + assert queue.submit([noop], "why the session ended", reserved=True) is True + + +def test_the_report_headroom_is_itself_bounded(): + """Reserved is headroom, not an exemption — otherwise a handler that raises + on every delivery would grow the queue through the reports about it.""" + queue = _delivery(max_queued=0) + noop = lambda payload: None + + accepted = sum( + 1 for _ in range(REPORT_RESERVE + 5) if queue.submit([noop], "r", reserved=True) + ) + assert accepted == REPORT_RESERVE + + +def test_drops_reach_the_channel_readout_from_live_and_retired_sessions(): + """A drop only this object knows about is exactly the silent loss the health + readout exists to make diagnosable — and it has to survive the session + ending, which is when somebody goes looking.""" + + class _Session: + def __init__(self, dropped): + self.dropped_deliveries = dropped + + def get_stats(self): + return {} + + live = _Session(2) + view = MeetingHealthView(lambda: [live]) + view.retire(_Session(3)) + + assert view.snapshot().dropped == 5 diff --git a/lark_channel/channel/meeting/tests/test_session.py b/lark_channel/channel/meeting/tests/test_session.py new file mode 100644 index 0000000..bc50a69 --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_session.py @@ -0,0 +1,285 @@ +"""Session lifecycle: entering, leaving, and being torn down.""" + +import asyncio +import gc +import logging +import time + +import pytest + +from lark_channel.channel.errors import FeishuChannelError, FeishuChannelErrorCode + +from . import fixtures as fx + + +async def test_joining_requires_a_connection_while_following_does_not( + vc, make_ch, uat_channel +): + """Joining depends on pushed activity, so it needs the socket. Following is + REST plus a user ticket, so opening a socket for it is pure overhead.""" + unconnected = make_ch(meeting=fx.meeting_config()) + with pytest.raises(FeishuChannelError) as excinfo: + await unconnected.join_meeting(fx.MEETING_NO) + assert excinfo.value.code is FeishuChannelErrorCode.NOT_CONNECTED + assert vc.count(fx.URI_JOIN) == 0 + + channel, _store, _flow = uat_channel() + with fx.fast_sleep(max_sleeps=3): + session = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + assert session.mode == "uat" + session.dispose() + + +async def test_join_sends_the_nine_digit_number_and_keeps_the_long_id(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + call = vc.last(fx.URI_JOIN) + assert call.body["join_type"] == 1 + assert call.body["join_identify"]["meeting_no"] == fx.MEETING_NO + assert session.meeting_no == fx.MEETING_NO + assert session.meeting_id == fx.MEETING_ID_STR + assert session.mode == "tat" + + +async def test_leave_and_dispose_are_both_idempotent(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + await session.leave() + await session.leave() + session.dispose() + session.dispose() + + assert vc.count(fx.URI_LEAVE) == 1 + + +async def test_leave_still_works_after_dispose(vc, tat_channel): + """This is what makes "dispose does not leave the meeting, so leave before + exiting the process" a usable instruction rather than a trap.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + session.dispose() + await session.leave() + + assert vc.count(fx.URI_LEAVE) == 1 + + +async def test_dispose_does_not_leave_the_meeting(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + session.dispose() + await fx.settle() + + assert vc.count(fx.URI_LEAVE) == 0 + + +async def test_meeting_ended_event_ends_the_session_and_leaves_once(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + fx.deliver(channel, fx.push_meeting_ended()) + + await fx.wait_for(lambda: ended, what="the end event") + assert ended[0].reason == "meeting_ended" + await fx.wait_for(lambda: vc.count(fx.URI_LEAVE) == 1, what="the leave call") + await fx.settle() + assert vc.count(fx.URI_LEAVE) == 1 + + +async def test_channel_disconnect_disposes_sessions_without_leaving_meetings( + vc, tat_channel +): + """A reconnect must not make the bot vanish from every meeting it is in.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + ended = [] + session.on("end", lambda event: ended.append(event)) + + await channel.disconnect() + await fx.settle() + + assert vc.count(fx.URI_LEAVE) == 0 + assert [event.reason for event in ended] == ["disposed"] + + +async def test_dispose_converges_every_task_the_session_started( + vc, tat_channel, caplog +): + channel = tat_channel( + liveness_probe_interval_seconds=0.02, active_meeting_check_interval_seconds=0.02 + ) + session = await channel.join_meeting(fx.MEETING_NO) + await fx.wait_for( + lambda: vc.count(fx.URI_EVENTS) >= 1, what="the first liveness probe" + ) + + with caplog.at_level(logging.WARNING): + session.dispose() + await asyncio.sleep(0.1) + probes_after_dispose = vc.count(fx.URI_EVENTS) + await asyncio.sleep(0.1) + assert vc.count(fx.URI_EVENTS) == probes_after_dispose + gc.collect() + await fx.settle() + + assert "Task was destroyed" not in caplog.text + + +async def test_failed_leave_still_tears_the_session_down(vc, tat_channel): + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + errors = [] + session.on("error", lambda err: errors.append(err)) + vc.route(fx.URI_LEAVE, lambda call: (500, fx.error_body(500, "upstream exploded"))) + + await session.leave() + + await fx.wait_for(lambda: errors, what="the leave failure on the error channel") + assert isinstance(errors[0], FeishuChannelError) + # The meeting is gone from the channel's routing table, so a further push + # for it reaches nobody rather than resurrecting a half-torn-down session. + got = [] + session.on("transcript", lambda event: got.append(event)) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.settle() + assert got == [] + + +async def test_leave_returns_while_a_handler_is_still_parked(vc, tat_channel): + """The handler here yields control and never comes back — a plausible bug + in someone else's code, and the only kind of stall this layer can defend + against. Waiting for it unconditionally would hang teardown, and would + hang the very timeout meant to rescue a stalled session.""" + channel = tat_channel( + max_concurrent_sessions=1, dispose_drain_timeout_seconds=0.2 + ) + vc.sequence( + fx.URI_JOIN, + [fx.join_body(fx.MEETING_ID_STR), fx.join_body(fx.OTHER_MEETING_ID_STR)], + ) + session = await channel.join_meeting(fx.MEETING_NO) + entered = [] + + async def parked(event): + entered.append(event) + await asyncio.Event().wait() + + session.on("transcript", parked) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: entered, what="the handler to be entered") + + started = time.monotonic() + await asyncio.wait_for(session.leave(), timeout=3.0) + assert time.monotonic() - started < 2.0 + + # The seat has to come back even though the queue never drained, otherwise + # one parked handler burns a slot for the life of the process. + second = await channel.join_meeting(fx.OTHER_MEETING_NO) + assert second.meeting_id == fx.OTHER_MEETING_ID_STR + + +async def test_leave_warns_when_the_delivery_queue_could_not_be_drained( + vc, tat_channel, caplog +): + channel = tat_channel(dispose_drain_timeout_seconds=0.2) + session = await channel.join_meeting(fx.MEETING_NO) + entered = [] + + async def parked(event): + entered.append(event) + await asyncio.Event().wait() + + session.on("transcript", parked) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: entered, what="the handler to be entered") + + with caplog.at_level(logging.WARNING, logger="Lark"): + await asyncio.wait_for(session.leave(), timeout=3.0) + + assert any( + record.levelno >= logging.WARNING and "drain" in record.getMessage().lower() + for record in caplog.records + ), caplog.text + + +async def test_disconnect_returns_while_a_handler_is_still_parked(vc, tat_channel): + channel = tat_channel(dispose_drain_timeout_seconds=0.2) + session = await channel.join_meeting(fx.MEETING_NO) + entered = [] + + async def parked(event): + entered.append(event) + await asyncio.Event().wait() + + session.on("transcript", parked) + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received")])) + await fx.wait_for(lambda: entered, what="the handler to be entered") + + started = time.monotonic() + await asyncio.wait_for(channel.disconnect(), timeout=8.0) + assert time.monotonic() - started < 6.0 + + +async def test_an_unknown_session_event_name_is_reported(vc, tat_channel, caplog): + """A typo here produces a handler that is simply never called, which looks + exactly like the platform not sending anything.""" + channel = tat_channel() + session = await channel.join_meeting(fx.MEETING_NO) + + with caplog.at_level(logging.WARNING, logger="Lark"): + session.on("transcripts", lambda event: None) # the name is "transcript" + + assert any( + record.levelno >= logging.WARNING and "transcripts" in record.getMessage() + for record in caplog.records + ), caplog.text + + +async def test_a_replaced_session_for_one_meeting_is_disposed_not_orphaned( + vc, tat_channel, uat_channel +): + """Two sessions for one meeting would leave the first out of the routing + table but still running — and for a follow session that means it keeps + polling the whole meeting with the user's ticket after the application + believes it is gone.""" + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + fx.mark_connected(channel) + + followed = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + assert followed.meeting_id == fx.MEETING_ID_STR + + # The bot then gets pulled into the very same meeting. + joined = await channel.join_meeting(fx.MEETING_NO) + assert joined.meeting_id == fx.MEETING_ID_STR + assert joined is not followed + + # Asserted as "stops growing" rather than "never grew": a round already in + # flight when the takeover happens is not a leak. + await asyncio.sleep(0.1) + settled = vc.count(fx.URI_EVENTS) + await asyncio.sleep(0.2) + assert vc.count(fx.URI_EVENTS) == settled + + +async def test_a_superseded_handle_cannot_eject_the_replacement(vc, uat_channel): + """Holding on to the old handle is easy, and calling ``leave()`` on it would + remove the bot from a meeting the replacement is actively serving.""" + channel, _store, _flow = uat_channel(active_meeting_check_interval_seconds=300.0) + fx.mark_connected(channel) + + followed = await channel.follow_my_meeting(user_open_id=fx.USER_OPEN_ID) + joined = await channel.join_meeting(fx.MEETING_NO) + assert joined is not followed + + await followed.leave() + await fx.settle() + + assert vc.count(fx.URI_LEAVE) == 0 + # And the replacement can still depart on its own behalf. + await joined.leave() + assert vc.count(fx.URI_LEAVE) == 1 diff --git a/lark_channel/channel/meeting/tests/test_stabilizer.py b/lark_channel/channel/meeting/tests/test_stabilizer.py new file mode 100644 index 0000000..c5a3ada --- /dev/null +++ b/lark_channel/channel/meeting/tests/test_stabilizer.py @@ -0,0 +1,85 @@ +"""Transcript settling: the debounce window, and what happens to the backlog.""" + +import asyncio + +from lark_channel.channel.meeting.stabilizer import MAX_PENDING_TRANSCRIPTS + +from . import fixtures as fx + + +def _transcript(text, sentence_id): + return fx.push_activity( + [ + fx.push_item( + "transcript_received", + [fx.transcript_item(shape="push", text=text, sentence_id=sentence_id)], + ) + ], + envelope_event_id="env-%s-%s" % (sentence_id, len(text)), + ) + + +async def test_zero_window_delivers_every_revision(vc, tat_channel): + channel = tat_channel(stabilize_seconds=0.0) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event.text)) + + for text in ("今", "今天", "今天来"): + fx.deliver(channel, _transcript(text, "s-1")) + + await fx.wait_for(lambda: len(got) >= 3, what="three revisions") + assert got == ["今", "今天", "今天来"] + + +async def test_positive_window_delivers_only_the_last_revision(vc, tat_channel): + channel = tat_channel(stabilize_seconds=0.05) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event.text)) + + fx.deliver(channel, _transcript("今", "s-1")) + fx.deliver(channel, _transcript("今天来讨论", "s-1")) + + await fx.wait_for(lambda: got, what="the settled sentence") + await asyncio.sleep(0.15) + assert got == ["今天来讨论"] + + +async def test_pending_sentence_is_flushed_on_dispose_not_dropped(vc, tat_channel): + """The debounce timer dies with the session; if the buffered sentence dies + with it, the last thing anybody said in the meeting is lost.""" + channel = tat_channel(stabilize_seconds=5.0) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event.text)) + + fx.deliver(channel, _transcript("最后一句", "s-1")) + await fx.settle() + assert got == [] + + session.dispose() + await fx.wait_for(lambda: got, what="the buffered sentence to be flushed") + assert got == ["最后一句"] + + +async def test_overflowing_buffer_flushes_the_oldest_rather_than_dropping_it( + vc, tat_channel +): + channel = tat_channel(stabilize_seconds=5.0) + session = await channel.join_meeting(fx.MEETING_NO) + got = [] + session.on("transcript", lambda event: got.append(event.sentence_id)) + + items = [ + fx.transcript_item( + shape="push", text="line-%d" % i, sentence_id="s-%d" % i + ) + for i in range(MAX_PENDING_TRANSCRIPTS + 1) + ] + fx.deliver(channel, fx.push_activity([fx.push_item("transcript_received", items)])) + + await fx.wait_for( + lambda: got, what="the oldest buffered sentence to be pushed out", timeout=10.0 + ) + assert got == ["s-0"] diff --git a/lark_channel/channel/meeting/types.py b/lark_channel/channel/meeting/types.py new file mode 100644 index 0000000..6d79f90 --- /dev/null +++ b/lark_channel/channel/meeting/types.py @@ -0,0 +1,329 @@ +"""Public types for the meeting channel. + +Fields marked *untrusted* are written by meeting participants or the meeting's +creator — who may be external or guest users. They must never be interpolated +into a log message body (only passed as lazy logging arguments, after control +characters are stripped), and never rendered into HTML or a chat message +without escaping. +""" + +from dataclasses import dataclass, field +from typing import Any, Dict, List, Optional + +#: Session-level event names accepted by :meth:`MeetingSession.on`. +MEETING_EVENT_NAMES = ( + "transcript", + "chat", + "participant", + "share", + "document_context", + "end", + "error", +) + + +class MeetingEvents: + """String constants for :meth:`MeetingSession.on` event names. + + Prefer these over literals so a typo surfaces as ``AttributeError`` at + import time rather than a handler that is never called. + """ + + TRANSCRIPT = "transcript" + CHAT = "chat" + PARTICIPANT = "participant" + SHARE = "share" + DOCUMENT_CONTEXT = "document_context" + END = "end" + ERROR = "error" + + +@dataclass +class MeetingActor: + """Whoever produced an item: speaker, chat sender, or participant. + + ``id`` is always an ``open_id`` — the channel pins ``user_id_type`` on + every request so this shares a namespace with the bot's own open_id. + """ + + id: Optional[str] = None + #: untrusted — the participant's display name. + name: Optional[str] = None + user_type: Optional[int] = None + user_role: Optional[int] = None + + +@dataclass +class ShareDocInfo: + """The document being shared. Both fields untrusted.""" + + url: Optional[str] = None + title: Optional[str] = None + + +@dataclass +class MeetingEventBase: + """Fields every in-meeting event carries.""" + + meeting_id: str + actor: MeetingActor + #: This item was produced by our own bot — an in-meeting message pushed + #: back to us, or our own speech transcribed. Delivered anyway: a + #: minute-taking application needs the bot's own turns. Always ``False`` + #: in ``uat`` mode, where the bot is not a participant. + #: + #: Conservative when unknown: before the bot's own open_id is resolved + #: this is ``True``, because ``False`` means "not me" and would wave a + #: feedback loop straight through. + self_echo: bool = False + #: The raw wire item, only when ``MeetingOptions.include_raw`` is set. + #: Unredacted and untrusted, same as an ``on_raw_event`` payload. + raw: Optional[Dict[str, Any]] = None + + +@dataclass +class TranscriptEvent(MeetingEventBase): + #: untrusted — spoken words, as transcribed. + text: str = "" + #: Stable within one sentence. The protocol has no "final" marker, so a + #: later item with the same id supersedes the earlier text; consumers + #: should upsert on this. + sentence_id: Optional[str] = None + language: Optional[str] = None + start_ms: Optional[int] = None + end_ms: Optional[int] = None + + +@dataclass +class MeetingChatEvent(MeetingEventBase): + #: untrusted — the message the participant typed. + content: str = "" + message_id: Optional[str] = None + message_type: Optional[int] = None + send_time: Optional[int] = None + + +@dataclass +class ParticipantEvent(MeetingEventBase): + #: ``"joined"`` or ``"left"``. + action: str = "" + join_time: Optional[int] = None + leave_time: Optional[int] = None + leave_reason: Optional[int] = None + + +@dataclass +class ShareEvent(MeetingEventBase): + #: ``"started"`` or ``"ended"``. + action: str = "" + share_id: Optional[str] = None + doc: Optional[ShareDocInfo] = None + time: Optional[int] = None + + +@dataclass +class DocumentContextEvent(MeetingEventBase): + """A change of context inside the shared document. + + Identifiers only, never content: a comment gives you ``comment_id``, an + image or board gives you ``element_token``. Fetching the body or the asset + is the application's job, within the shared document's temporary grant, + with the permissions it applied for itself. + + ``context_type`` is derived from whichever sub-object is present, because + the generated model has no discriminator field. A payload that does carry + an explicit ``context_type`` is believed over the derivation — that is the + platform having moved ahead of the generated types. + """ + + #: ``"comment_focus"`` / ``"section_location"`` / ``"element_preview"``, + #: or whatever the platform sends explicitly. + context_type: str = "" + share_id: Optional[str] = None + doc: Optional[ShareDocInfo] = None + time: Optional[int] = None + comment_focus: Optional[Dict[str, Any]] = None + #: untrusted — headings come from the document. + section_location: Optional[Dict[str, Any]] = None + #: untrusted — ``element_token`` is an opaque identifier from the document. + element_preview: Optional[Dict[str, Any]] = None + + +@dataclass +class MeetingEndEvent: + """The session is over. No further events will be delivered on it.""" + + meeting_id: str + #: ``"meeting_ended"`` the platform said the meeting ended (tat) + #: ``"no_longer_active"`` the user left it (uat) or a probe proved the bot + #: is no longer a participant (tat) + #: ``"idle_timeout"`` no activity for ``idle_timeout_seconds`` (tat) + #: ``"error"`` the event source stopped unrecoverably + #: ``"left"`` the application called ``leave()`` + #: ``"disposed"`` the application called ``dispose()``, or the + #: channel disconnected + reason: str = "" + + +@dataclass +class MeetingInvitedEvent: + """The bot was invited into a meeting. Delivered on the channel, not on a + session — at this point no session exists yet. + + **This is not covered by** :class:`~..config.PolicyConfig`. It is the only + way ``join_meeting`` gets triggered, and anybody who can add the bot to a + meeting can trigger it. ``MeetingChannelConfig.invite_allowlist`` is the gate. + """ + + meeting_no: str + meeting_id: Optional[str] = None + #: untrusted — the meeting's title. + topic: Optional[str] = None + inviter: Optional[MeetingActor] = None + bot: Optional[MeetingActor] = None + call_id: Optional[str] = None + invite_time: Optional[int] = None + raw: Optional[Dict[str, Any]] = None + + +@dataclass +class MeetingOptions: + """Per-session knobs.""" + + #: Settle window for transcripts, in seconds. ``0.0`` delivers every text + #: change, so consumers see a sentence grow word by word. A positive value + #: delivers a sentence once, after it stops changing for that long — at + #: the cost of one window of latency, and of a sentence never settling + #: while somebody keeps talking. + stabilize_seconds: float = 0.0 + #: Attach the raw wire item to each event. Off by default: the raw payload + #: is unredacted and carries full user ids and message bodies. + include_raw: bool = False + + +@dataclass +class ActivityTypeStats: + """Per ``activity_event_type`` parse accounting. + + ``received`` counts activity objects that arrived; ``empty`` counts those + that unpacked to nothing. Splitting them separates two diagnoses that look + identical from outside and point in opposite directions: "the platform + never sent it" (check the meeting setting, the subscription declaration, + whether the bot is in the meeting) versus "it sent it and we could not + read it" (check the field names). + """ + + received: int = 0 + empty: int = 0 + + +@dataclass +class LivenessHealth: + """The probe's own health. + + Without this, a tenant where the probe's permission assumption does not + hold degrades to "reclamation never happens" with no outward sign. + """ + + last_probe_at: Optional[float] = None + #: ``"in_meeting"`` / ``"not_in_meeting"`` / ``"unknown"``. + last_verdict: Optional[str] = None + consecutive_unknown: int = 0 + + +@dataclass +class MembershipHealth: + """Server-side participation accounting, which drives the session gate. + + Mis-accounting here shows up at the far end as ``join_meeting`` refusing + forever; these counters are how you see it coming. + """ + + #: Seats currently held *as server-side membership* — the entries this + #: ledger tracks. The gate's own reading is wider: it also counts live + #: sessions, so a pure follow deployment can be refused a seat while this + #: number is still zero. + held: int = 0 + #: Entries kept because a departure result was inconclusive, whose session + #: is already gone. These are what lazy reconciliation works through. + retained_without_session: int = 0 + #: Evidence kind -> release count. Keys include ``"ok"``, ``"404"``, + #: ``"121105"``, ``"120004"``, ``"meeting_ended"``, ``"ttl"``. + released_by_evidence: Dict[str, int] = field(default_factory=dict) + reconcile_attempts: int = 0 + + +@dataclass +class MeetingEventHealth: + """Channel-wide view of the in-meeting event path.""" + + #: Whether the internal ``vc.bot.*`` registration took effect on the + #: current dispatcher. The dispatcher is rebuilt on every ``start()``, so + #: this reflects the most recent rebuild. + registered: bool = False + reason: Optional[str] = None + received: int = 0 + last_at: Optional[float] = None + #: Events refused because a session's delivery queue was at its ceiling, + #: which happens when a handler is slower than the meeting. Non-zero means + #: activity was normalized and then never handed to a handler, so the + #: application's picture of the meeting has gaps. + dropped: int = 0 + stats: Dict[str, ActivityTypeStats] = field(default_factory=dict) + liveness: LivenessHealth = field(default_factory=LivenessHealth) + membership: MembershipHealth = field(default_factory=MembershipHealth) + + +#: Activity type -> the array field that carries its items. +ACTIVITY_ITEM_FIELDS = { + "transcript_received": "transcript_received_items", + "chat_received": "chat_received_items", + "participant_joined": "participant_joined_items", + "participant_left": "participant_left_items", + "magic_share_started": "magic_share_started_items", + "magic_share_ended": "magic_share_ended_items", + "document_context_changed": "document_context_changed_items", +} + +#: Activity type -> the field naming whoever produced the item. The platform +#: spells it differently per type; the channel normalizes all of them to +#: ``actor``. +ACTIVITY_ACTOR_FIELDS = { + "transcript_received": "speaker", + "chat_received": "operator", + "participant_joined": "participant", + "participant_left": "participant", + "magic_share_started": "operator", + "magic_share_ended": "operator", + "document_context_changed": "operator", +} + +#: Sub-object -> the ``context_type`` it implies, in probe order. +DOCUMENT_CONTEXT_KINDS = ( + "comment_focus", + "section_location", + "element_preview", +) + +__all__ = [ + "ACTIVITY_ACTOR_FIELDS", + "ACTIVITY_ITEM_FIELDS", + "ActivityTypeStats", + "DOCUMENT_CONTEXT_KINDS", + "DocumentContextEvent", + "LivenessHealth", + "MEETING_EVENT_NAMES", + "MeetingActor", + "MeetingChatEvent", + "MeetingEndEvent", + "MeetingEventBase", + "MeetingEventHealth", + "MeetingEvents", + "MeetingInvitedEvent", + "MeetingOptions", + "MembershipHealth", + "ParticipantEvent", + "ShareDocInfo", + "ShareEvent", + "TranscriptEvent", +] diff --git a/lark_channel/channel/raw_events.py b/lark_channel/channel/raw_events.py new file mode 100644 index 0000000..75d68c7 --- /dev/null +++ b/lark_channel/channel/raw_events.py @@ -0,0 +1,245 @@ +"""Subscribing to Feishu event types the channel has not wrapped. + +Without this there is no legitimate way to receive one: reaching into the +dispatcher's private tables breaks on the next release, and opening a second +socket for the same app is worse — Feishu splits delivery between connections, +so the channel's own message path starts losing events. + +Two properties of the underlying dispatcher shape everything here. + +**There are two tables, and the callback one is consulted first.** Events whose +return value goes back to Feishu (a card button, a link preview) live in a +separate map. A callback type registered into the plain map is never reached — +silently, with no error and no log line — and, worse, the callback answers with +whatever the plain path returns, which leaves the button dead in the user's +client. + +**The whole table is rebuilt on every ``start()``.** A subscription that only +exists on the current dispatcher instance goes quiet after one restart. So +subscriptions live here, in a registry the channel owns, and are replayed onto +each new dispatcher. + +Replay groups by event type and installs **one** processor per type. Registering +per handler would hit the dispatcher's duplicate-key guard on the second +handler for a type, and that exception would escape the rebuild and take +``start()`` down — the message path with it. + +Security, stated because it cannot be inferred: a raw handler runs **after** +signature verification and decryption, so the payload is authentic. But it sits +**outside** the channel's safety pipeline — no policy gate, no dedup, no +processing lock, no loop guard. Subscribing to a type the channel already +handles therefore opens an unpoliced path into that type. That is what an +escape hatch is; it is deliberate, and it is pinned by a test so it stays a +documented property. +""" + +import inspect +from typing import Any, Callable, Dict, List, Optional, Tuple + +from lark_channel.core.log import logger + +from . import _coerce +from .meeting.errors import sanitize_for_log +from .errors import FeishuChannelError, FeishuChannelErrorCode + +Unsubscribe = Callable[[], None] + +#: Event types whose return value is sent back to Feishu, with the builder +#: method that installs them and the empty response each one needs. +_CALLBACK_TYPES: Dict[str, Tuple[str, str]] = { + "card.action.trigger": ( + "register_p2_card_action_trigger", + "P2CardActionTriggerResponse", + ), + "url.preview.get": ("register_p2_url_preview_get", "P2URLPreviewGetResponse"), +} + + +def _empty_response(name: str) -> Any: + from lark_channel.event.callback.model.p2_card_action_trigger import ( + P2CardActionTriggerResponse, + ) + from lark_channel.event.callback.model.p2_url_preview_get import ( + P2URLPreviewGetResponse, + ) + + return { + "P2CardActionTriggerResponse": P2CardActionTriggerResponse, + "P2URLPreviewGetResponse": P2URLPreviewGetResponse, + }[name]({}) + + +class _RawEventProcessor: + """Runs the built-in processor, if any, then the raw handlers. + + The built-in result is the one returned. A raw subscriber must not be able + to change what Feishu is told — the return value of a card callback decides + whether the button the user clicked does anything. + """ + + def __init__(self, *, inner: Any, dispatch: Callable[[Any], None], fallback_response=None): + self._inner = inner + self._dispatch = dispatch + self._fallback_response = fallback_response + + def type(self): + if self._inner is not None: + return self._inner.type() + from lark_channel.event.custom import CustomizedEvent + + return CustomizedEvent + + def do(self, data: Any) -> Any: + result = None + if self._inner is not None: + result = self._inner.do(data) + self._dispatch(data) + if result is not None: + return result + if self._fallback_response is not None: + return self._fallback_response() + return None + + +def _builtin_under(existing: Any) -> Any: + """The genuine built-in processor beneath ``existing``, if any. + + A previous installation of ours must be **replaced**, not wrapped: wrapping + it would run its dispatch and the new one's, calling every handler once per + layer. Only a processor we did not create counts as the built-in. + """ + if isinstance(existing, _RawEventProcessor): + return existing._inner + return existing + + +class RawEventRegistry: + """The channel's own record of raw subscriptions, replayed on each rebuild.""" + + def __init__(self, *, schedule: Callable[[Any], Any], report: Callable[[BaseException], Any]): + self._handlers: Dict[str, List[Callable]] = {} + self._schedule = schedule + self._report = report + + def subscribe(self, event_type: str, handler: Callable) -> Unsubscribe: + if not isinstance(event_type, str) or not event_type: + raise FeishuChannelError( + FeishuChannelErrorCode.FORMAT_ERROR, + "on_raw_event needs a Feishu event type", + ) + if event_type.startswith("p1.") or event_type.startswith("p2."): + # The schema prefix is added internally. Accepting one here would + # build a key like `p2.p2.x`, which no incoming event can match — + # and nothing would report it. + raise FeishuChannelError( + FeishuChannelErrorCode.FORMAT_ERROR, + "on_raw_event takes the event type without a schema prefix, " + "for example 'im.message.receive_v1'", + ) + self._handlers.setdefault(event_type, []).append(handler) + + def unsubscribe() -> None: + handlers = self._handlers.get(event_type) + if not handlers: + return + try: + handlers.remove(handler) + except ValueError: + return + # The key stays even when empty. Removing the processor would make + # the dispatcher raise for every later event of this type, which on + # the socket path prints a full traceback each time. + + return unsubscribe + + @property + def event_types(self) -> List[str]: + return list(self._handlers) + + def install(self, dispatcher: Any) -> Optional[str]: + """Install every subscription onto a **built** dispatcher, in place. + + The builder and the built handler keep the two processor tables under + the same attribute names, so the same installation logic serves both. + Used when a subscription arrives while a dispatcher is already running. + """ + return self.apply(dispatcher) + + def apply(self, builder: Any) -> Optional[str]: + """Install every subscription onto ``builder``. + + Returns a description of the first failure, or ``None``. Failures are + reported rather than raised: one broken subscription must not stop the + channel from starting. + """ + problem = None + for event_type in list(self._handlers): + try: + self._install(builder, event_type) + except Exception as exc: + detail = "%s: %s" % (event_type, type(exc).__name__) + logger.warning( + "channel: could not install raw subscription for %s (%s)", + sanitize_for_log(event_type), + type(exc).__name__, + ) + problem = problem or detail + return problem + + def _install(self, target: Any, event_type: str) -> None: + """Install one event type's processor onto a builder or a built handler. + + Both shapes keep the two processor tables under the same attribute + names, so the maps are written directly. That also side-steps the + builder's duplicate-key guard, which this needs to do: replaying is + expected to overwrite, and merging every handler for a type into a + single processor is the whole point — registering per handler is what + would trip that guard and take the rebuild down with it. + """ + dispatch = self._dispatcher_for(event_type) + key = "p2.%s" % event_type + callback_map = target._callback_processor_map + event_map = target._processorMap + + if event_type in _CALLBACK_TYPES: + _method, response_name = _CALLBACK_TYPES[event_type] + callback_map[key] = _RawEventProcessor( + inner=_builtin_under(callback_map.get(key)), + dispatch=dispatch, + fallback_response=lambda name=response_name: _empty_response(name), + ) + return + if key in callback_map: + callback_map[key] = _RawEventProcessor( + inner=_builtin_under(callback_map.get(key)), dispatch=dispatch + ) + return + event_map[key] = _RawEventProcessor( + inner=_builtin_under(event_map.get(key)), dispatch=dispatch + ) + + def _dispatcher_for(self, event_type: str) -> Callable[[Any], None]: + def dispatch(data: Any) -> None: + handlers = list(self._handlers.get(event_type) or ()) + if not handlers: + return + payload = _coerce.obj_to_dict(data) or {} + self._schedule(self._run(handlers, payload)) + + return dispatch + + async def _run(self, handlers: List[Callable], payload: Dict[str, Any]) -> None: + for handler in handlers: + try: + result = handler(payload) + if inspect.isawaitable(result): + await result + except Exception as exc: + # A raw handler is application code on an escape hatch; its + # failure must not touch the built-in path. + outcome = self._report(exc) + if inspect.isawaitable(outcome): + await outcome + + +__all__ = ["RawEventRegistry"] diff --git a/lark_channel/channel/tests/test_client_lifecycle.py b/lark_channel/channel/tests/test_client_lifecycle.py index 2d7aba5..378e101 100644 --- a/lark_channel/channel/tests/test_client_lifecycle.py +++ b/lark_channel/channel/tests/test_client_lifecycle.py @@ -6,7 +6,9 @@ """ import asyncio +import gc import threading +import time from unittest.mock import patch import pytest @@ -492,3 +494,203 @@ def test_emit_reject_dispatches_to_registered_handler(): )) assert len(got) == 1 assert got[0].reason == "policy_dm_disabled" + + +def _spin_up_loop(): + """A real loop on its own thread, the way `FeishuChannel` runs one.""" + loop = asyncio.new_event_loop() + thread = threading.Thread(target=loop.run_forever, daemon=True) + thread.start() + return loop, thread + + +def _tear_down_loop(loop, thread): + loop.call_soon_threadsafe(loop.stop) + thread.join(timeout=5) + loop.close() + + +def _pending_sweep_tasks(loop): + """Snapshot the sweep tasks still alive on `loop`, and whether each is spent. + + "Spent" means the coroutine has no frame left — it ran to completion or was + closed. A task that is still pending behind a spent coroutine is the state + asyncio can never produce on its own, and the one that later surfaces as + "Task was destroyed but it is pending". + """ + + async def _snapshot(): + return [ + getattr(task.get_coro(), "cr_frame", None) is None + for task in asyncio.all_tasks() + if getattr(task.get_coro(), "__name__", "") == "_sweep" + ] + + return asyncio.run_coroutine_threadsafe(_snapshot(), loop).result(timeout=5) + + +def test_sweep_finishes_even_when_a_task_refuses_its_cancellation(): + """A task that will not die must not strand the sweep on the loop. + + The bg loop carries tasks that swallow `CancelledError` (a WS receive loop + mid-reconnect, say). If the sweep waits for all of them to converge, it + hangs until the loop is stopped underneath it and then *it* becomes the + task destroyed while pending — the residue it exists to prevent. An + implementation that gathers instead of bounding the wait leaves its own + task on the loop here and must fail this test. + """ + loop, thread = _spin_up_loop() + started = threading.Event() + release = threading.Event() + cancellations = [] + + async def _stubborn(): + started.set() + while True: + try: + await asyncio.sleep(0.05) + except asyncio.CancelledError: + cancellations.append(1) + if release.is_set(): + raise + + stubborn = asyncio.run_coroutine_threadsafe(_stubborn(), loop) + assert started.wait(timeout=5) + + try: + _ChannelClient._sweep_bg_loop_tasks(loop, timeout=0.3) + + assert cancellations, "the sweep never cancelled what was on the loop" + leftover = _pending_sweep_tasks(loop) + assert not leftover, ( + "the sweep's own task is still pending on the loop; it waited for a " + "task that refuses to converge instead of bounding the wait" + ) + finally: + release.set() + stubborn.cancel() + _tear_down_loop(loop, thread) + + +def test_sweep_leaves_nothing_behind_when_the_loop_refuses_the_task(): + """A loop that is already closing refuses `create_task`, mid-callback. + + That happens on the loop's own thread, after the coroutine exists, so it is + the one place the sweep really does have to clean up after itself. Dropping + the coroutine there surfaces later as a never-awaited warning blamed on + unrelated code; `pytest.ini` turns that into an error, so a leak fails this + test. The sweep must also stop rather than try to cancel a task that was + never created. + """ + + class _LoopRefusingTasks: + def __init__(self): + self.calls = 0 + + def is_running(self): + return True + + def create_task(self, coro): + raise RuntimeError("Event loop is closed") + + def call_soon_threadsafe(self, callback, *args): + self.calls += 1 + callback(*args) + + loop = _LoopRefusingTasks() + _ChannelClient._sweep_bg_loop_tasks(loop, timeout=0.05) + assert loop.calls == 1, "nothing should have been cancelled" + gc.collect() + + +def test_sweep_creates_no_coroutine_when_the_loop_will_not_take_it(): + """If the loop closes before scheduling, there must be nothing left behind. + + The coroutine is built on the loop's own thread, so a loop that refuses the + callback never causes one to exist. Were it built up front instead, dropping + it here would surface as a never-awaited warning attributed to unrelated + code — an error under `pytest.ini`, so a leak fails this test. + """ + + class _LoopThatClosesMidCall: + def is_running(self): + return True + + def call_soon_threadsafe(self, *args, **kwargs): + raise RuntimeError("Event loop is closed") + + _ChannelClient._sweep_bg_loop_tasks(_LoopThatClosesMidCall(), timeout=0.1) + gc.collect() + + +def test_sweep_stays_quiet_when_the_loop_closes_before_the_cancel_lands(caplog): + """A loop that dies mid-sweep must not turn shutdown into a stack trace. + + The window is: the sweep was scheduled, the loop then stopped servicing + callbacks, and by the time we go to cancel it the loop is gone — the normal + shape when something else (a failed WS start, say) tore it down. That has to + end quietly. Cancelling through a `concurrent.futures` future cannot satisfy + this: its cancel callback re-enters the closed loop and that module logs the + failure itself, out of reach of any `try` here. + """ + import logging + + class _Task: + def __init__(self): + self.cancelled = False + + def add_done_callback(self, _callback): + pass # never fires: this task does not finish + + def cancel(self): + self.cancelled = True + + class _LoopClosingAfterSchedule: + def __init__(self): + self.task = _Task() + self.calls = 0 + + def is_running(self): + return True + + def create_task(self, coro): + coro.close() # stands in for the loop running it + return self.task + + def call_soon_threadsafe(self, callback, *args): + self.calls += 1 + if self.calls == 1: + callback(*args) + return + raise RuntimeError("Event loop is closed") + + loop = _LoopClosingAfterSchedule() + with caplog.at_level(logging.DEBUG, logger="lark_channel"): + _ChannelClient._sweep_bg_loop_tasks(loop, timeout=0.05) + + assert loop.calls == 2, "the cancel was never attempted" + assert not loop.task.cancelled, "the fake loop was supposed to refuse it" + assert not [r for r in caplog.records if r.levelno >= logging.WARNING] + gc.collect() + + +def test_an_identity_retry_is_not_scheduled_onto_a_stopped_loop(): + """Scheduling onto a stopped-but-open loop silently goes nowhere. + + `run_coroutine_threadsafe` only refuses a *closed* loop; for one that has + merely stopped it queues a callback that never runs. The coroutine handed to + it is then never awaited — surfacing at GC time as an unraisable warning + blamed on whatever test happens to be running — and the future stored here + never completes, so a later retry sees "one is already in flight" and never + starts one. An implementation that schedules anyway must fail this test. + """ + c = _client() + loop = asyncio.new_event_loop() + try: + c._bg_loop = loop # stopped, not closed: the window that leaked + c._start_bot_identity_retry_loop() + assert c._bot_identity_retry_future is None, ( + "a retry was scheduled onto a loop that will never run it" + ) + finally: + loop.close() diff --git a/lark_channel/channel/tests/test_errors_helpers.py b/lark_channel/channel/tests/test_errors_helpers.py index 3344728..d7016f9 100644 --- a/lark_channel/channel/tests/test_errors_helpers.py +++ b/lark_channel/channel/tests/test_errors_helpers.py @@ -12,11 +12,14 @@ ) -def test_feishu_channel_error_code_has_10_canonical_values(): +def test_feishu_channel_error_code_has_13_canonical_values(): canonical = { "format_error", "target_revoked", "rate_limited", "permission_denied", "upload_failed", "download_failed", "ssrf_blocked", "send_timeout", "not_connected", "unknown", + # Appended for the meeting channel; the ten above keep their values, so + # `except`/comparison on any of them is unaffected. + "not_supported", "meeting_not_found", "too_many_sessions", } assert {m.value for m in FeishuChannelErrorCode} == canonical diff --git a/lark_channel/channel/tests/test_security_config.py b/lark_channel/channel/tests/test_security_config.py index 929b7c5..3b3aeff 100644 --- a/lark_channel/channel/tests/test_security_config.py +++ b/lark_channel/channel/tests/test_security_config.py @@ -133,11 +133,16 @@ def test_channel_config_keeps_existing_positional_arguments_stable(): signature = inspect.signature(ChannelConfig) params = list(signature.parameters) - # The bot-at-bot knobs are appended AFTER `security`, so the original - # positional prefix (through `security`, adjacent to `media_cache`) is - # unchanged and existing positional callers still map correctly. + # Every later addition — the bot-at-bot knobs, then the meeting config — + # goes AFTER `security`, so the original positional prefix (through + # `security`, adjacent to `media_cache`) is unchanged and existing + # positional callers still map correctly. assert params.index("security") == params.index("media_cache") + 1 - assert params[-2:] == ["resolve_sender_names", "resolve_chat_members"] + assert params[-3:] == [ + "resolve_sender_names", + "resolve_chat_members", + "meeting", + ] assert params.index("security") > params.index("media_cache") config = ChannelConfig( @@ -175,6 +180,7 @@ def test_channel_config_field_order_is_stable_for_positional_callers(): # Bot-at-bot knobs, appended at the end (positional-compat preserving). "resolve_sender_names", "resolve_chat_members", + "meeting", ] diff --git a/lark_channel/core/cache/expiring_cache.py b/lark_channel/core/cache/expiring_cache.py index 653a7cc..aff07e8 100644 --- a/lark_channel/core/cache/expiring_cache.py +++ b/lark_channel/core/cache/expiring_cache.py @@ -1,23 +1,30 @@ -import asyncio import time from typing import Dict, Tuple, Any class ExpiringCache(object): + """A dict whose entries expire, reclaimed as it is used. + + Reclamation is opportunistic rather than scheduled: ``set`` sweeps expired + entries once ``clear_interval`` has passed since the last sweep, so the + cadence matches a timer's without owning one. + + It used to own one. ``__init__`` created a task for a sweep coroutine, which + meant every instance needed a **running** event loop to ever start it — and + the instance is built in ``__init__`` of its holder, where there may not be + one yet. A loop that never ran left the coroutine unawaited, surfacing at GC + time as "coroutine ... was never awaited" attributed to whatever happened to + be running then, and ``__del__`` cancelling that task raised "Event loop is + closed" during interpreter shutdown. Neither is worth a timer here: the only + caller keeps entries for five seconds, and ``get`` already drops an expired + entry when it reads one, so the sweep exists purely to keep keys that are + never read again from accumulating. + """ def __init__(self, clear_interval=60): self._cache: Dict[str, Tuple[Any, float]] = {} self._clear_interval: int = clear_interval - - try: - loop = asyncio.get_event_loop() - except RuntimeError: - loop = asyncio.new_event_loop() - asyncio.set_event_loop(loop) - self._cron = loop.create_task(self._start_clear_cron()) - - def __del__(self): - self._cron.cancel() + self._last_clear: float = time.time() def get(self, key: str) -> Any: elem = self._cache.get(key) @@ -32,16 +39,18 @@ def get(self, key: str) -> Any: # ttl: time to live, in seconds def set(self, key: str, value: Any, ttl: int): - expire = time.time() + ttl + now = time.time() + # Amortized: the scan is O(n), but it runs at most once per interval, so + # a per-message `set` stays O(1) on average — the cost the timer had. + if now - self._last_clear >= self._clear_interval: + self._clear(now) + expire = now + ttl self._cache[key] = (value, expire) - def _clear(self): - now = time.time() + def _clear(self, now: float = None): + if now is None: + now = time.time() + self._last_clear = now expired_keys = [key for key, (value, expire) in self._cache.items() if expire < now] for key in expired_keys: del self._cache[key] - - async def _start_clear_cron(self): - while True: - await asyncio.sleep(self._clear_interval) - self._clear() diff --git a/lark_channel/core/cache/tests/__init__.py b/lark_channel/core/cache/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/lark_channel/core/cache/tests/test_expiring_cache.py b/lark_channel/core/cache/tests/test_expiring_cache.py new file mode 100644 index 0000000..f011207 --- /dev/null +++ b/lark_channel/core/cache/tests/test_expiring_cache.py @@ -0,0 +1,71 @@ +"""Coverage for `ExpiringCache`, which had none. + +The cache is internal — one caller, `ws.client`, dedups inbound message ids +with it — but it used to own a background task per instance, and that task was +the single largest source of shutdown noise in the suite. +""" + +import asyncio +import gc + +from lark_channel.core.cache.expiring_cache import ExpiringCache + + +def test_a_value_is_readable_until_it_expires(): + cache = ExpiringCache() + cache.set("k", "v", 60) + assert cache.get("k") == "v" + + +def test_an_expired_value_reads_as_absent(): + cache = ExpiringCache() + cache.set("k", "v", -1) # already expired + assert cache.get("k") is None + + +def test_a_missing_key_reads_as_absent(): + assert ExpiringCache().get("nope") is None + + +def test_entries_nobody_reads_again_do_not_accumulate(): + """`get` drops an expired entry when it reads one, so the sweep only matters + for keys that are never read again — which is the common case for message-id + dedup, where a repeat is the exception.""" + cache = ExpiringCache(clear_interval=0) # sweep on every set + for i in range(50): + cache.set("stale-%d" % i, "v", -1) + cache.set("fresh", "v", 60) + + assert cache.get("fresh") == "v" + assert len(cache._cache) == 1, "expired entries were never reclaimed" + + +def test_the_sweep_waits_for_its_interval(): + """The sweep is O(n) and `set` runs per message, so it must not run every + time. Holding off until the interval elapses is what keeps `set` O(1) + amortized — the same cost the timer it replaced had.""" + cache = ExpiringCache(clear_interval=3600) + cache.set("stale", "v", -1) + cache.set("other", "v", 60) + + assert "stale" in cache._cache, "swept on every set; that is O(n) per message" + + +def test_construction_creates_no_task_and_needs_no_running_loop(): + """An instance is built inside its holder's `__init__`, where there may be + no running loop yet. Owning a task there left a coroutine that never ran, + which surfaced at GC time as an unraisable warning attributed to unrelated + code, and cancelling it from `__del__` raised on an already-closed loop. + An implementation that goes back to owning a task fails this test. + """ + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + try: + cache = ExpiringCache(clear_interval=1) + cache.set("k", "v", 60) + assert not asyncio.all_tasks(loop), "the cache scheduled work on the loop" + del cache + gc.collect() # a leaked coroutine would warn here; pytest.ini errors on it + finally: + asyncio.set_event_loop(None) + loop.close() diff --git a/lark_channel/core/const.py b/lark_channel/core/const.py index 09f6a55..b799593 100644 --- a/lark_channel/core/const.py +++ b/lark_channel/core/const.py @@ -1,6 +1,6 @@ # Info PROJECT = "channel-sdk-python" -VERSION = "1.2.0" +VERSION = "1.3.0" # Domain FEISHU_DOMAIN = "https://open.feishu.cn" diff --git a/lark_channel/core/log.py b/lark_channel/core/log.py index 43ceb30..a9fc934 100644 --- a/lark_channel/core/log.py +++ b/lark_channel/core/log.py @@ -28,6 +28,11 @@ "set_cookie", "secret", "password", + # A one-click authorization link is a signed capability — a credential in + # link form. `authorization_url` needs no entry: the `authorization` + # substring above already covers it. `_normalize_key` folds camelCase, so + # this one key covers `consoleUrl` too. + "console_url", ) _SENSITIVE_EXACT_KEYS = { "token", diff --git a/lark_channel/ws/client.py b/lark_channel/ws/client.py index c9148a4..2320434 100644 --- a/lark_channel/ws/client.py +++ b/lark_channel/ws/client.py @@ -206,6 +206,14 @@ def __init__(self, self._reconnect_interval: int = 120 self._ping_interval: int = 120 self._cache: ExpiringCache = ExpiringCache(clear_interval=30) + # On 3.8/3.9 `asyncio.Lock()` and `asyncio.Semaphore()` bind to a loop + # at construction, so building a client from a thread that has none + # raises. This used to be papered over from an unlikely place: the + # cache's constructor installed a loop as a side effect of looking one + # up for its own background task. Removing that task removed the prop, + # so the requirement is stated here, where the primitives that need it + # actually live. On 3.10+ they bind on first use and this is a no-op. + self._ensure_thread_event_loop() self._lock = asyncio.Lock() self._reconnect_lock = asyncio.Lock() self._handler_semaphore = ( @@ -222,6 +230,14 @@ def __init__(self, self.on_reconnected: Callable[[], None] = lambda: None logger.setLevel(log_level.value) + @staticmethod + def _ensure_thread_event_loop() -> None: + """Make sure the calling thread has an event loop to bind to.""" + try: + asyncio.get_event_loop() + except RuntimeError: + asyncio.set_event_loop(asyncio.new_event_loop()) + def start(self) -> None: try: loop.run_until_complete(self._connect()) diff --git a/lark_channel/ws/tests/test_ws_security.py b/lark_channel/ws/tests/test_ws_security.py index 51b878f..220bb9d 100644 --- a/lark_channel/ws/tests/test_ws_security.py +++ b/lark_channel/ws/tests/test_ws_security.py @@ -287,3 +287,28 @@ async def noop_disconnect(*_args, **_kwargs): await task assert started == ["message-1", "message-2"] assert max_active == 1 + + +def test_a_client_can_be_built_from_a_thread_with_no_event_loop(): + """Construction must not require the calling thread to already have a loop. + + On 3.8/3.9 `asyncio.Lock()` and `asyncio.Semaphore()` resolve a loop at + construction, and `__init__` builds three of them. That requirement was met + by accident for a long time: `ExpiringCache.__init__`, the line above, looked + up a loop for its own background task and installed one when there was none. + Nothing in the suite pinned it, so removing that task turned "you may build a + client anywhere" into a `RuntimeError` on 3.9 only — invisible on 3.10+. + """ + previous = None + try: + previous = asyncio.get_event_loop() + except RuntimeError: + pass + asyncio.set_event_loop(None) + try: + client = ws_client.Client(app_id="cli_test", app_secret="secret") + assert client is not None + finally: + # Restore what was there: leaving the thread without a loop would fail + # every later test that builds one of these, which is this bug again. + asyncio.set_event_loop(previous) diff --git a/pytest.ini b/pytest.ini index 3cf089f..f98e036 100644 --- a/pytest.ini +++ b/pytest.ini @@ -2,6 +2,8 @@ asyncio_mode = auto testpaths = lark_channel/channel/tests + lark_channel/channel/meeting/tests + lark_channel/core/cache/tests lark_channel/core/token/tests lark_channel/event/tests lark_channel/ws/tests @@ -13,5 +15,16 @@ python_files = test_*.py # regressions (e.g. asyncio.get_event_loop() being removed in 3.14). filterwarnings = ignore:FileTokenStore is not for production:UserWarning + # A dropped coroutine is a bug, not a warning: it means work that was + # supposed to happen silently did not. This exact class hid a swallowed + # error report through a full green suite. + # + # Both lines are needed. The RuntimeWarning is raised from the coroutine's + # __del__, so turning it into an exception cannot propagate — CPython prints + # "Exception ignored in:" and pytest re-packages it as an unraisable- + # exception warning. Only the second line turns that into a failure. Checked + # by deliberately dropping a coroutine and confirming the run goes red. + error:coroutine .* was never awaited:RuntimeWarning + error::pytest.PytestUnraisableExceptionWarning markers = slow: marks tests that take non-trivial wall-clock time diff --git a/samples/channel/meeting_follow_agenda.py b/samples/channel/meeting_follow_agenda.py new file mode 100644 index 0000000..e971b61 --- /dev/null +++ b/samples/channel/meeting_follow_agenda.py @@ -0,0 +1,76 @@ +"""Follow a meeting under the user's own token, without joining it. + +The bot never appears in the meeting. It reads the meeting a user is already +in, under that user's own authorization, and nudges them over IM when the +discussion drifts off the agenda. + +Needs ``vc:meeting.meetingevent:read`` on a user access token. No socket is +required — this path is REST only, so ``connect()`` is not called. + +**Compliance.** This reads every participant's speech for the whole meeting +while the bot is invisible. Telling participants and getting their consent is +your responsibility. Note also that what the platform grants is whatever your +app applied for, which is usually much broader than meeting reads, and the +ticket is stored per user for the whole process to reuse. +""" + +import asyncio +import os + +from lark_channel import FeishuChannel +from lark_channel.channel.auth import FileTokenStore + + +def _require_env(name): + value = os.environ.get(name) + if not value: + raise SystemExit( + f"Missing {name}. Set it before running, for example: " + f"export {name}=your_value" + ) + return value + + +AGENDA = ["progress update", "risks", "next week"] + + +async def main(): + channel = FeishuChannel( + app_id=_require_env("LARK_APP_ID"), + app_secret=_require_env("LARK_APP_SECRET"), + # Development only — it stores tickets as plaintext JSON. In production + # implement TokenStore against your own secret manager. + token_store=FileTokenStore("./.uat-tickets.json"), + ) + + # Must be the person who asked for this, not a value taken from an inbound + # message: the SDK cannot tell the difference, and a cached ticket resolves + # without notifying its owner. + user_open_id = _require_env("LARK_USER_OPEN_ID") + + session = await channel.follow_my_meeting(user_open_id=user_open_id) + print("following meeting %s (%s)" % (session.meeting_no, session.mode)) + + recent = [] + + def on_transcript(event): + if event.self_echo: + return + recent.append("%s: %s" % (event.actor.name, event.text)) + del recent[:-200] + + session.on("transcript", on_transcript) + session.on("end", lambda e: print("session over: %s" % e.reason)) + + try: + while True: + await asyncio.sleep(60) + if not recent: + continue + print("agenda %s / last %d lines" % (AGENDA, len(recent))) + finally: + await session.leave() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/samples/channel/meeting_join_bot.py b/samples/channel/meeting_join_bot.py new file mode 100644 index 0000000..cbdcda5 --- /dev/null +++ b/samples/channel/meeting_join_bot.py @@ -0,0 +1,82 @@ +"""The bot joins a meeting as a real participant, under the app's own token. + +Run it, then add the bot to a meeting from the Feishu client. It answers +questions typed into the meeting chat and keeps a running transcript. + +Needs ``vc:meeting.bot.join:write`` and ``vc:meeting.message:write``, the three +``vc.bot.*`` event subscriptions declared in the developer console, and the +meeting's "allow agents to join" setting turned on. +""" + +import asyncio +import os + +from lark_channel import FeishuChannel + + +def _require_env(name): + value = os.environ.get(name) + if not value: + raise SystemExit( + f"Missing {name}. Set it before running, for example: " + f"export {name}=your_value" + ) + return value + + +async def main(): + channel = FeishuChannel( + app_id=_require_env("LARK_APP_ID"), + app_secret=_require_env("LARK_APP_SECRET"), + ) + + async def on_invited(invitation): + # `meetingInvited` is the only way into a joined meeting, and it does + # not pass through the message policy: anybody who can add the bot to a + # meeting can trigger this. Set `meeting.invite_allowlist` if only + # certain people should be able to. + session = await channel.join_meeting(invitation.meeting_no) + transcript = {} + + def on_transcript(event): + if event.self_echo: + # Our own speech, transcribed back to us. Keeping it here is + # deliberate — a full record wants the bot's turns too. + return + # A sentence id is an upsert handle, not a unique key: the platform + # resends the same sentence as the speaker keeps talking. + transcript[event.sentence_id] = event.text + + async def on_chat(event): + if event.self_echo: + # Without this the reply below arrives back as meeting chat and + # the bot answers itself at network speed. + return + if not event.content.startswith("@assistant"): + return + question = event.content[len("@assistant"):].strip() + answer = "%d sentences so far. You asked: %s" % ( + len(transcript), + question, + ) + await session.send_message(answer) + + session.on("transcript", on_transcript) + session.on("chat", on_chat) + session.on("participant", lambda e: print("%s %s" % (e.actor.name, e.action))) + session.on("end", lambda e: print("meeting over: %s" % e.reason)) + # A slow handler holds up this meeting's stream, because delivery is + # ordered. A handler that blocks *without* awaiting holds up the whole + # process — hand blocking work to an executor. + + channel.on("meetingInvited", on_invited) + try: + await channel.connect() + finally: + # `dispose()` does not leave a meeting, so a plain shutdown leaves the + # bot sitting in every meeting it joined until the server ends them. + await channel.disconnect() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/tests/bridge/test_media_cache.py b/tests/bridge/test_media_cache.py index 902e6c3..c7d4dee 100644 --- a/tests/bridge/test_media_cache.py +++ b/tests/bridge/test_media_cache.py @@ -18,8 +18,13 @@ def test_media_cache_config_is_appended_for_positional_compatibility(): # media_cache/security keep their positions; the bot-at-bot knobs # (resolve_sender_names / resolve_chat_members) were appended after them, - # so existing positional callers are unaffected. - assert names[-2:] == ["resolve_sender_names", "resolve_chat_members"] + # and the meeting config after those, so existing positional callers are + # unaffected — every addition goes on the end. + assert names[-3:] == [ + "resolve_sender_names", + "resolve_chat_members", + "meeting", + ] assert names[names.index("security") - 1] == "media_cache" assert names[names.index("chat_mode_cache") + 1] == "policy" diff --git a/tests/package_identity/test_import_identity.py b/tests/package_identity/test_import_identity.py index 898c1d9..69d9472 100644 --- a/tests/package_identity/test_import_identity.py +++ b/tests/package_identity/test_import_identity.py @@ -29,7 +29,7 @@ def test_transport_keepalive_config_imports_from_package_root(): assert KeepaliveConfig is ChannelKeepaliveConfig -def test_release_version_is_1_2_0(): +def test_release_version_is_1_3_0(): from lark_channel.core.const import VERSION - assert VERSION == "1.2.0" + assert VERSION == "1.3.0" diff --git a/tests/runtime/api_file_allowlist.txt b/tests/runtime/api_file_allowlist.txt index 41b52cf..c14c795 100644 --- a/tests/runtime/api_file_allowlist.txt +++ b/tests/runtime/api_file_allowlist.txt @@ -245,3 +245,5 @@ lark_channel/api/im/v1/resource/message_resource.py lark_channel/api/im/v1/version.py lark_channel/api/wiki/__init__.py lark_channel/api/wiki/node.py +lark_channel/api/vc/__init__.py +lark_channel/api/vc/bot.py diff --git a/tests/runtime/test_api_allowlist.py b/tests/runtime/test_api_allowlist.py index a249b49..f317c91 100644 --- a/tests/runtime/test_api_allowlist.py +++ b/tests/runtime/test_api_allowlist.py @@ -8,7 +8,7 @@ ROOT = Path(__file__).resolve().parents[2] -ALLOWED_API_ROOTS = {"im", "contact", "cardkit", "drive", "wiki"} +ALLOWED_API_ROOTS = {"im", "contact", "cardkit", "drive", "wiki", "vc"} DIRECT_API_MODULES = { "lark_channel.api.cardkit.v1.model.content_card_element_request", @@ -19,6 +19,7 @@ "lark_channel.api.cardkit.v1.model.settings_card_request_body", "lark_channel.api.contact.v3.model.batch_user_request", "lark_channel.api.drive.comment", + "lark_channel.api.vc.bot", "lark_channel.api.im.v1.model.create_file_request", "lark_channel.api.im.v1.model.create_file_request_body", "lark_channel.api.im.v1.model.create_file_response", @@ -73,7 +74,7 @@ def test_required_direct_api_modules_import(): def test_non_channel_api_roots_are_not_packaged(): api_path = Path(api_root.__file__).resolve().parent - for name in ("calendar", "bitable", "drive_full", "docx", "vc", "admin"): + for name in ("calendar", "bitable", "drive_full", "docx", "admin"): assert not (api_path / name).exists(), name