Repository navigation
feat(calls): アクティビティとして絵チャを追加 #34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mattyatea
wants to merge
8
commits into
codex/calls-activities
Choose a base branch
from
codex/t07-drawing-chat
base: codex/calls-activities
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
1daab57
feat: Callsベースの絵チャを追加
mattyatea 6ff770c
fix: 絵チャの通信復旧と閲覧状態を維持
mattyatea b9891c6
fix: 絵チャの状態ずれと入力エラーの表示を修正
mattyatea 842a13c
fix: 絵チャの閲覧権限と画面ライフサイクルを維持
mattyatea b7e00f2
chore: 絵チャに最新developを取り込みSDKを再生成
mattyatea 48daa01
feat(calls): 最新developを取り込みルーム内アクティビティに絵チャを追加
mattyatea eee7ed4
feat(calls): アクティビティの土台を絵チャから分離
mattyatea ddd38c7
feat(calls): 独立したActivityの土台へ絵チャを接続
mattyatea File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: syuilo and misskey-project | ||
| * SPDX-License-Identifier: AGPL-3.0-only | ||
| */ | ||
|
|
||
| import { randomUUID } from 'node:crypto'; | ||
| import { Inject, Injectable } from '@nestjs/common'; | ||
| import * as Redis from 'ioredis'; | ||
| import { DI } from '@/di-symbols.js'; | ||
| import type { CallsParticipantsRepository, MiUser } from '@/models/_.js'; | ||
| import { CallsRoomService } from '@/core/calls/CallsRoomService.js'; | ||
| import { CallsLiveConnectionService } from '@/core/calls/CallsLiveConnectionService.js'; | ||
| import { ChatService } from '@/core/ChatService.js'; | ||
| import { GlobalEventService } from '@/core/GlobalEventService.js'; | ||
|
|
||
| export type DrawingScope = 'public' | 'chatRoom' | 'calls'; | ||
| export type DrawingStroke = { color: string; width: number; eraser: boolean; points: number[][] }; | ||
| export type DrawingMessage = { userId: string; text: string; id: string }; | ||
| export type DrawingControl = { | ||
| canvasId: string; | ||
| version: number; | ||
| scope: DrawingScope; | ||
| ended: boolean; | ||
| participantIds: string[]; | ||
| removedIds: string[]; | ||
| lastSeen: Record<string, number>; | ||
| }; | ||
| export type DrawingSnapshot = Omit<DrawingControl, 'removedIds' | 'lastSeen'> & { strokes: DrawingStroke[]; messages: DrawingMessage[] }; | ||
| export type DrawingEvent = Omit<DrawingControl, 'removedIds' | 'lastSeen'> & { action: string; stroke: DrawingStroke | null; message: DrawingMessage | null }; | ||
| export type DrawingAction = 'start' | 'join' | 'leave' | 'stroke' | 'clear' | 'end' | 'kick' | 'message'; | ||
| export class DrawingError extends Error { | ||
| constructor(public readonly code: 'accessDenied' | 'invalidState' | 'roomFull' | 'canvasFull') { super(code); } | ||
| } | ||
|
|
||
| @Injectable() | ||
| export class DrawingService { | ||
| constructor( | ||
| @Inject(DI.redis) private redis: Redis.Redis, | ||
| @Inject(DI.callsParticipantsRepository) private callsParticipants: CallsParticipantsRepository, | ||
| private callsRoomService: CallsRoomService, | ||
| private liveConnections: CallsLiveConnectionService, | ||
| private chatService: ChatService, | ||
| private events: GlobalEventService, | ||
| ) {} | ||
|
|
||
| private key(roomId: string) { return `drawing:${roomId}`; } | ||
|
|
||
| private async control(roomId: string): Promise<DrawingControl | null> { | ||
| const raw = await this.redis.get(this.key(roomId)); | ||
| return raw == null ? null : JSON.parse(raw) as DrawingControl; | ||
| } | ||
|
|
||
| public async assertAccess(user: MiUser, roomId: string, control?: DrawingControl | null, write = false) { | ||
| const room = await this.callsRoomService.getRoom(roomId); | ||
| await this.callsRoomService.assertCanAccess(user, room); | ||
| const availability = await this.chatService.getChatAvailability(user.id); | ||
| if (!availability[write ? 'write' : 'read']) throw new DrawingError('accessDenied'); | ||
| const state = control === undefined ? await this.control(roomId) : control; | ||
| if (state?.removedIds.includes(user.id)) throw new DrawingError('accessDenied'); | ||
| const participant = await this.callsParticipants.findOneBy({ roomId, userId: user.id }); | ||
| if (participant?.state === 'removed') throw new DrawingError('accessDenied'); | ||
| if (state?.scope === 'calls') { | ||
| if (participant?.state !== 'active' || await this.liveConnections.get(participant.id) == null) throw new DrawingError('accessDenied'); | ||
| } | ||
| // Calls moderators may inspect a ChatRoom, but drawing is limited to its members. | ||
| if (state?.scope === 'chatRoom') { | ||
| const chatRoom = room.chatRoomId == null ? null : await this.chatService.findRoomById(room.chatRoomId); | ||
| if (chatRoom == null || !await this.chatService.isRoomMember(chatRoom, user.id)) throw new DrawingError('accessDenied'); | ||
| } | ||
| return room; | ||
| } | ||
|
|
||
| public async snapshot(user: MiUser, roomId: string): Promise<DrawingSnapshot | null> { | ||
| return this.liveConnections.withRoomLock(roomId, async () => { | ||
| const control = await this.control(roomId); | ||
| await this.assertAccess(user, roomId, control); | ||
| if (control == null) return null; | ||
| const { removedIds, lastSeen, ...visible } = control; | ||
| const [strokes, messages] = await Promise.all([ | ||
| this.redis.lrange(`${this.key(roomId)}:strokes`, 0, -1), | ||
| this.redis.lrange(`${this.key(roomId)}:messages`, 0, -1), | ||
| ]); | ||
| const room = await this.callsRoomService.getRoom(roomId); | ||
| return { ...visible, ended: visible.ended || room.state !== 'open', participantIds: visible.participantIds.filter(id => (control.lastSeen[id] ?? 0) > Date.now() - 90000), strokes: strokes.map(x => JSON.parse(x) as DrawingStroke), messages: messages.map(x => JSON.parse(x) as DrawingMessage) }; | ||
| }); | ||
| } | ||
|
|
||
| public async update(user: MiUser, roomId: string, action: DrawingAction, params: { canvasId?: string; scope?: DrawingScope; userId?: string; stroke?: DrawingStroke; text?: string }): Promise<void> { | ||
| await this.liveConnections.withRoomLock(roomId, async assertHeld => { | ||
| let control = await this.control(roomId); | ||
| const room = await this.assertAccess(user, roomId, control, true); | ||
| if (room.state !== 'open') throw new DrawingError('invalidState'); | ||
| const ownerAction = ['start', 'clear', 'end', 'kick'].includes(action); | ||
| if (ownerAction && room.ownerUserId !== user.id) throw new DrawingError('accessDenied'); | ||
| const transaction = this.redis.multi(); | ||
| const key = this.key(roomId); | ||
| if (action === 'start') { | ||
| if (control != null || params.scope == null) throw new DrawingError('invalidState'); | ||
| if (params.scope === 'public' && (room.attachmentType !== 'personal' || room.visibility !== 'public')) throw new DrawingError('accessDenied'); | ||
| if (params.scope === 'chatRoom' && room.chatRoomId == null) throw new DrawingError('accessDenied'); | ||
| control = { canvasId: randomUUID(), version: 0, scope: params.scope, ended: false, participantIds: [user.id], removedIds: [], lastSeen: { [user.id]: Date.now() } }; | ||
| await this.assertAccess(user, roomId, control, true); | ||
| } else { | ||
| if (control == null || control.ended || control.canvasId !== params.canvasId) throw new DrawingError('invalidState'); | ||
| control.participantIds = control.participantIds.filter(id => (control!.lastSeen[id] ?? 0) > Date.now() - 90000); | ||
| if (action === 'join') { | ||
| control.lastSeen[user.id] = Date.now(); | ||
| if (!control.participantIds.includes(user.id)) { | ||
| if (control.participantIds.length >= 16) throw new DrawingError('roomFull'); | ||
| control.participantIds.push(user.id); | ||
| } | ||
| } else if (action === 'leave') { | ||
| control.participantIds = control.participantIds.filter(id => id !== user.id); | ||
| } else if (action === 'kick') { | ||
| if (params.userId == null || params.userId === room.ownerUserId || !control.participantIds.includes(params.userId)) throw new DrawingError('invalidState'); | ||
| control.participantIds = control.participantIds.filter(id => id !== params.userId); | ||
| control.removedIds.push(params.userId); | ||
| } else if (action === 'clear') { | ||
| control.canvasId = randomUUID(); | ||
| transaction.del(`${key}:strokes`); | ||
| } else if (action === 'end') { | ||
| control.ended = true; | ||
| } else { | ||
| if (!control.participantIds.includes(user.id)) throw new DrawingError('accessDenied'); | ||
| if (action === 'stroke') { | ||
| if (params.stroke == null) throw new DrawingError('invalidState'); | ||
| if (await this.redis.llen(`${key}:strokes`) >= 10000) throw new DrawingError('canvasFull'); | ||
| transaction.rpush(`${key}:strokes`, JSON.stringify(params.stroke)); | ||
| } else if (action === 'message') { | ||
| if (params.text == null || params.text.trim().length === 0) throw new DrawingError('invalidState'); | ||
| } | ||
| } | ||
| } | ||
| const message = action === 'message' ? { userId: user.id, text: params.text!.trim(), id: randomUUID() } : null; | ||
| if (message != null) transaction.rpush(`${key}:messages`, JSON.stringify(message)).ltrim(`${key}:messages`, -50, -1); | ||
| control.version++; | ||
| await assertHeld(); | ||
| transaction.set(key, JSON.stringify(control)); | ||
| const result = await transaction.exec(); | ||
| const error = result?.find(([err]) => err != null)?.[0]; | ||
| if (error != null) throw error; | ||
| // Drawing without audio still counts as the host being present in the Calls room. | ||
| if (room.ownerUserId === user.id && (action === 'start' || action === 'join')) await this.liveConnections.touchHost(roomId); | ||
| const { removedIds, lastSeen, ...visible } = control; | ||
| this.events.publishDrawingStream(roomId, { ...visible, action, stroke: action === 'stroke' ? params.stroke! : null, message }); | ||
| }); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
packages/backend/src/server/api/endpoints/drawing/_shared.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: syuilo and misskey-project | ||
| * SPDX-License-Identifier: AGPL-3.0-only | ||
| */ | ||
|
|
||
| import { DrawingError } from '@/core/drawing/DrawingService.js'; | ||
| import { ApiError } from '@/server/api/error.js'; | ||
| import { callsApiError, callsErrors } from '../calls/_shared.js'; | ||
|
|
||
| export const drawingErrors = { | ||
| ...callsErrors, | ||
| drawingAccessDenied: { message: 'You cannot access this canvas.', code: 'DRAWING_ACCESS_DENIED', id: '3e6830dc-0b80-41fe-84d3-dc7d4fe0d725' }, | ||
| drawingInvalidState: { message: 'The canvas is not in the required state.', code: 'DRAWING_INVALID_STATE', id: '175cdb1a-5156-472b-8387-a94f64e2cc12' }, | ||
| drawingRoomFull: { message: 'The canvas has reached its participant limit.', code: 'DRAWING_ROOM_FULL', id: '5489fcf0-205d-4800-b460-4bba2929da29' }, | ||
| drawingCanvasFull: { message: 'Export or clear the canvas before continuing.', code: 'DRAWING_CANVAS_FULL', id: 'f47a90a1-766c-4c4c-a7c3-78ace3ca86ed' }, | ||
| } as const; | ||
|
|
||
| export function drawingApiError(error: unknown): never { | ||
| if (error instanceof DrawingError) { | ||
| const definition = { | ||
| accessDenied: drawingErrors.drawingAccessDenied, | ||
| invalidState: drawingErrors.drawingInvalidState, | ||
| roomFull: drawingErrors.drawingRoomFull, | ||
| canvasFull: drawingErrors.drawingCanvasFull, | ||
| }[error.code]; | ||
| throw new ApiError(definition); | ||
| } | ||
| return callsApiError(error); | ||
| } | ||
|
|
||
| export const strokeSchema = { | ||
| type: 'object', optional: false, nullable: false, | ||
| properties: { | ||
| color: { type: 'string', optional: false, nullable: false, pattern: '^#[0-9a-fA-F]{6}$' }, | ||
| width: { type: 'number', optional: false, nullable: false, minimum: 1, maximum: 64 }, | ||
| eraser: { type: 'boolean', optional: false, nullable: false }, | ||
| points: { type: 'array', optional: false, nullable: false, minItems: 1, maxItems: 64, items: { | ||
| type: 'array', optional: false, nullable: false, minItems: 2, maxItems: 2, | ||
| items: { type: 'number', optional: false, nullable: false, minimum: 0, maximum: 1280 }, | ||
| } }, | ||
| }, required: ['color', 'width', 'eraser', 'points'], additionalProperties: false, | ||
| } as const; | ||
|
|
||
| export const messageSchema = { type: 'object', optional: false, nullable: false, properties: { | ||
| id: { type: 'string', optional: false, nullable: false }, | ||
| userId: { type: 'string', optional: false, nullable: false }, | ||
| text: { type: 'string', optional: false, nullable: false }, | ||
| } } as const; | ||
|
|
||
| export const snapshotSchema = { | ||
| type: 'object', optional: false, nullable: true, | ||
| properties: { | ||
| canvasId: { type: 'string', optional: false, nullable: false }, | ||
| version: { type: 'number', optional: false, nullable: false }, | ||
| scope: { type: 'string', optional: false, nullable: false, enum: ['public', 'chatRoom', 'calls'] }, | ||
| ended: { type: 'boolean', optional: false, nullable: false }, | ||
| participantIds: { type: 'array', optional: false, nullable: false, items: { type: 'string', optional: false, nullable: false } }, | ||
| strokes: { type: 'array', optional: false, nullable: false, items: strokeSchema }, | ||
| messages: { type: 'array', optional: false, nullable: false, items: messageSchema }, | ||
| }, | ||
| } as const; |
24 changes: 24 additions & 0 deletions
24
packages/backend/src/server/api/endpoints/drawing/clear.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: syuilo and misskey-project | ||
| * SPDX-License-Identifier: AGPL-3.0-only | ||
| */ | ||
|
|
||
| import { Injectable } from '@nestjs/common'; | ||
| import { DrawingService } from '@/core/drawing/DrawingService.js'; | ||
| import { Endpoint } from '@/server/api/endpoint-base.js'; | ||
| import { drawingApiError, drawingErrors } from './_shared.js'; | ||
|
|
||
| export const meta = { | ||
| tags: ['calls'], requireCredential: true, prohibitMoved: true, kind: 'write:calls', | ||
| limit: { duration: 1000 * 60, max: 60 }, errors: drawingErrors, | ||
| } as const; | ||
| export const paramDef = { type: 'object', properties: { roomId: { type: 'string', format: 'misskey:id' }, canvasId: { type: 'string', pattern: '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' } }, required: ['roomId', 'canvasId'] } as const; | ||
|
|
||
| @Injectable() | ||
| export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export | ||
| constructor(drawingService: DrawingService) { | ||
| super(meta, paramDef, async (ps, me) => { | ||
| try { await drawingService.update(me, ps.roomId, 'clear', { canvasId: ps.canvasId }); } catch (error) { drawingApiError(error); } | ||
| }); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| /* | ||
| * SPDX-FileCopyrightText: syuilo and misskey-project | ||
| * SPDX-License-Identifier: AGPL-3.0-only | ||
| */ | ||
|
|
||
| import { Injectable } from '@nestjs/common'; | ||
| import { DrawingService } from '@/core/drawing/DrawingService.js'; | ||
| import { Endpoint } from '@/server/api/endpoint-base.js'; | ||
| import { drawingApiError, drawingErrors } from './_shared.js'; | ||
|
|
||
| export const meta = { | ||
| tags: ['calls'], requireCredential: true, prohibitMoved: true, kind: 'write:calls', | ||
| limit: { duration: 1000 * 60, max: 60 }, errors: drawingErrors, | ||
| } as const; | ||
| export const paramDef = { type: 'object', properties: { roomId: { type: 'string', format: 'misskey:id' }, canvasId: { type: 'string', pattern: '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' } }, required: ['roomId', 'canvasId'] } as const; | ||
|
|
||
| @Injectable() | ||
| export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export | ||
| constructor(drawingService: DrawingService) { | ||
| super(meta, paramDef, async (ps, me) => { | ||
| try { await drawingService.update(me, ps.roomId, 'end', { canvasId: ps.canvasId }); } catch (error) { drawingApiError(error); } | ||
| }); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
Repository: Type4ny-Project/Misskey
Length of output: 7093
🏁 Script executed:
Repository: Type4ny-Project/Misskey
Length of output: 42708
🏁 Script executed:
Repository: Type4ny-Project/Misskey
Length of output: 7200
🏁 Script executed:
Repository: Type4ny-Project/Misskey
Length of output: 4614
描画処理をCallsのルームロックから分離してください
DrawingService.snapshotとupdateは、Calls 操作と同じルームロックを取得します。updateはロック内でアクセス検査、Redis の読み書き、Calls ルーム情報の確認を実行します。drawing/strokeは毎分600回まで呼び出せます。描画要求が集中すると、参加・退出やホストタイムアウト処理がロック待ちになり、約2秒の再試行後にTimed out while locking a Calls roomで失敗することがあります。描画データの直列化には専用のロックキーを使用し、Calls の参加者状態やホスト期限更新と必要な同期を維持する構成をご検討ください。
snapshotのロックを外す場合も、制御情報とストロークの一貫性を別の方法で保証してください。🤖 Prompt for AI Agents