From bdebc5094dfcc3258e27153bd296442451bb88f1 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 14:16:46 +0000 Subject: [PATCH 1/3] Initial plan From cc60f08b4693339399d85dcb92841c3d506db1cc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 14:29:47 +0000 Subject: [PATCH 2/3] Complete HLD for Automated Phone Calls feature Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-automated-phone-calls-summary.md | 202 ++++ docs/HLD-automated-phone-calls.md | 1102 +++++++++++++++++++++ 2 files changed, 1304 insertions(+) create mode 100644 docs/HLD-automated-phone-calls-summary.md create mode 100644 docs/HLD-automated-phone-calls.md diff --git a/docs/HLD-automated-phone-calls-summary.md b/docs/HLD-automated-phone-calls-summary.md new file mode 100644 index 00000000..722f32e8 --- /dev/null +++ b/docs/HLD-automated-phone-calls-summary.md @@ -0,0 +1,202 @@ +# Automated Phone Calls HLD - Executive Summary + +## Overview +This document provides a high-level design for implementing automated outbound phone calling functionality integrated with the Yemot telephony platform. + +## Key Features +1. **Phone Message Template Management** - Create reusable message templates with TTS or audio files +2. **Bulk Triggering** - Send calls to multiple students from existing UI lists +3. **Call Tracking** - Monitor call status and history +4. **User-Specific Configuration** - Per-organization message customization + +## Architecture Summary + +### Database Entities +- `phone_messages` - Stores message templates (TTS text or audio file paths) +- `phone_call_history` - Tracks individual call attempts and results + +### Backend Components +- **PhoneMessage Entity** - Manages message templates +- **PhoneCallService** - Orchestrates call initiation and tracking +- **YemotApiService** - Integrates with Yemot API (ApiOutCallV2) + +### Frontend Components +- **Phone Message CRUD** - UI for managing templates (`/client/src/entities/phone-message.jsx`) +- **BulkPhoneCallButton** - Bulk action button for entity lists +- **Call History Viewer** - Display call logs and status + +## Critical Design Questions Requiring Your Input + +### Q1: Variable Substitution +**Should messages support placeholders like `{studentName}`, `{className}`?** +- ✅ **RECOMMENDED**: Yes - More flexible and personalized +- Alternative: Fixed messages only - Simpler but less useful + +### Q2: Message Ownership Pattern +**Should messages follow the Text/TextByUser pattern?** +- ✅ **RECOMMENDED**: Hybrid (system defaults + user overrides, userId=0 for system) +- Alternative: Pure user-specific (userId always set) + +### Q5: Yemot API Endpoint +**Which API should we use?** +- ✅ **RECOMMENDED**: ApiOutCallV2 (modern, feature-rich) +- Alternative: ApiOutCall (legacy, broader compatibility) + +### Q6: Audio File Hosting +**Where should audio files be stored?** +- ✅ **RECOMMENDED for Phase 1**: Upload to Yemot storage (ApiUploadFile) +- Alternative: Host on application server with public URL +- Future: Cloud storage (S3/GCS) if scale demands + +### Q7: Yemot Token Management +**How should authentication tokens be handled?** +- ✅ **RECOMMENDED for MVP**: Single system-wide token (env variable) +- **RECOMMENDED for Production**: User-specific tokens (stored encrypted in User entity) + +### Q8: Phone Number Validation +**Should we validate phone numbers before calling?** +- ✅ **RECOMMENDED for MVP**: Accept any format, let Yemot validate +- Alternative: Validate Israeli format (regex pattern) +- Future: Use libphonenumber-js for international support + +### Q10: Bulk Operations Processing +**How should large bulk calls be handled?** +- ✅ **RECOMMENDED**: Asynchronous with job queue (Bull/BullMQ) +- Alternative: Synchronous (risk of timeout) + +### Q11: Yemot Account Setup ⚠️ CRITICAL +**Required Information:** +- Do we have an existing Yemot account with API access? +- What is the account tier and rate limits? +- Do we need separate accounts for staging/production? +- What is the authentication token? + +### Q12: User Workflow Priorities +**Which lists should have bulk call buttons?** +- ✅ Student Klasses Report (specified in requirements) +- Student List? +- Teacher List (staff notifications)? + +### Q13: Message Content Validation +- Should we validate content (max length, forbidden words)? +- Should admins preview/test messages before bulk use? + +### Q14: Billing & Cost Control +- How to track Yemot call costs? +- Should we implement quota limits per user? +- Approval required for large operations (>100 calls)? + +## Implementation Phases + +### Phase 1: MVP (2-3 weeks) +- Create database entities (PhoneMessage, PhoneCallHistory) +- Implement PhoneCallService and YemotApiService +- Build phone message CRUD UI +- Add BulkPhoneCallButton to student-klasses-report +- Basic call history viewer +- **TTS only** (no audio files yet) + +### Phase 2: Enhanced (1-2 weeks) +- Audio file upload support +- Variable substitution +- Retry logic for failed calls +- User-specific token management + +### Phase 3: Advanced (2-3 weeks) +- Scheduled calls (job queue) +- Opt-out management +- Enhanced analytics +- CSV export + +### Phase 4: Production Hardening (1 week) +- Performance optimization +- Monitoring/alerting +- Security audit +- Load testing + +## Integration Points + +### Existing Entities Modified +1. **Student Klasses Report** (`/client/src/entities/student-klasses-report.jsx`) + - Add `BulkPhoneCallButton` to `additionalBulkButtons` + +2. **App.jsx** (`/client/src/App.jsx`) + - Register `phone_message` resource (settings menu) + - Register `phone_call_history` resource (admin menu) + +3. **Student Entity** (optional) + - Consider adding `parentPhone` field + - Consider adding `phoneCallOptOut` flag + +### New Files Created +**Backend:** +- `/server/src/db/entities/PhoneMessage.entity.ts` +- `/server/src/db/entities/PhoneCallHistory.entity.ts` +- `/server/src/entity-modules/phone-message.config.ts` +- `/server/src/entity-modules/phone-call-history.config.ts` +- `/server/src/services/phone-call.service.ts` +- `/server/shared/utils/yemot/yemot-api.service.ts` + +**Frontend:** +- `/client/src/entities/phone-message.jsx` +- `/client/src/entities/phone-call-history.jsx` +- `/client/src/components/bulk-actions/BulkPhoneCallButton.jsx` + +## Security Considerations +- **Rate Limiting**: Prevent abuse (10 templates/hour, 100 calls/batch) +- **Phone Masking**: Log masked numbers only (05X-XXX-1234) +- **Token Security**: Encrypt Yemot tokens at rest +- **Permissions**: Admin-only access +- **Audit Trail**: Log all call actions + +## Performance Considerations +- **Bulk Operations**: Use job queue (Bull) for async processing +- **Database Indexing**: Index userId, status, createdAt on history table +- **Caching**: Cache active templates (60s TTL) + +## Testing Strategy +- **Unit Tests**: Service methods, API client, validation +- **Integration Tests**: End-to-end flow with Yemot sandbox +- **Manual Tests**: Real Yemot account with test phone numbers + +## Success Metrics +- **Performance**: Bulk calls queued <2s, processed <5min +- **Reliability**: 95%+ success rate (excluding invalid numbers) +- **Adoption**: 50%+ admins use within first month +- **Time Savings**: 80% reduction in manual calling time + +## Risks & Mitigation + +| Risk | Mitigation | +|------|------------| +| Yemot API changes | Version API calls, monitor docs | +| Rate limiting | Implement queue, respect limits | +| Invalid numbers cost money | Validate before calling, opt-out list | +| User errors in bulk ops | Confirmation dialogs, preview | + +## Next Steps (Action Items) + +1. **Immediate**: Answer critical questions (Q11-Q14) +2. **Week 1**: Obtain Yemot credentials, set up sandbox +3. **Week 1-2**: Implement Phase 1 MVP backend +4. **Week 2-3**: Implement Phase 1 MVP frontend +5. **Week 3**: Integration testing +6. **Week 4+**: Phase 2 enhancements + +## Questions for You + +Please review the full HLD document (`docs/HLD-automated-phone-calls.md`) and provide answers to: + +1. **Q11**: Yemot account details and credentials +2. **Q12**: Which entity lists need bulk call buttons +3. **Q13**: Message content validation requirements +4. **Q14**: Billing and cost control preferences +5. **General**: Any concerns or modifications to the proposed design + +Once these are answered, we can proceed with implementation! + +--- + +**Full HLD Document**: `/docs/HLD-automated-phone-calls.md` +**Created**: 2026-01-15 +**Status**: Awaiting stakeholder review diff --git a/docs/HLD-automated-phone-calls.md b/docs/HLD-automated-phone-calls.md new file mode 100644 index 00000000..55004200 --- /dev/null +++ b/docs/HLD-automated-phone-calls.md @@ -0,0 +1,1102 @@ +# High-Level Design: Automated Phone Calls Feature + +## Executive Summary + +This document outlines the design for an automated outbound phone calling system integrated with the Yemot telephony platform. The system will allow administrators to create reusable phone message templates and trigger automated calls to students' phone numbers through bulk actions in the existing UI. + +**Version:** 1.0 +**Date:** 2026-01-15 +**Status:** Draft for Review + +--- + +## 1. Feature Overview + +### 1.1 Business Goals +- Enable automated communication with students/parents via phone calls +- Reduce manual effort in sending routine notifications +- Provide a flexible, template-based system for various message types +- Integrate seamlessly with existing educational workflows + +### 1.2 Key Capabilities +1. **Message Template Management**: Create, edit, and manage phone message templates +2. **Flexible Message Content**: Support both text-to-speech (TTS) and pre-recorded audio files +3. **Bulk Triggering**: Initiate calls to multiple recipients from entity lists (e.g., student-klasses-report) +4. **Call Tracking**: Monitor call status and results +5. **User-Specific Configuration**: Allow each organization to customize messages + +--- + +## 2. Terminology + +### 2.1 Core Terms + +**Proposed Term: "Phone Message Template" (or "Call Template")** + +**Alternative Options:** +1. **"Phone Message Template"** ✅ RECOMMENDED + - **Pros**: Clear, descriptive, aligns with existing "Text" entity pattern + - **Cons**: Slightly verbose + - **Use Case**: Best for UI labels and user-facing documentation + +2. **"Call Template"** + - **Pros**: Concise, developer-friendly + - **Cons**: May be ambiguous (template for what?) + - **Use Case**: Good for code/API naming + +3. **"Voice Message"** + - **Pros**: User-friendly, intuitive + - **Cons**: Doesn't convey the template/reusable nature + - **Use Case**: Good for non-technical users + +4. **"Phone Notification Template"** + - **Pros**: Emphasizes notification purpose + - **Cons**: Too long for frequent use + - **Use Case**: Good for formal documentation + +**Decision Required**: Which terminology should we use for: +- Entity name in database? +- UI labels? +- API endpoints? + +**Recommendation**: Use "Phone Message Template" for UI, "phone_message" for database entities, and "phoneMessage" in code. + +--- + +## 3. System Architecture + +### 3.1 Component Overview + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Client Layer │ +│ ┌──────────────────┐ ┌──────────────────────────────┐ │ +│ │ Phone Message │ │ Bulk Action Buttons │ │ +│ │ CRUD UI │ │ (Student List, etc.) │ │ +│ └──────────────────┘ └──────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Server Layer │ +│ ┌──────────────────┐ ┌──────────────────────────────┐ │ +│ │ Phone Message │ │ Phone Call Service │ │ +│ │ Entity & Config │ │ (Orchestration) │ │ +│ └──────────────────┘ └──────────────────────────────┘ │ +│ │ │ +│ ┌────────┴────────┐ │ +│ ▼ ▼ │ +│ ┌──────────────────┐ ┌──────────────┐ │ +│ │ Yemot API Client │ │ Call Tracking│ │ +│ └──────────────────┘ └──────────────┘ │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ External Services │ +│ ┌──────────────────┐ │ +│ │ Yemot IVR System │ │ +│ └──────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. Database Design + +### 4.1 Phone Message Template Entity + +**Table Name**: `phone_messages` + +**Schema**: +```typescript +@Entity('phone_messages') +export class PhoneMessage implements IHasUserId { + @PrimaryGeneratedColumn() + id: number; + + @Column('int', { name: 'user_id' }) + userId: number; + + @Column('varchar', { length: 100, unique: true }) + name: string; // Unique identifier/key + + @Column('varchar', { length: 500 }) + description: string; // Human-readable description + + @Column('enum', { enum: ['tts', 'audio_file'] }) + messageType: 'tts' | 'audio_file'; + + @Column('text', { nullable: true }) + textContent: string; // For TTS messages + + @Column('varchar', { length: 255, nullable: true }) + audioFilePath: string; // For pre-recorded audio + + @Column('boolean', { default: false }) + isActive: boolean; // Enable/disable without deletion + + @Column('varchar', { length: 100, nullable: true }) + yemotInstanceId: string; // Reference to Yemot configuration + + @Column('simple-json', { nullable: true }) + additionalParams: Record; // Flexible params for future use + + @CreateDateColumn() + createdAt: Date; + + @UpdateDateColumn() + updatedAt: Date; +} +``` + +**Design Questions:** + +**Q1: Should we support variable substitution in messages?** +- **Option A**: Fixed messages only (simpler implementation) + - Pro: Easier to implement and test + - Con: Less flexible +- **Option B**: Support placeholders like `{studentName}`, `{className}` ✅ RECOMMENDED + - Pro: More dynamic and personalized + - Con: Requires parameter mapping and validation + - Implementation: Follow existing `FormatString` pattern from `yemot.interface.ts` + +**Q2: Should messages be user-specific or system-wide?** +- **Option A**: User-specific (userId field required) ✅ RECOMMENDED + - Pro: Allows per-organization customization + - Con: More records in database + - Aligns with: Existing `Text` entity pattern +- **Option B**: System-wide with user overrides + - Pro: Reduced duplication + - Con: More complex implementation +- **Option C**: Hybrid (userId = 0 for system, specific userId for overrides) + - Pro: Best of both worlds + - Con: Most complex, but aligns with `TextByUser` view pattern + +**Recommendation**: Use Option C (Hybrid approach) to align with existing `Text`/`TextByUser` pattern. + +### 4.2 Call History Entity + +**Table Name**: `phone_call_history` + +**Schema**: +```typescript +@Entity('phone_call_history') +export class PhoneCallHistory implements IHasUserId { + @PrimaryGeneratedColumn() + id: number; + + @Column('int', { name: 'user_id' }) + userId: number; + + @Column('int', { name: 'phone_message_id' }) + phoneMessageId: number; + + @Column('varchar', { length: 20 }) + phoneNumber: string; + + @Column('varchar', { length: 100, nullable: true }) + recipientName: string; // For reference + + @Column('varchar', { length: 100, nullable: true }) + recipientType: string; // e.g., 'student', 'parent', 'teacher' + + @Column('int', { nullable: true }) + recipientReferenceId: number; // Reference to Student.id, etc. + + @Column('enum', { enum: ['pending', 'queued', 'calling', 'completed', 'failed', 'cancelled'] }) + status: string; + + @Column('varchar', { length: 255, nullable: true }) + yemotCallId: string; // External call ID from Yemot + + @Column('simple-json', { nullable: true }) + callResult: Record; // Response from Yemot API + + @Column('text', { nullable: true }) + errorMessage: string; + + @Column('datetime', { nullable: true }) + initiatedAt: Date; + + @Column('datetime', { nullable: true }) + completedAt: Date; + + @CreateDateColumn() + createdAt: Date; + + @UpdateDateColumn() + updatedAt: Date; +} +``` + +**Design Questions:** + +**Q3: Should we track call attempts or just final results?** +- **Option A**: Single record per call attempt ✅ RECOMMENDED + - Pro: Simpler schema + - Con: Lost history if retry +- **Option B**: Separate attempts table + - Pro: Full audit trail + - Con: More complex queries +- **Recommendation**: Start with Option A, consider B if retry logic is needed + +**Q4: Should we support scheduled calls?** +- **Option A**: Immediate execution only (MVP) + - Pro: Simpler implementation + - Con: Limited functionality +- **Option B**: Support scheduled time ✅ RECOMMENDED for Phase 2 + - Pro: More flexible (e.g., call during business hours) + - Con: Requires job scheduler (consider Bull, node-cron) + - Implementation: Add `scheduledFor: Date` column, use NestJS Bull Queue + +--- + +## 5. Backend Implementation + +### 5.1 Entity Module Configuration + +**File**: `/server/src/entity-modules/phone-message.config.ts` + +```typescript +import { PhoneMessage } from '@/db/entities/PhoneMessage.entity'; +import { getBaseEntityConfig } from '@shared/base-entity/base-entity-config'; + +export default getBaseEntityConfig(PhoneMessage, { + dto: { + create: CreatePhoneMessageDto, + update: UpdatePhoneMessageDto, + }, + permissions: { + list: 'admin', + get: 'admin', + create: 'admin', + update: 'admin', + delete: 'admin', + }, +}); +``` + +### 5.2 Phone Call Service + +**File**: `/server/src/services/phone-call.service.ts` + +**Key Responsibilities:** +1. Validate phone message template +2. Prepare message content (resolve variables, validate audio files) +3. Call Yemot API to initiate outbound call +4. Create call history records +5. Handle bulk call operations + +**Core Methods:** +```typescript +@Injectable() +export class PhoneCallService { + constructor( + @InjectRepository(PhoneMessage) private phoneMessageRepo: Repository, + @InjectRepository(PhoneCallHistory) private callHistoryRepo: Repository, + private yemotApiService: YemotApiService, + ) {} + + async initiateCall(params: InitiateCallDto): Promise; + async bulkInitiateCalls(params: BulkInitiateCallDto): Promise; + async getCallStatus(callId: number): Promise; + async cancelCall(callId: number): Promise; +} +``` + +### 5.3 Yemot API Integration + +**File**: `/server/shared/utils/yemot/yemot-api.service.ts` + +**Design Questions:** + +**Q5: Which Yemot API endpoint should we use for outbound calls?** + +Based on the Yemot API documentation (https://f2.freeivr.co.il), the relevant endpoints are: + +- **Option A**: `ApiOutCallV2` - Modern outbound call API ✅ RECOMMENDED + - Pro: Latest API, better features + - Con: May require specific Yemot account level + - **Parameters**: + - `token`: Authentication token + - `phone`: Target phone number + - `text` OR `file`: Message content (TTS or audio file URL) + - `CallerID`: Display number (optional) + - `params`: Additional parameters + +- **Option B**: `ApiOutCall` - Legacy outbound call API + - Pro: Broader compatibility + - Con: Older, potentially fewer features + +- **Option C**: `ApiSendMultipleMessage` - Bulk message API + - Pro: Optimized for bulk operations + - Con: May have rate limits + +**Recommendation**: Use `ApiOutCallV2` with fallback to `ApiOutCall` if needed. + +**Q6: How should we handle audio file hosting?** + +- **Option A**: Upload files to Yemot storage ✅ RECOMMENDED for Phase 1 + - Pro: Integrated with Yemot, no external hosting needed + - Con: Requires additional API call, file management + - Yemot API: `ApiUploadFile` + +- **Option B**: Host files on application server with public URL + - Pro: Full control over files + - Con: Requires public-facing endpoint, CDN consideration + - Implementation: Serve from `/server/public/audio/` with proper access controls + +- **Option C**: Use cloud storage (S3, Google Cloud Storage) + - Pro: Scalable, reliable + - Con: Additional infrastructure and costs + +**Recommendation**: Start with Option A (Yemot storage), plan for Option C if scale demands it. + +**Q7: How should we handle Yemot authentication tokens?** + +- **Option A**: Single system-wide token in environment variables + - Pro: Simple configuration + - Con: All calls from same account, single point of failure + +- **Option B**: User-specific tokens (stored per user) ✅ RECOMMENDED + - Pro: Better isolation, per-organization billing + - Con: More complex configuration, requires UI for token management + - Implementation: Add `yemotToken` field to User entity, encrypted storage + +- **Option C**: Centralized token management service + - Pro: Best security, token rotation support + - Con: Over-engineered for MVP + +**Recommendation**: Start with Option A for MVP, migrate to Option B for production. + +**Implementation Approach:** + +```typescript +@Injectable() +export class YemotApiService { + constructor(private httpService: HttpService) {} + + async initiateOutboundCall(params: YemotOutboundCallParams): Promise { + const endpoint = 'https://www.call2all.co.il/ym/api/ApiOutCallV2'; + const payload = { + token: params.token, + phone: params.phoneNumber, + ...(params.messageType === 'tts' + ? { text: params.textContent } + : { file: params.audioFileUrl }), + CallerID: params.callerId || undefined, + }; + + try { + const response = await this.httpService.post(endpoint, payload).toPromise(); + return this.parseYemotResponse(response.data); + } catch (error) { + throw new YemotApiException(`Failed to initiate call: ${error.message}`); + } + } + + async uploadAudioFile(token: string, filePath: string): Promise { + // Implementation for ApiUploadFile + } + + async getCallStatus(token: string, callId: string): Promise { + // Implementation for checking call status + } +} +``` + +--- + +## 6. Frontend Implementation + +### 6.1 Phone Message Template Management + +**File**: `/client/src/entities/phone-message.jsx` + +**UI Components:** +1. **List View**: Display all phone message templates with status +2. **Create Form**: Create new template with validation +3. **Edit Form**: Modify existing template +4. **Preview**: Test message content (TTS or play audio) + +**Key Fields:** +- Name (unique identifier) +- Description +- Message Type (TTS / Audio File) +- Content (text or file upload) +- Active status toggle +- Variable placeholders help text + +**Design Pattern**: Follow existing `text.jsx` entity pattern with Material-UI components. + +### 6.2 Bulk Action Button + +**File**: `/client/src/components/bulk-actions/BulkPhoneCallButton.jsx` + +**User Workflow:** +1. User selects students from list (e.g., student-klasses-report) +2. Clicks "Send Phone Call" bulk action button +3. Modal opens with: + - Dropdown to select phone message template + - Preview of message + - Phone number source field selection (e.g., student phone, parent phone) + - Confirmation button +4. System validates and queues calls +5. Success notification with summary + +**Implementation:** + +```jsx +export const BulkPhoneCallButton = () => { + const { selectedIds } = useListContext(); + const [selectedTemplate, setSelectedTemplate] = useState(null); + const { data: templates } = useGetList('phone_message', { + filter: { isActive: true } + }); + + const handleSendCalls = async () => { + // Call dataProvider to trigger bulk phone calls + await dataProvider.create('phone_call_history/bulk', { + data: { + phoneMessageId: selectedTemplate.id, + recipientIds: selectedIds, + recipientType: 'student', + } + }); + }; + + return ( + } + mutate={handleSendCalls} + // ... additional props + > + + + + ); +}; +``` + +**Integration Points:** +- Add to `student-klasses-report.jsx` additionalBulkButtons +- Add to `student-klass.jsx` additionalBulkButtons +- Consider adding to `student.jsx` for individual calls + +### 6.3 Call History Viewer + +**File**: `/client/src/entities/phone-call-history.jsx` + +**Display Information:** +- Call timestamp +- Recipient details +- Message template used +- Status (with color coding) +- Error messages (if failed) +- Filters: by date, status, recipient, message template + +**Actions:** +- Retry failed calls +- Cancel pending calls +- Export call log + +--- + +## 7. API Endpoints + +### 7.1 Phone Message Templates + +| Method | Endpoint | Description | Permissions | +|--------|----------|-------------|-------------| +| GET | `/api/phone_message` | List templates | admin | +| GET | `/api/phone_message/:id` | Get template details | admin | +| POST | `/api/phone_message` | Create template | admin | +| PATCH | `/api/phone_message/:id` | Update template | admin | +| DELETE | `/api/phone_message/:id` | Delete template | admin | + +### 7.2 Phone Call Operations + +| Method | Endpoint | Description | Permissions | +|--------|----------|-------------|-------------| +| POST | `/api/phone_call_history` | Initiate single call | admin | +| POST | `/api/phone_call_history/bulk` | Initiate bulk calls | admin | +| GET | `/api/phone_call_history` | List call history | admin | +| GET | `/api/phone_call_history/:id` | Get call details | admin | +| POST | `/api/phone_call_history/:id/retry` | Retry failed call | admin | +| POST | `/api/phone_call_history/:id/cancel` | Cancel pending call | admin | + +--- + +## 8. Security Considerations + +### 8.1 Phone Number Validation + +**Q8: Should we validate/sanitize phone numbers?** + +- **Option A**: Accept any format, pass to Yemot ✅ RECOMMENDED for MVP + - Pro: Flexible, Yemot handles validation + - Con: May allow invalid numbers + +- **Option B**: Validate Israeli phone number format + - Pro: Catch errors early, reduce costs + - Con: May reject valid international formats + - Pattern: `/^0\d{1,2}-?\d{7}$/` or `/^972\d{9}$/` + +- **Option C**: Use libphonenumber-js for international validation + - Pro: Robust, international support + - Con: Additional dependency + +**Recommendation**: Implement basic validation (Option B) with whitelist of prefixes. + +### 8.2 Rate Limiting + +**Considerations:** +- Prevent abuse of bulk call feature +- Respect Yemot API rate limits +- Implement NestJS ThrottlerModule (already in use) + +**Proposed Limits:** +- Phone message creation: 10 per hour per user +- Bulk calls: 100 calls per batch, 5 batches per hour per user +- Individual calls: 50 per hour per user + +### 8.3 Sensitive Data + +**Protect:** +- Phone numbers: Log only masked versions (e.g., `05X-XXX-1234`) +- Yemot tokens: Encrypt at rest, never log +- Audio files: Validate file types, scan for malware if user-uploaded + +### 8.4 Permissions + +**Access Control:** +- Only admin users can create/manage phone message templates +- Only admin users can initiate calls +- Consider adding specific permission: `canInitiatePhoneCalls` +- Audit all call actions in `audit_log` table + +--- + +## 9. Error Handling + +### 9.1 Failure Scenarios + +| Scenario | Handling Strategy | +|----------|------------------| +| Invalid phone number | Validate before queuing, mark as failed immediately | +| Yemot API timeout | Retry with exponential backoff (3 attempts) | +| Audio file not found | Fail fast, notify admin | +| Yemot account quota exceeded | Graceful failure, queue for retry later | +| Invalid template variables | Validate on template save and call initiation | +| Recipient opted out | Check opt-out list, skip silently | + +### 9.2 Retry Logic + +**Q9: How should we handle retries?** + +- **Option A**: Manual retry only (admin clicks retry button) + - Pro: Simple, controlled + - Con: Requires manual intervention + +- **Option B**: Automatic retry with exponential backoff ✅ RECOMMENDED + - Pro: Resilient, hands-off + - Con: More complex implementation + - Strategy: Retry at 1min, 5min, 15min intervals for transient failures + +- **Option C**: Queue-based with worker process + - Pro: Most robust, decoupled + - Con: Requires job queue infrastructure (Bull, BullMQ) + +**Recommendation**: Start with Option A, implement Option B for production. + +--- + +## 10. Integration with Existing Entities + +### 10.1 Student Entity + +**Changes Required:** +- Ensure `phone` field exists and is accessible +- Consider adding `parentPhone` field if not present +- Add opt-out flag: `phoneCallOptOut` (boolean) + +### 10.2 Student Klasses Report + +**Changes Required:** +- Add `BulkPhoneCallButton` to `additionalBulkButtons` array +- Ensure phone numbers are loaded with report data + +**Example Integration:** +```jsx +const additionalBulkButtons = [ + , + , // NEW +]; +``` + +### 10.3 App Registration + +**Changes Required:** +- Register `phone_message` resource in `/client/src/App.jsx` +- Register `phone_call_history` resource for admin users +- Add appropriate icons (PhoneIcon, CallIcon) +- Place in 'settings' menu group for templates +- Place in 'admin' menu group for call history + +--- + +## 11. Testing Strategy + +### 11.1 Unit Tests + +**Backend:** +- `PhoneCallService` methods +- Yemot API client mock responses +- Message template validation +- Variable substitution logic + +**Frontend:** +- Phone message CRUD components +- Bulk action button rendering +- Form validation + +### 11.2 Integration Tests + +- End-to-end flow: Create template → Trigger bulk call → Verify history +- Yemot API integration (use sandbox/test account) +- Error handling scenarios + +### 11.3 Manual Testing + +- Test with real Yemot account (use test phone numbers) +- Verify TTS quality and audio playback +- Test all bulk action scenarios +- Verify call history accuracy + +--- + +## 12. Performance Considerations + +### 12.1 Bulk Operations + +**Q10: How should we handle large bulk operations?** + +- **Option A**: Synchronous processing (block until done) + - Pro: Simple, immediate feedback + - Con: Slow for large batches, request timeout risk + +- **Option B**: Asynchronous with job queue ✅ RECOMMENDED + - Pro: Fast response, no timeout, resilient + - Con: Requires queue infrastructure (Bull) + - Implementation: + ```typescript + @Processor('phone-calls') + export class PhoneCallProcessor { + @Process('bulk-initiate') + async handleBulkCalls(job: Job) { + // Process calls in batches of 10 + } + } + ``` + +- **Option C**: Hybrid (queue for >50 calls, sync for smaller) + - Pro: Best UX for common cases + - Con: Dual code paths + +**Recommendation**: Implement Option B from the start to avoid refactoring. + +### 12.2 Database Indexing + +**Indexes Required:** +- `phone_messages`: `userId`, `name`, `isActive` +- `phone_call_history`: `userId`, `phoneMessageId`, `status`, `createdAt` + +### 12.3 Caching + +- Cache active phone message templates (60 seconds TTL) +- Use existing TypeORM caching for `TextByUser` pattern + +--- + +## 13. Monitoring & Observability + +### 13.1 Metrics to Track + +- Total calls initiated (by template, by user) +- Success/failure rates +- Average call duration +- Yemot API response times +- Queue depth (if using job queue) + +### 13.2 Logging + +**Log Events:** +- Template creation/modification +- Bulk call initiation (with count) +- Individual call status changes +- Yemot API errors +- Retry attempts + +**Log Level Guidelines:** +- INFO: Call initiated, completed +- WARN: Retry attempt, invalid phone number +- ERROR: Yemot API failure, system error + +### 13.3 Alerts + +**Alert Conditions:** +- Failure rate > 10% in 1-hour window +- Yemot API unavailable +- Queue depth > 1000 pending calls +- Token authentication failures + +--- + +## 14. Implementation Phases + +### Phase 1: MVP (Core Functionality) +**Timeline: 2-3 weeks** + +**Backend:** +- [ ] Create `PhoneMessage` entity and migration +- [ ] Create `PhoneCallHistory` entity and migration +- [ ] Implement `PhoneCallService` with basic operations +- [ ] Create Yemot API client service (TTS only) +- [ ] Add phone message entity module configuration +- [ ] Add bulk call endpoint + +**Frontend:** +- [ ] Create phone message CRUD UI (list, create, edit) +- [ ] Implement `BulkPhoneCallButton` component +- [ ] Integrate button into `student-klasses-report` +- [ ] Create call history viewer (read-only) + +**Testing:** +- [ ] Unit tests for services +- [ ] Integration test for bulk call flow +- [ ] Manual testing with Yemot sandbox + +### Phase 2: Enhanced Features +**Timeline: 1-2 weeks** + +- [ ] Audio file upload support +- [ ] Variable substitution in messages +- [ ] Retry logic for failed calls +- [ ] User-specific Yemot token management +- [ ] Call status webhook endpoint (if Yemot supports) + +### Phase 3: Advanced Features +**Timeline: 2-3 weeks** + +- [ ] Scheduled calls (job queue with Bull) +- [ ] Call templates with approval workflow +- [ ] Opt-out management UI +- [ ] Enhanced reporting and analytics +- [ ] Export call history to CSV +- [ ] Integration with additional entities (teachers, parents) + +### Phase 4: Production Hardening +**Timeline: 1 week** + +- [ ] Performance optimization +- [ ] Enhanced error handling +- [ ] Monitoring dashboard +- [ ] Load testing +- [ ] Security audit +- [ ] Documentation + +--- + +## 15. Open Questions & Decisions Required + +### 15.1 Critical Decisions + +**Q11: Yemot Account Setup** +- Do we have an existing Yemot account with API access? +- What is the account tier and associated rate limits? +- Do we need separate accounts for staging/production? + +**Q12: User Workflow Priorities** +- Which entity lists should have bulk call buttons initially? + - Student klasses report ✅ (specified in requirements) + - Student list? + - Teacher list (for staff notifications)? + - Custom recipient lists? + +**Q13: Message Content Validation** +- Should we validate message content (e.g., max length, forbidden words)? +- Should admins be able to preview/test messages before using in bulk? + +**Q14: Billing & Cost Control** +- How should we track Yemot call costs? +- Should we implement quota limits per user/organization? +- Should we require additional approval for large bulk operations (e.g., >100 calls)? + +### 15.2 Future Considerations + +**Q15: Interactive Calls (IVR)** +- Current design focuses on one-way outbound messages +- Should we plan for interactive responses in the future? +- Would require webhook endpoint for Yemot to send responses back + +**Q16: Multi-Language Support** +- Should TTS support multiple languages? +- Yemot supports Hebrew TTS by default +- Consider adding language field to template if needed + +**Q17: Integration with Existing Yemot Functionality** +- The system has existing inbound call handling (`YemotHandlerService`, chains) +- Should outbound calls be integrated with the existing call tracking? +- Current `YemotCall` entity tracks inbound calls - should outbound use the same or separate entity? + +--- + +## 16. Dependencies & Prerequisites + +### 16.1 External Services +- **Yemot Account**: Active account with API access, token obtained +- **Audio File Hosting**: If using pre-recorded audio (Yemot storage or internal CDN) + +### 16.2 Technical Dependencies +- **NestJS HttpModule**: For Yemot API calls +- **Bull/BullMQ** (Phase 2): For job queue (optional) +- **@nestjs/schedule** (Phase 3): For scheduled calls (optional) +- **class-validator**: For DTO validation (already in use) + +### 16.3 Infrastructure +- **Database**: Migrations for new entities +- **Storage**: For uploaded audio files (if applicable) +- **Redis** (Phase 2): For job queue (if using Bull) + +--- + +## 17. Risk Assessment + +| Risk | Impact | Likelihood | Mitigation | +|------|--------|------------|------------| +| Yemot API changes | High | Medium | Version API calls, monitor Yemot docs | +| Rate limiting by Yemot | Medium | Medium | Implement queue, respect limits | +| Invalid phone numbers cause costs | Medium | High | Validate before calling, maintain opt-out list | +| Audio file storage costs | Low | Low | Use Yemot storage, set file size limits | +| User error in bulk operations | High | Medium | Confirmation dialogs, preview, undo option | +| Privacy concerns with call logs | Medium | Low | Mask phone numbers in logs, retention policy | + +--- + +## 18. Success Criteria + +### 18.1 Functional Requirements +- ✅ Admins can create and manage phone message templates +- ✅ Admins can select students and trigger bulk phone calls +- ✅ System tracks call history with status +- ✅ Supports both TTS and audio file messages +- ✅ Integrates seamlessly with existing UI + +### 18.2 Non-Functional Requirements +- **Performance**: Bulk calls queued in <2 seconds, processed within 5 minutes +- **Reliability**: 95%+ call success rate (excluding invalid numbers) +- **Usability**: New users can create and send calls within 5 minutes +- **Security**: No unauthorized access, audit trail for all actions + +### 18.3 Business Metrics +- Time savings: Reduce manual calling time by 80% +- Adoption: 50%+ of admins use feature within first month +- Satisfaction: Positive feedback from users + +--- + +## 19. References + +### 19.1 Yemot API Documentation +- Main API Documentation: https://f2.freeivr.co.il/topic/55/api-גישת-מפתחים-למערכות/5 +- Outbound Calls: (Follow links from main page to ApiOutCallV2 and ApiOutCall) +- File Upload: (Follow links from main page to ApiUploadFile) + +### 19.2 Existing Code Patterns +- Entity Module Configuration: `/server/src/entity-modules/*.config.ts` +- Bulk Action Buttons: `/client/src/reports/studentReportCardReactButton.jsx` +- Text Entity Pattern: `/server/shared/entities/Text.entity.ts` +- Yemot Integration: `/server/shared/utils/yemot/` + +### 19.3 Related Documentation +- Project Index: `/project-index.md` +- Agents Workflow: `/AGENTS.md` +- New Project Checklist: `/new-project-checklist.md` + +--- + +## 20. Appendices + +### Appendix A: Database Migration Script (Skeleton) + +```sql +-- Phone Message Templates +CREATE TABLE phone_messages ( + id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, + user_id INT NOT NULL, + name VARCHAR(100) NOT NULL UNIQUE, + description VARCHAR(500) NOT NULL, + message_type ENUM('tts', 'audio_file') NOT NULL, + text_content TEXT NULL, + audio_file_path VARCHAR(255) NULL, + is_active BOOLEAN DEFAULT FALSE, + yemot_instance_id VARCHAR(100) NULL, + additional_params JSON NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + INDEX idx_user_id (user_id), + INDEX idx_name (name), + INDEX idx_is_active (is_active) +); + +-- Phone Call History +CREATE TABLE phone_call_history ( + id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, + user_id INT NOT NULL, + phone_message_id INT UNSIGNED NOT NULL, + phone_number VARCHAR(20) NOT NULL, + recipient_name VARCHAR(100) NULL, + recipient_type VARCHAR(100) NULL, + recipient_reference_id INT NULL, + status ENUM('pending', 'queued', 'calling', 'completed', 'failed', 'cancelled') NOT NULL, + yemot_call_id VARCHAR(255) NULL, + call_result JSON NULL, + error_message TEXT NULL, + initiated_at DATETIME NULL, + completed_at DATETIME NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + INDEX idx_user_id (user_id), + INDEX idx_phone_message_id (phone_message_id), + INDEX idx_status (status), + INDEX idx_created_at (created_at), + FOREIGN KEY (phone_message_id) REFERENCES phone_messages(id) ON DELETE CASCADE +); +``` + +### Appendix B: TypeScript Types/Interfaces + +```typescript +// DTOs +export class CreatePhoneMessageDto { + @IsString() + @MaxLength(100) + name: string; + + @IsString() + @MaxLength(500) + description: string; + + @IsEnum(['tts', 'audio_file']) + messageType: 'tts' | 'audio_file'; + + @IsOptional() + @IsString() + textContent?: string; + + @IsOptional() + @IsString() + @MaxLength(255) + audioFilePath?: string; + + @IsBoolean() + @IsOptional() + isActive?: boolean; +} + +export class BulkInitiateCallDto { + @IsNumber() + phoneMessageId: number; + + @IsArray() + @IsNumber({}, { each: true }) + recipientIds: number[]; + + @IsEnum(['student', 'teacher', 'parent']) + recipientType: 'student' | 'teacher' | 'parent'; + + @IsOptional() + @IsString() + phoneNumberField?: string; // e.g., 'phone', 'parentPhone' +} + +// Yemot API Types +export interface YemotOutboundCallParams { + token: string; + phoneNumber: string; + messageType: 'tts' | 'audio_file'; + textContent?: string; + audioFileUrl?: string; + callerId?: string; +} + +export interface YemotCallResponse { + success: boolean; + callId?: string; + errorMessage?: string; +} +``` + +### Appendix C: Sample UI Mockups (Descriptions) + +**Phone Message Template List:** +``` ++--------------------------------------------------+ +| Phone Message Templates [+ New] | ++--------------------------------------------------+ +| Name | Type | Active | Actions| +|-------------------|-------------|--------|--------| +| Welcome Message | TTS | ✓ | Edit | +| Absence Reminder | Audio File | ✓ | Edit | +| Payment Due | TTS | ✗ | Edit | ++--------------------------------------------------+ +``` + +**Bulk Call Dialog:** +``` ++-----------------------------------------------+ +| Send Phone Calls to 15 Students | ++-----------------------------------------------+ +| Select Message Template: | +| [▼ Welcome Message ] | +| | +| Phone Number Field: | +| [▼ Student Phone ] | +| | +| Preview: | +| "שלום {studentName}, זוהי תזכורת..." | +| | +| [Cancel] [Send Calls] | ++-----------------------------------------------+ +``` + +--- + +## Revision History + +| Version | Date | Author | Changes | +|---------|------|--------|---------| +| 1.0 | 2026-01-15 | GitHub Copilot | Initial HLD draft | + +--- + +**Document Status**: Draft - Awaiting stakeholder review and decision on open questions. + +**Next Steps**: +1. Review and answer open questions (Q1-Q17) +2. Approve/modify database schema +3. Confirm Yemot API approach and obtain credentials +4. Approve implementation phases and timeline +5. Begin Phase 1 development From 9f93604ec74a0e66b271f3f7c7daf81aa76c61e3 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 15:06:23 +0000 Subject: [PATCH 3/3] Update HLD to use Yemot Templates/Campaigns API (v2.0) - Shifted from generic outbound calls to Yemot Templates/Campaigns system - Updated entities: YemotCampaignTemplate, YemotCampaign - Documented complete Templates/Campaigns API workflow - Added comprehensive API method references - Updated all backend services, frontend components, and workflows - Revised implementation phases to reflect campaign-based approach Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-automated-phone-calls-summary.md | 250 +---- docs/HLD-automated-phone-calls.md | 1217 ++++++++++++++------- 2 files changed, 868 insertions(+), 599 deletions(-) diff --git a/docs/HLD-automated-phone-calls-summary.md b/docs/HLD-automated-phone-calls-summary.md index 722f32e8..b45c58fc 100644 --- a/docs/HLD-automated-phone-calls-summary.md +++ b/docs/HLD-automated-phone-calls-summary.md @@ -1,202 +1,66 @@ # Automated Phone Calls HLD - Executive Summary ## Overview -This document provides a high-level design for implementing automated outbound phone calling functionality integrated with the Yemot telephony platform. - -## Key Features -1. **Phone Message Template Management** - Create reusable message templates with TTS or audio files -2. **Bulk Triggering** - Send calls to multiple students from existing UI lists -3. **Call Tracking** - Monitor call status and history -4. **User-Specific Configuration** - Per-organization message customization - -## Architecture Summary - -### Database Entities -- `phone_messages` - Stores message templates (TTS text or audio file paths) -- `phone_call_history` - Tracks individual call attempts and results - -### Backend Components -- **PhoneMessage Entity** - Manages message templates -- **PhoneCallService** - Orchestrates call initiation and tracking -- **YemotApiService** - Integrates with Yemot API (ApiOutCallV2) - -### Frontend Components -- **Phone Message CRUD** - UI for managing templates (`/client/src/entities/phone-message.jsx`) -- **BulkPhoneCallButton** - Bulk action button for entity lists -- **Call History Viewer** - Display call logs and status - -## Critical Design Questions Requiring Your Input - -### Q1: Variable Substitution -**Should messages support placeholders like `{studentName}`, `{className}`?** -- ✅ **RECOMMENDED**: Yes - More flexible and personalized -- Alternative: Fixed messages only - Simpler but less useful - -### Q2: Message Ownership Pattern -**Should messages follow the Text/TextByUser pattern?** -- ✅ **RECOMMENDED**: Hybrid (system defaults + user overrides, userId=0 for system) -- Alternative: Pure user-specific (userId always set) - -### Q5: Yemot API Endpoint -**Which API should we use?** -- ✅ **RECOMMENDED**: ApiOutCallV2 (modern, feature-rich) -- Alternative: ApiOutCall (legacy, broader compatibility) - -### Q6: Audio File Hosting -**Where should audio files be stored?** -- ✅ **RECOMMENDED for Phase 1**: Upload to Yemot storage (ApiUploadFile) -- Alternative: Host on application server with public URL -- Future: Cloud storage (S3/GCS) if scale demands - -### Q7: Yemot Token Management -**How should authentication tokens be handled?** -- ✅ **RECOMMENDED for MVP**: Single system-wide token (env variable) -- **RECOMMENDED for Production**: User-specific tokens (stored encrypted in User entity) - -### Q8: Phone Number Validation -**Should we validate phone numbers before calling?** -- ✅ **RECOMMENDED for MVP**: Accept any format, let Yemot validate -- Alternative: Validate Israeli format (regex pattern) -- Future: Use libphonenumber-js for international support - -### Q10: Bulk Operations Processing -**How should large bulk calls be handled?** -- ✅ **RECOMMENDED**: Asynchronous with job queue (Bull/BullMQ) -- Alternative: Synchronous (risk of timeout) - -### Q11: Yemot Account Setup ⚠️ CRITICAL -**Required Information:** -- Do we have an existing Yemot account with API access? -- What is the account tier and rate limits? -- Do we need separate accounts for staging/production? -- What is the authentication token? - -### Q12: User Workflow Priorities -**Which lists should have bulk call buttons?** -- ✅ Student Klasses Report (specified in requirements) -- Student List? -- Teacher List (staff notifications)? - -### Q13: Message Content Validation -- Should we validate content (max length, forbidden words)? -- Should admins preview/test messages before bulk use? - -### Q14: Billing & Cost Control -- How to track Yemot call costs? -- Should we implement quota limits per user? -- Approval required for large operations (>100 calls)? - -## Implementation Phases - -### Phase 1: MVP (2-3 weeks) -- Create database entities (PhoneMessage, PhoneCallHistory) -- Implement PhoneCallService and YemotApiService -- Build phone message CRUD UI -- Add BulkPhoneCallButton to student-klasses-report -- Basic call history viewer -- **TTS only** (no audio files yet) - -### Phase 2: Enhanced (1-2 weeks) -- Audio file upload support -- Variable substitution -- Retry logic for failed calls -- User-specific token management - -### Phase 3: Advanced (2-3 weeks) -- Scheduled calls (job queue) -- Opt-out management -- Enhanced analytics -- CSV export - -### Phase 4: Production Hardening (1 week) -- Performance optimization -- Monitoring/alerting -- Security audit -- Load testing - -## Integration Points - -### Existing Entities Modified -1. **Student Klasses Report** (`/client/src/entities/student-klasses-report.jsx`) - - Add `BulkPhoneCallButton` to `additionalBulkButtons` - -2. **App.jsx** (`/client/src/App.jsx`) - - Register `phone_message` resource (settings menu) - - Register `phone_call_history` resource (admin menu) - -3. **Student Entity** (optional) - - Consider adding `parentPhone` field - - Consider adding `phoneCallOptOut` flag - -### New Files Created +This document provides a high-level design for implementing automated outbound phone calling functionality integrated with the **Yemot Templates/Campaigns API** system. + +**Version**: 2.0 (Major revision - now using Yemot Templates/Campaigns) +**Date**: 2026-01-15 +**API Reference**: [Yemot Templates/Campaigns](https://f2.freeivr.co.il/topic/55/api-גישת-מפתחים-למערכות/5) - חלק 2 ניהול קמפיינים + +## Key Changes from v1.0 +- ✅ Now uses Yemot Templates/Campaigns API (per stakeholder request) +- ✅ Template-based campaign approach vs individual call approach +- ✅ Comprehensive campaign management and reporting +- ✅ Full Yemot API method references and workflow documentation + +## Yemot Templates/Campaigns Workflow + +1. **Create/Sync Template**: Admin creates template in system (calls `CreateTemplate` API) or syncs existing templates from Yemot (`GetTemplates`) +2. **Configure Content**: Template content (audio/IVR) configured in Yemot web interface (Phase 1) or via API (Phase 2) +3. **Select Recipients**: Admin selects students from list (e.g., student-klasses-report) +4. **Upload Phone List**: System uploads phone numbers to Yemot template (`UploadPhoneList`) +5. **Run Campaign**: System triggers campaign execution (`RunCampaign`) +6. **Monitor Progress**: System polls campaign status (`GetCampaignStatus`) +7. **Download Report**: System retrieves detailed results (`DownloadCampaignReport`) + +## Critical Questions + +### Q11: Yemot Account ⚠️ URGENT +- Do you have Yemot account with **Templates/Campaigns API access**? +- Account tier and rate limits? +- Authentication token? + +### Q13: Template Management +- Create new templates via API OR sync existing templates? +- **Recommended**: Hybrid - support both approaches + +### Q14: Cost Control +- Track campaign costs? +- Quota limits per user? +- Approval for large campaigns (>100 calls)? + +## Implementation (Phase 1: 2-3 weeks) + **Backend:** -- `/server/src/db/entities/PhoneMessage.entity.ts` -- `/server/src/db/entities/PhoneCallHistory.entity.ts` -- `/server/src/entity-modules/phone-message.config.ts` -- `/server/src/entity-modules/phone-call-history.config.ts` -- `/server/src/services/phone-call.service.ts` -- `/server/shared/utils/yemot/yemot-api.service.ts` +- `YemotCampaignTemplate` & `YemotCampaign` entities +- `YemotTemplatesApiService` (API client) +- `YemotCampaignService` (orchestration) **Frontend:** -- `/client/src/entities/phone-message.jsx` -- `/client/src/entities/phone-call-history.jsx` -- `/client/src/components/bulk-actions/BulkPhoneCallButton.jsx` - -## Security Considerations -- **Rate Limiting**: Prevent abuse (10 templates/hour, 100 calls/batch) -- **Phone Masking**: Log masked numbers only (05X-XXX-1234) -- **Token Security**: Encrypt Yemot tokens at rest -- **Permissions**: Admin-only access -- **Audit Trail**: Log all call actions - -## Performance Considerations -- **Bulk Operations**: Use job queue (Bull) for async processing -- **Database Indexing**: Index userId, status, createdAt on history table -- **Caching**: Cache active templates (60s TTL) - -## Testing Strategy -- **Unit Tests**: Service methods, API client, validation -- **Integration Tests**: End-to-end flow with Yemot sandbox -- **Manual Tests**: Real Yemot account with test phone numbers - -## Success Metrics -- **Performance**: Bulk calls queued <2s, processed <5min -- **Reliability**: 95%+ success rate (excluding invalid numbers) -- **Adoption**: 50%+ admins use within first month -- **Time Savings**: 80% reduction in manual calling time - -## Risks & Mitigation - -| Risk | Mitigation | -|------|------------| -| Yemot API changes | Version API calls, monitor docs | -| Rate limiting | Implement queue, respect limits | -| Invalid numbers cost money | Validate before calling, opt-out list | -| User errors in bulk ops | Confirmation dialogs, preview | - -## Next Steps (Action Items) - -1. **Immediate**: Answer critical questions (Q11-Q14) -2. **Week 1**: Obtain Yemot credentials, set up sandbox -3. **Week 1-2**: Implement Phase 1 MVP backend -4. **Week 2-3**: Implement Phase 1 MVP frontend -5. **Week 3**: Integration testing -6. **Week 4+**: Phase 2 enhancements - -## Questions for You - -Please review the full HLD document (`docs/HLD-automated-phone-calls.md`) and provide answers to: - -1. **Q11**: Yemot account details and credentials -2. **Q12**: Which entity lists need bulk call buttons -3. **Q13**: Message content validation requirements -4. **Q14**: Billing and cost control preferences -5. **General**: Any concerns or modifications to the proposed design - -Once these are answered, we can proceed with implementation! +- Campaign template CRUD with sync +- `BulkCampaignButton` component +- Campaign history viewer + +**Key APIs:** +- `CreateTemplate`, `GetTemplates`, `UploadPhoneList`, `RunCampaign`, `GetCampaignStatus`, `DownloadCampaignReport` + +## Next Steps + +1. Answer Q11 (Yemot account verification) +2. Review full HLD: `/docs/HLD-automated-phone-calls.md` +3. Approve database schema +4. Begin Phase 1 development --- -**Full HLD Document**: `/docs/HLD-automated-phone-calls.md` -**Created**: 2026-01-15 -**Status**: Awaiting stakeholder review +**Full Documentation**: `/docs/HLD-automated-phone-calls.md` (comprehensive 1500+ line design document) diff --git a/docs/HLD-automated-phone-calls.md b/docs/HLD-automated-phone-calls.md index 55004200..f4866d94 100644 --- a/docs/HLD-automated-phone-calls.md +++ b/docs/HLD-automated-phone-calls.md @@ -2,28 +2,36 @@ ## Executive Summary -This document outlines the design for an automated outbound phone calling system integrated with the Yemot telephony platform. The system will allow administrators to create reusable phone message templates and trigger automated calls to students' phone numbers through bulk actions in the existing UI. +This document outlines the design for an automated outbound phone calling system integrated with the Yemot telephony platform using the **Templates/Campaigns API**. The system will allow administrators to create reusable campaign templates in Yemot and trigger automated calls to students' phone numbers through bulk actions in the existing UI. -**Version:** 1.0 +**Version:** 2.0 **Date:** 2026-01-15 -**Status:** Draft for Review +**Status:** Draft for Review +**API Reference:** [Yemot API Templates/Campaigns](https://f2.freeivr.co.il/topic/55/api-גישת-מפתחים-למערכות/5) - חלק 2 ניהול קמפיינים --- ## 1. Feature Overview ### 1.1 Business Goals -- Enable automated communication with students/parents via phone calls +- Enable automated communication with students/parents via phone calls using Yemot Campaigns - Reduce manual effort in sending routine notifications -- Provide a flexible, template-based system for various message types +- Provide a flexible, template-based system leveraging Yemot's campaign infrastructure - Integrate seamlessly with existing educational workflows ### 1.2 Key Capabilities -1. **Message Template Management**: Create, edit, and manage phone message templates -2. **Flexible Message Content**: Support both text-to-speech (TTS) and pre-recorded audio files -3. **Bulk Triggering**: Initiate calls to multiple recipients from entity lists (e.g., student-klasses-report) -4. **Call Tracking**: Monitor call status and results -5. **User-Specific Configuration**: Allow each organization to customize messages +1. **Campaign Template Management**: Create, configure, and manage Yemot campaign templates +2. **Flexible Message Content**: Support both text-to-speech (TTS) and pre-recorded audio files via Yemot +3. **Bulk Triggering**: Initiate campaigns to multiple recipients from entity lists (e.g., student-klasses-report) +4. **Campaign Tracking**: Monitor campaign status, results, and download reports +5. **User-Specific Configuration**: Allow each organization to manage their own campaign templates + +### 1.3 Yemot Templates/Campaigns System +The Yemot platform provides a robust Templates/Campaigns API that: +- **Templates**: Reusable campaign configurations stored in Yemot (message content, IVR flow, settings) +- **Campaigns**: Executions of templates with specific recipient lists +- **Phone Lists**: Managed lists of phone numbers associated with each template +- **Reports**: Detailed campaign execution reports (call status, duration, user interactions) --- @@ -31,35 +39,33 @@ This document outlines the design for an automated outbound phone calling system ### 2.1 Core Terms -**Proposed Term: "Phone Message Template" (or "Call Template")** +**Proposed Term: "Campaign Template" (Yemot Template)** -**Alternative Options:** -1. **"Phone Message Template"** ✅ RECOMMENDED - - **Pros**: Clear, descriptive, aligns with existing "Text" entity pattern - - **Cons**: Slightly verbose - - **Use Case**: Best for UI labels and user-facing documentation +**Key Concepts:** +1. **"Campaign Template"** ✅ RECOMMENDED + - **Definition**: A reusable configuration in Yemot that defines the message content, IVR flow, and settings + - **Yemot API Term**: "Template" (תבנית קמפיין) + - **Use Case**: Aligns with Yemot's terminology, clear for users + - **Database/Code**: `campaign_template` or `yemot_template` -2. **"Call Template"** - - **Pros**: Concise, developer-friendly - - **Cons**: May be ambiguous (template for what?) - - **Use Case**: Good for code/API naming +2. **"Campaign"** + - **Definition**: An execution instance of a template with a specific recipient list + - **Yemot API Term**: "Campaign" (קמפיין) + - **Use Case**: Represents an actual outbound call batch -3. **"Voice Message"** - - **Pros**: User-friendly, intuitive - - **Cons**: Doesn't convey the template/reusable nature - - **Use Case**: Good for non-technical users +3. **"Phone List Entry"** + - **Definition**: Individual phone numbers in a campaign's recipient list + - **Yemot API Term**: "Template Entry" (רשימת תפוצה) + - **Use Case**: Phone numbers to call for a specific campaign -4. **"Phone Notification Template"** - - **Pros**: Emphasizes notification purpose - - **Cons**: Too long for frequent use - - **Use Case**: Good for formal documentation +**Decision Required**: Should we: +- Store full template configuration locally OR just reference Yemot template IDs? +- Sync template data from Yemot OR manage independently? -**Decision Required**: Which terminology should we use for: -- Entity name in database? -- UI labels? -- API endpoints? - -**Recommendation**: Use "Phone Message Template" for UI, "phone_message" for database entities, and "phoneMessage" in code. +**Recommendation**: +- **Database**: Store `yemot_campaign_template` with reference to Yemot template ID +- **UI**: Use "Campaign Template" terminology +- **Code**: Use `campaignTemplate` in TypeScript/JavaScript --- @@ -71,8 +77,8 @@ This document outlines the design for an automated outbound phone calling system ┌─────────────────────────────────────────────────────────────┐ │ Client Layer │ │ ┌──────────────────┐ ┌──────────────────────────────┐ │ -│ │ Phone Message │ │ Bulk Action Buttons │ │ -│ │ CRUD UI │ │ (Student List, etc.) │ │ +│ │ Campaign │ │ Bulk Action Buttons │ │ +│ │ Template CRUD │ │ (Student List, etc.) │ │ │ └──────────────────┘ └──────────────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ │ @@ -80,67 +86,90 @@ This document outlines the design for an automated outbound phone calling system ┌─────────────────────────────────────────────────────────────┐ │ Server Layer │ │ ┌──────────────────┐ ┌──────────────────────────────┐ │ -│ │ Phone Message │ │ Phone Call Service │ │ -│ │ Entity & Config │ │ (Orchestration) │ │ +│ │ Campaign │ │ Campaign Service │ │ +│ │ Template Entity │ │ (Orchestration) │ │ │ └──────────────────┘ └──────────────────────────────┘ │ │ │ │ │ ┌────────┴────────┐ │ │ ▼ ▼ │ │ ┌──────────────────┐ ┌──────────────┐ │ -│ │ Yemot API Client │ │ Call Tracking│ │ +│ │ Yemot Templates │ │ Campaign │ │ +│ │ API Client │ │ History │ │ │ └──────────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ -│ External Services │ -│ ┌──────────────────┐ │ -│ │ Yemot IVR System │ │ -│ └──────────────────┘ │ +│ Yemot IVR System │ +│ ┌──────────────────┐ ┌──────────────────────────────┐ │ +│ │ Templates │ │ Campaigns │ │ +│ │ (Stored in │ │ (Active/Scheduled) │ │ +│ │ Yemot) │ │ │ │ +│ └──────────────────┘ └──────────────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ ``` +### 3.2 Workflow Overview + +**Template Creation Flow:** +1. Admin creates campaign template in local system +2. System calls `CreateTemplate` API to create template in Yemot +3. System stores Yemot template ID and metadata locally +4. Admin can configure message content via Yemot UI or API + +**Campaign Execution Flow:** +1. Admin selects students from list (e.g., student-klasses-report) +2. Admin clicks bulk action button and selects campaign template +3. System uploads phone numbers to Yemot via `UploadPhoneList` API +4. System triggers campaign via `RunCampaign` API +5. Yemot executes campaign (calls all numbers) +6. System polls campaign status via `GetCampaignStatus` API +7. System downloads campaign report via `DownloadCampaignReport` API +``` + --- ## 4. Database Design -### 4.1 Phone Message Template Entity +### 4.1 Campaign Template Entity + +**Table Name**: `yemot_campaign_templates` -**Table Name**: `phone_messages` +**Purpose**: Store local references to Yemot campaign templates with metadata **Schema**: ```typescript -@Entity('phone_messages') -export class PhoneMessage implements IHasUserId { +@Entity('yemot_campaign_templates') +export class YemotCampaignTemplate implements IHasUserId { @PrimaryGeneratedColumn() id: number; @Column('int', { name: 'user_id' }) userId: number; - @Column('varchar', { length: 100, unique: true }) - name: string; // Unique identifier/key + @Column('varchar', { length: 100 }) + name: string; // Local friendly name @Column('varchar', { length: 500 }) description: string; // Human-readable description - @Column('enum', { enum: ['tts', 'audio_file'] }) - messageType: 'tts' | 'audio_file'; - - @Column('text', { nullable: true }) - textContent: string; // For TTS messages + @Column('varchar', { length: 50 }) + yemotTemplateId: string; // Template ID from Yemot (e.g., "1234") - @Column('varchar', { length: 255, nullable: true }) - audioFilePath: string; // For pre-recorded audio + @Column('varchar', { length: 100, nullable: true }) + yemotTemplateName: string; // Template name in Yemot system - @Column('boolean', { default: false }) + @Column('boolean', { default: true }) isActive: boolean; // Enable/disable without deletion - @Column('varchar', { length: 100, nullable: true }) - yemotInstanceId: string; // Reference to Yemot configuration + @Column('boolean', { default: false }) + isSynced: boolean; // Whether template exists in Yemot @Column('simple-json', { nullable: true }) - additionalParams: Record; // Flexible params for future use + templateConfig: Record; // Cached template configuration from Yemot + + @Column('datetime', { nullable: true }) + lastSyncedAt: Date; // Last sync with Yemot @CreateDateColumn() createdAt: Date; @@ -152,72 +181,85 @@ export class PhoneMessage implements IHasUserId { **Design Questions:** -**Q1: Should we support variable substitution in messages?** -- **Option A**: Fixed messages only (simpler implementation) - - Pro: Easier to implement and test - - Con: Less flexible -- **Option B**: Support placeholders like `{studentName}`, `{className}` ✅ RECOMMENDED - - Pro: More dynamic and personalized - - Con: Requires parameter mapping and validation - - Implementation: Follow existing `FormatString` pattern from `yemot.interface.ts` - -**Q2: Should messages be user-specific or system-wide?** -- **Option A**: User-specific (userId field required) ✅ RECOMMENDED - - Pro: Allows per-organization customization - - Con: More records in database - - Aligns with: Existing `Text` entity pattern -- **Option B**: System-wide with user overrides - - Pro: Reduced duplication +**Q1: Should we create templates in Yemot or sync existing ones?** +- **Option A**: Create new templates via API (`CreateTemplate`) ✅ RECOMMENDED + - Pro: Full control, integrated workflow + - Con: Templates managed separately in Yemot UI + - Implementation: Use `CreateTemplate` API on save + +- **Option B**: Sync existing templates from Yemot (`GetTemplates`) + - Pro: Use templates already configured in Yemot + - Con: No local template creation + - Implementation: Import templates via `GetTemplates` API + +- **Option C**: Hybrid - support both creation and sync ✅ BEST APPROACH + - Pro: Maximum flexibility - Con: More complex implementation -- **Option C**: Hybrid (userId = 0 for system, specific userId for overrides) - - Pro: Best of both worlds - - Con: Most complex, but aligns with `TextByUser` view pattern + - Implementation: "Create New" and "Import Existing" buttons + +**Q2: How should we handle template content (audio files, TTS)?** +- **Option A**: Manage entirely in Yemot ✅ RECOMMENDED for Phase 1 + - Pro: Leverage Yemot's content management + - Con: Users must use Yemot UI for content + - Implementation: Store only template reference + +- **Option B**: Upload content via API + - Pro: Integrated workflow + - Con: Complex file management + - Implementation: Use `UploadFile` and audio management APIs -**Recommendation**: Use Option C (Hybrid approach) to align with existing `Text`/`TextByUser` pattern. +**Recommendation**: +- Use Option C (Hybrid) for template management +- Use Option A for content management in Phase 1, Option B for Phase 2 -### 4.2 Call History Entity +### 4.2 Campaign History Entity -**Table Name**: `phone_call_history` +**Table Name**: `yemot_campaigns` + +**Purpose**: Track campaign executions and results **Schema**: ```typescript -@Entity('phone_call_history') -export class PhoneCallHistory implements IHasUserId { +@Entity('yemot_campaigns') +export class YemotCampaign implements IHasUserId { @PrimaryGeneratedColumn() id: number; @Column('int', { name: 'user_id' }) userId: number; - @Column('int', { name: 'phone_message_id' }) - phoneMessageId: number; + @Column('int', { name: 'campaign_template_id' }) + campaignTemplateId: number; - @Column('varchar', { length: 20 }) - phoneNumber: string; + @ManyToOne(() => YemotCampaignTemplate) + campaignTemplate: YemotCampaignTemplate; - @Column('varchar', { length: 100, nullable: true }) - recipientName: string; // For reference + @Column('varchar', { length: 50 }) + yemotCampaignId: string; // Campaign ID from Yemot @Column('varchar', { length: 100, nullable: true }) - recipientType: string; // e.g., 'student', 'parent', 'teacher' + recipientSource: string; // e.g., 'student_klasses_report' - @Column('int', { nullable: true }) - recipientReferenceId: number; // Reference to Student.id, etc. + @Column('simple-json', { nullable: true }) + recipientIds: number[]; // Local entity IDs (student IDs, etc.) - @Column('enum', { enum: ['pending', 'queued', 'calling', 'completed', 'failed', 'cancelled'] }) - status: string; + @Column('int', { default: 0 }) + totalRecipients: number; // Total phone numbers uploaded - @Column('varchar', { length: 255, nullable: true }) - yemotCallId: string; // External call ID from Yemot + @Column('enum', { + enum: ['pending', 'uploading_phones', 'ready', 'running', 'completed', 'failed', 'cancelled'], + default: 'pending' + }) + status: string; @Column('simple-json', { nullable: true }) - callResult: Record; // Response from Yemot API + campaignReport: Record; // Downloaded report from Yemot @Column('text', { nullable: true }) errorMessage: string; @Column('datetime', { nullable: true }) - initiatedAt: Date; + startedAt: Date; @Column('datetime', { nullable: true }) completedAt: Date; @@ -232,40 +274,40 @@ export class PhoneCallHistory implements IHasUserId { **Design Questions:** -**Q3: Should we track call attempts or just final results?** -- **Option A**: Single record per call attempt ✅ RECOMMENDED - - Pro: Simpler schema - - Con: Lost history if retry -- **Option B**: Separate attempts table - - Pro: Full audit trail - - Con: More complex queries -- **Recommendation**: Start with Option A, consider B if retry logic is needed +**Q3: Should we store individual call results?** +- **Option A**: Store only campaign-level summary ✅ RECOMMENDED for MVP + - Pro: Simpler schema, less data + - Con: No per-call details without downloading report +- **Option B**: Extract and store per-call results from campaign report + - Pro: Queryable call details + - Con: Larger database, complex parsing + - Implementation: Parse `DownloadCampaignReport` response -**Q4: Should we support scheduled calls?** +**Q4: Should we support scheduled campaigns?** - **Option A**: Immediate execution only (MVP) - Pro: Simpler implementation - Con: Limited functionality -- **Option B**: Support scheduled time ✅ RECOMMENDED for Phase 2 - - Pro: More flexible (e.g., call during business hours) - - Con: Requires job scheduler (consider Bull, node-cron) - - Implementation: Add `scheduledFor: Date` column, use NestJS Bull Queue +- **Option B**: Support scheduled campaigns via Yemot ✅ RECOMMENDED for Phase 2 + - Pro: Leverage Yemot's `ScheduleCampaign` API + - Con: Additional API calls and tracking + - Implementation: Add `scheduledFor: Date` column, use `ScheduleCampaign` API ---- +**Recommendation**: Start with Option A for both, implement Option B in Phase 2 ## 5. Backend Implementation ### 5.1 Entity Module Configuration -**File**: `/server/src/entity-modules/phone-message.config.ts` +**File**: `/server/src/entity-modules/yemot-campaign-template.config.ts` ```typescript -import { PhoneMessage } from '@/db/entities/PhoneMessage.entity'; +import { YemotCampaignTemplate } from '@/db/entities/YemotCampaignTemplate.entity'; import { getBaseEntityConfig } from '@shared/base-entity/base-entity-config'; -export default getBaseEntityConfig(PhoneMessage, { +export default getBaseEntityConfig(YemotCampaignTemplate, { dto: { - create: CreatePhoneMessageDto, - update: UpdatePhoneMessageDto, + create: CreateYemotCampaignTemplateDto, + update: UpdateYemotCampaignTemplateDto, }, permissions: { list: 'admin', @@ -277,132 +319,283 @@ export default getBaseEntityConfig(PhoneMessage, { }); ``` -### 5.2 Phone Call Service +### 5.2 Campaign Service -**File**: `/server/src/services/phone-call.service.ts` +**File**: `/server/src/services/yemot-campaign.service.ts` **Key Responsibilities:** -1. Validate phone message template -2. Prepare message content (resolve variables, validate audio files) -3. Call Yemot API to initiate outbound call -4. Create call history records -5. Handle bulk call operations +1. Manage campaign template lifecycle (create, sync, update, delete) +2. Upload phone lists to Yemot templates +3. Trigger campaign execution +4. Poll campaign status +5. Download and store campaign reports **Core Methods:** ```typescript @Injectable() -export class PhoneCallService { +export class YemotCampaignService { constructor( - @InjectRepository(PhoneMessage) private phoneMessageRepo: Repository, - @InjectRepository(PhoneCallHistory) private callHistoryRepo: Repository, - private yemotApiService: YemotApiService, + @InjectRepository(YemotCampaignTemplate) private templateRepo: Repository, + @InjectRepository(YemotCampaign) private campaignRepo: Repository, + private yemotTemplatesApiService: YemotTemplatesApiService, ) {} - async initiateCall(params: InitiateCallDto): Promise; - async bulkInitiateCalls(params: BulkInitiateCallDto): Promise; - async getCallStatus(callId: number): Promise; - async cancelCall(callId: number): Promise; + // Template Management + async createTemplate(dto: CreateTemplateDto): Promise; + async syncTemplatesFromYemot(userId: number): Promise; + async updateTemplate(id: number, dto: UpdateTemplateDto): Promise; + async deleteTemplate(id: number): Promise; + + // Campaign Execution + async runCampaign(params: RunCampaignDto): Promise; + async uploadPhoneList(templateId: number, phoneNumbers: string[]): Promise; + + // Monitoring + async getCampaignStatus(campaignId: number): Promise; + async downloadCampaignReport(campaignId: number): Promise; + async cancelCampaign(campaignId: number): Promise; } ``` -### 5.3 Yemot API Integration +### 5.3 Yemot Templates API Integration -**File**: `/server/shared/utils/yemot/yemot-api.service.ts` +**File**: `/server/shared/utils/yemot/yemot-templates-api.service.ts` -**Design Questions:** +**Based on Yemot API Documentation** (חלק 2 ניהול קמפיינים) -**Q5: Which Yemot API endpoint should we use for outbound calls?** +**Available API Methods:** -Based on the Yemot API documentation (https://f2.freeivr.co.il), the relevant endpoints are: +| Yemot API Method | Purpose | Our Usage | +|------------------|---------|-----------| +| `GetTemplates` | Get all campaign templates | Sync/import existing templates | +| `CreateTemplate` | Create new campaign template | Create template from app | +| `UpdateTemplate` | Update template configuration | Modify template settings | +| `DeleteTemplate` | Delete template | Remove template | +| `UploadPhoneList` | Upload phone numbers to template | Bulk add recipients | +| `GetTemplateEntries` | Get phone list for template | View recipients | +| `UpdateTemplateEntry` | Update single phone number | Edit recipient | +| `ClearTemplateEntries` | Clear all phone numbers | Reset recipients | +| `RunCampaign` | Execute campaign | Trigger calls | +| `GetCampaignStatus` | Get campaign execution status | Monitor progress | +| `DownloadCampaignReport` | Download campaign report | Get results | +| `GetActiveCampaigns` | Get currently running campaigns | Monitor active | +| `CampaignAction` | Control campaign (pause/resume/stop) | Manage execution | +| `ScheduleCampaign` | Schedule campaign for later | Future: scheduled calls | -- **Option A**: `ApiOutCallV2` - Modern outbound call API ✅ RECOMMENDED - - Pro: Latest API, better features - - Con: May require specific Yemot account level - - **Parameters**: - - `token`: Authentication token - - `phone`: Target phone number - - `text` OR `file`: Message content (TTS or audio file URL) - - `CallerID`: Display number (optional) - - `params`: Additional parameters +**Design Questions:** -- **Option B**: `ApiOutCall` - Legacy outbound call API - - Pro: Broader compatibility - - Con: Older, potentially fewer features +**Q5: Which Yemot API approach should we use?** + +- **Option A**: Use Templates + RunCampaign API ✅ RECOMMENDED + - Pro: Aligns with Yemot's campaign system, reusable templates + - Con: Requires template creation/management + - **APIs Used**: + - `CreateTemplate` - Create template in Yemot + - `UploadPhoneList` - Add phone numbers + - `RunCampaign` - Execute campaign + - `GetCampaignStatus` - Monitor progress + - `DownloadCampaignReport` - Get results + +- **Option B**: Use generic outbound call API (deprecated approach) + - Pro: Simpler for one-off calls + - Con: Not aligned with user's requirement to focus on Templates/Campaigns + - **Not recommended based on user feedback** + +**Q6: How should we manage template content (audio, IVR flow)?** + +- **Option A**: Manage in Yemot web interface ✅ RECOMMENDED for Phase 1 + - Pro: Leverage Yemot's robust content management UI + - Con: Split workflow (create template in app, configure in Yemot) + - Implementation: Create template shell via API, user configures in Yemot + +- **Option B**: Manage content via API + - Pro: Fully integrated workflow + - Con: Complex implementation with file uploads + - APIs: `UploadFile`, `GetIVR2Dir`, `FileAction`, etc. + - **Recommended for Phase 2** + +- **Option C**: Hybrid - basic TTS via API, advanced via Yemot UI + - Pro: Best of both worlds + - Con: Most complex + - Implementation: Support `text` parameter in `CreateTemplate`, advanced features in Yemot -- **Option C**: `ApiSendMultipleMessage` - Bulk message API - - Pro: Optimized for bulk operations - - Con: May have rate limits +**Q7: How should we handle campaign reports?** -**Recommendation**: Use `ApiOutCallV2` with fallback to `ApiOutCall` if needed. +- **Option A**: Download and parse full report ✅ RECOMMENDED + - Pro: Complete data, queryable locally + - Con: Larger storage, parsing complexity + - Implementation: Use `DownloadCampaignReport` API, parse CSV/JSON -**Q6: How should we handle audio file hosting?** +- **Option B**: Store only campaign status + - Pro: Minimal storage + - Con: No detailed per-call data + - Implementation: Use `GetCampaignStatus` only -- **Option A**: Upload files to Yemot storage ✅ RECOMMENDED for Phase 1 - - Pro: Integrated with Yemot, no external hosting needed - - Con: Requires additional API call, file management - - Yemot API: `ApiUploadFile` +**Recommendation**: Use Option A for API approach, Option A for content (Phase 1), Option A for reports -- **Option B**: Host files on application server with public URL - - Pro: Full control over files - - Con: Requires public-facing endpoint, CDN consideration - - Implementation: Serve from `/server/public/audio/` with proper access controls +**Implementation Approach:** -- **Option C**: Use cloud storage (S3, Google Cloud Storage) - - Pro: Scalable, reliable - - Con: Additional infrastructure and costs +```typescript +@Injectable() +export class YemotTemplatesApiService { + private readonly baseUrl = 'https://www.call2all.co.il/ym/api'; + + constructor( + private httpService: HttpService, + private configService: ConfigService, + ) {} -**Recommendation**: Start with Option A (Yemot storage), plan for Option C if scale demands it. + /** + * Get all templates from Yemot + * API: GetTemplates + */ + async getTemplates(token: string): Promise { + const response = await this.httpService.post(`${this.baseUrl}/GetTemplates`, { + token, + }).toPromise(); + + return this.parseTemplatesResponse(response.data); + } -**Q7: How should we handle Yemot authentication tokens?** + /** + * Create new campaign template + * API: CreateTemplate + */ + async createTemplate(token: string, params: CreateTemplateParams): Promise { + const response = await this.httpService.post(`${this.baseUrl}/CreateTemplate`, { + token, + templateName: params.name, + text: params.text || '', // TTS text (optional) + // Additional parameters as needed + }).toPromise(); + + // Returns template ID + return this.parseTemplateId(response.data); + } -- **Option A**: Single system-wide token in environment variables - - Pro: Simple configuration - - Con: All calls from same account, single point of failure + /** + * Upload phone numbers to template + * API: UploadPhoneList + */ + async uploadPhoneList( + token: string, + templateId: string, + phoneNumbers: string[] + ): Promise { + // Create text file content with phone numbers + const phoneListContent = phoneNumbers.join('\n'); + + const response = await this.httpService.post(`${this.baseUrl}/UploadPhoneList`, { + token, + template: templateId, + fileContent: Buffer.from(phoneListContent).toString('base64'), + // Options: 'add' or 'replace' + mode: 'replace', + }).toPromise(); + + this.validateResponse(response.data); + } -- **Option B**: User-specific tokens (stored per user) ✅ RECOMMENDED - - Pro: Better isolation, per-organization billing - - Con: More complex configuration, requires UI for token management - - Implementation: Add `yemotToken` field to User entity, encrypted storage + /** + * Run campaign + * API: RunCampaign + */ + async runCampaign(token: string, templateId: string): Promise { + const response = await this.httpService.post(`${this.baseUrl}/RunCampaign`, { + token, + template: templateId, + }).toPromise(); + + // Returns campaign ID + return this.parseCampaignId(response.data); + } -- **Option C**: Centralized token management service - - Pro: Best security, token rotation support - - Con: Over-engineered for MVP + /** + * Get campaign status + * API: GetCampaignStatus + */ + async getCampaignStatus( + token: string, + campaignId: string + ): Promise { + const response = await this.httpService.post(`${this.baseUrl}/GetCampaignStatus`, { + token, + campaign: campaignId, + }).toPromise(); + + return this.parseCampaignStatus(response.data); + } -**Recommendation**: Start with Option A for MVP, migrate to Option B for production. + /** + * Download campaign report + * API: DownloadCampaignReport + */ + async downloadCampaignReport( + token: string, + campaignId: string + ): Promise { + const response = await this.httpService.post(`${this.baseUrl}/DownloadCampaignReport`, { + token, + campaign: campaignId, + }).toPromise(); + + return this.parseReport(response.data); + } -**Implementation Approach:** + /** + * Delete template + * API: DeleteTemplate + */ + async deleteTemplate(token: string, templateId: string): Promise { + const response = await this.httpService.post(`${this.baseUrl}/DeleteTemplate`, { + token, + template: templateId, + }).toPromise(); + + this.validateResponse(response.data); + } +} +``` + +**TypeScript Interfaces:** ```typescript -@Injectable() -export class YemotApiService { - constructor(private httpService: HttpService) {} - - async initiateOutboundCall(params: YemotOutboundCallParams): Promise { - const endpoint = 'https://www.call2all.co.il/ym/api/ApiOutCallV2'; - const payload = { - token: params.token, - phone: params.phoneNumber, - ...(params.messageType === 'tts' - ? { text: params.textContent } - : { file: params.audioFileUrl }), - CallerID: params.callerId || undefined, - }; - - try { - const response = await this.httpService.post(endpoint, payload).toPromise(); - return this.parseYemotResponse(response.data); - } catch (error) { - throw new YemotApiException(`Failed to initiate call: ${error.message}`); - } - } +export interface YemotTemplate { + id: string; + name: string; + status: string; + phoneCount: number; + // Additional fields from API +} - async uploadAudioFile(token: string, filePath: string): Promise { - // Implementation for ApiUploadFile - } +export interface CreateTemplateParams { + name: string; + text?: string; // TTS text + // Additional configuration +} - async getCallStatus(token: string, callId: string): Promise { - // Implementation for checking call status - } +export interface CampaignStatus { + status: 'running' | 'completed' | 'paused' | 'stopped'; + totalCalls: number; + completedCalls: number; + successfulCalls: number; + failedCalls: number; + // Additional status fields +} + +export interface CampaignReport { + campaignId: string; + totalNumbers: number; + calls: CampaignCallDetail[]; + summary: CampaignSummary; +} + +export interface CampaignCallDetail { + phoneNumber: string; + status: string; + duration: number; + timestamp: Date; + // Additional call details } ``` @@ -410,81 +603,96 @@ export class YemotApiService { ## 6. Frontend Implementation -### 6.1 Phone Message Template Management +### 6.1 Campaign Template Management -**File**: `/client/src/entities/phone-message.jsx` +**File**: `/client/src/entities/yemot-campaign-template.jsx` **UI Components:** -1. **List View**: Display all phone message templates with status -2. **Create Form**: Create new template with validation -3. **Edit Form**: Modify existing template -4. **Preview**: Test message content (TTS or play audio) +1. **List View**: Display all campaign templates with sync status +2. **Create Form**: Create new template (creates in Yemot via API) +3. **Import Button**: Sync existing templates from Yemot +4. **Edit Form**: Modify template metadata (actual content managed in Yemot) +5. **Sync Button**: Re-sync template data from Yemot **Key Fields:** -- Name (unique identifier) +- Name (local friendly name) - Description -- Message Type (TTS / Audio File) -- Content (text or file upload) +- Yemot Template ID (auto-populated after creation) +- Yemot Template Name - Active status toggle -- Variable placeholders help text +- Sync status indicator +- Last synced timestamp +- Link to Yemot UI for content configuration -**Design Pattern**: Follow existing `text.jsx` entity pattern with Material-UI components. +**Design Pattern**: Follow existing entity patterns with Material-UI components, similar to `text.jsx` ### 6.2 Bulk Action Button -**File**: `/client/src/components/bulk-actions/BulkPhoneCallButton.jsx` +**File**: `/client/src/components/bulk-actions/BulkCampaignButton.jsx` **User Workflow:** 1. User selects students from list (e.g., student-klasses-report) -2. Clicks "Send Phone Call" bulk action button +2. Clicks "Send Campaign" bulk action button 3. Modal opens with: - - Dropdown to select phone message template - - Preview of message + - Dropdown to select campaign template - Phone number source field selection (e.g., student phone, parent phone) + - Preview of recipient count - Confirmation button -4. System validates and queues calls -5. Success notification with summary +4. System: + - Extracts phone numbers from selected students + - Uploads phone list to Yemot template via API + - Triggers campaign execution via RunCampaign + - Creates campaign history record +5. Success notification with campaign tracking link **Implementation:** ```jsx -export const BulkPhoneCallButton = () => { +export const BulkCampaignButton = () => { const { selectedIds } = useListContext(); const [selectedTemplate, setSelectedTemplate] = useState(null); - const { data: templates } = useGetList('phone_message', { - filter: { isActive: true } + const { data: templates } = useGetList('yemot_campaign_template', { + filter: { isActive: true, isSynced: true } }); - const handleSendCalls = async () => { - // Call dataProvider to trigger bulk phone calls - await dataProvider.create('phone_call_history/bulk', { + const handleRunCampaign = async (values) => { + // Call dataProvider to trigger campaign + await dataProvider.create('yemot_campaign/run', { data: { - phoneMessageId: selectedTemplate.id, + campaignTemplateId: values.templateId, recipientIds: selectedIds, - recipientType: 'student', + recipientSource: 'student_klasses_report', + phoneNumberField: values.phoneNumberField, } }); }; return ( } - mutate={handleSendCalls} - // ... additional props + label="שליחת קמפיין טלפוני" + icon={} + mutate={handleRunCampaign} + dialogTitle="בחירת תבנית קמפיין" > + + {selectedIds.length} נמענים נבחרו + ); }; @@ -493,51 +701,94 @@ export const BulkPhoneCallButton = () => { **Integration Points:** - Add to `student-klasses-report.jsx` additionalBulkButtons - Add to `student-klass.jsx` additionalBulkButtons -- Consider adding to `student.jsx` for individual calls +- Consider adding to `student.jsx` for individual campaigns -### 6.3 Call History Viewer +### 6.3 Campaign History Viewer -**File**: `/client/src/entities/phone-call-history.jsx` +**File**: `/client/src/entities/yemot-campaign.jsx` **Display Information:** -- Call timestamp -- Recipient details -- Message template used -- Status (with color coding) -- Error messages (if failed) -- Filters: by date, status, recipient, message template +- Campaign timestamp +- Template used +- Status (with color coding: running, completed, failed) +- Total recipients +- Completed calls count +- Success rate +- Campaign report download button +- Filters: by date, status, template **Actions:** -- Retry failed calls -- Cancel pending calls -- Export call log - ---- +- View campaign report (opens modal with detailed results) +- Download report as CSV +- Cancel running campaign +- Re-run campaign with same recipients + +**Report Display:** +Shows parsed campaign report data: +- Total calls made +- Success/failure breakdown +- Per-call details (phone number, status, duration) +- Interactive charts (success rate, call distribution) ## 7. API Endpoints -### 7.1 Phone Message Templates +### 7.1 Campaign Templates | Method | Endpoint | Description | Permissions | |--------|----------|-------------|-------------| -| GET | `/api/phone_message` | List templates | admin | -| GET | `/api/phone_message/:id` | Get template details | admin | -| POST | `/api/phone_message` | Create template | admin | -| PATCH | `/api/phone_message/:id` | Update template | admin | -| DELETE | `/api/phone_message/:id` | Delete template | admin | +| GET | `/api/yemot_campaign_template` | List templates | admin | +| GET | `/api/yemot_campaign_template/:id` | Get template details | admin | +| POST | `/api/yemot_campaign_template` | Create template (+ Yemot) | admin | +| POST | `/api/yemot_campaign_template/sync` | Sync from Yemot | admin | +| PATCH | `/api/yemot_campaign_template/:id` | Update template | admin | +| DELETE | `/api/yemot_campaign_template/:id` | Delete template (+ Yemot) | admin | -### 7.2 Phone Call Operations +### 7.2 Campaign Operations | Method | Endpoint | Description | Permissions | |--------|----------|-------------|-------------| -| POST | `/api/phone_call_history` | Initiate single call | admin | -| POST | `/api/phone_call_history/bulk` | Initiate bulk calls | admin | -| GET | `/api/phone_call_history` | List call history | admin | -| GET | `/api/phone_call_history/:id` | Get call details | admin | -| POST | `/api/phone_call_history/:id/retry` | Retry failed call | admin | -| POST | `/api/phone_call_history/:id/cancel` | Cancel pending call | admin | - ---- +| POST | `/api/yemot_campaign/run` | Run campaign | admin | +| GET | `/api/yemot_campaign` | List campaign history | admin | +| GET | `/api/yemot_campaign/:id` | Get campaign details | admin | +| GET | `/api/yemot_campaign/:id/status` | Get campaign status | admin | +| GET | `/api/yemot_campaign/:id/report` | Download campaign report | admin | +| POST | `/api/yemot_campaign/:id/cancel` | Cancel running campaign | admin | + +**Workflow Example:** + +1. **Create Template**: + ``` + POST /api/yemot_campaign_template + { + "name": "Welcome Message", + "description": "Welcome call to new students", + "text": "שלום, ברוכים הבאים" + } + Response: { id: 1, yemotTemplateId: "1234" } + ``` + +2. **Run Campaign**: + ``` + POST /api/yemot_campaign/run + { + "campaignTemplateId": 1, + "recipientIds": [10, 20, 30], + "recipientSource": "student_klasses_report", + "phoneNumberField": "phone" + } + Response: { id: 5, yemotCampaignId: "CAMP123", status: "pending" } + ``` + +3. **Check Status**: + ``` + GET /api/yemot_campaign/5/status + Response: { + status: "running", + totalCalls: 3, + completedCalls: 2, + successfulCalls: 2 + } + ``` ## 8. Security Considerations @@ -630,30 +881,46 @@ export const BulkPhoneCallButton = () => { **Changes Required:** - Ensure `phone` field exists and is accessible - Consider adding `parentPhone` field if not present -- Add opt-out flag: `phoneCallOptOut` (boolean) +- Add opt-out flag: `campaignOptOut` (boolean) ### 10.2 Student Klasses Report **Changes Required:** -- Add `BulkPhoneCallButton` to `additionalBulkButtons` array +- Add `BulkCampaignButton` to `additionalBulkButtons` array - Ensure phone numbers are loaded with report data **Example Integration:** ```jsx const additionalBulkButtons = [ , - , // NEW + , // NEW ]; ``` ### 10.3 App Registration **Changes Required:** -- Register `phone_message` resource in `/client/src/App.jsx` -- Register `phone_call_history` resource for admin users -- Add appropriate icons (PhoneIcon, CallIcon) -- Place in 'settings' menu group for templates -- Place in 'admin' menu group for call history +- Register `yemot_campaign_template` resource in `/client/src/App.jsx` +- Register `yemot_campaign` resource for admin users +- Add appropriate icons (CampaignIcon, PhoneIcon) +- Place templates in 'settings' menu group +- Place campaigns in 'admin' menu group for history + +**Example App.jsx Integration:** +```jsx +import yemotCampaignTemplate from 'src/entities/yemot-campaign-template'; +import yemotCampaign from 'src/entities/yemot-campaign'; +import CampaignIcon from '@mui/icons-material/Campaign'; + +// In admin section: + + +{isAdmin(permissions) && <> + +} +``` --- @@ -662,28 +929,33 @@ const additionalBulkButtons = [ ### 11.1 Unit Tests **Backend:** -- `PhoneCallService` methods -- Yemot API client mock responses -- Message template validation -- Variable substitution logic +- `YemotCampaignService` methods +- Yemot Templates API client mock responses +- Template creation/sync logic +- Phone list upload formatting +- Campaign report parsing **Frontend:** -- Phone message CRUD components -- Bulk action button rendering +- Campaign template CRUD components +- Bulk campaign button rendering - Form validation +- Template sync status display ### 11.2 Integration Tests -- End-to-end flow: Create template → Trigger bulk call → Verify history -- Yemot API integration (use sandbox/test account) -- Error handling scenarios +- End-to-end flow: Create template → Upload phones → Run campaign → Monitor status → Download report +- Yemot Templates API integration (use test account) +- Error handling scenarios (API failures, invalid templates) +- Campaign status polling ### 11.3 Manual Testing - Test with real Yemot account (use test phone numbers) -- Verify TTS quality and audio playback +- Verify template creation in Yemot +- Configure message content in Yemot UI +- Test campaign execution +- Verify report accuracy - Test all bulk action scenarios -- Verify call history accuracy --- @@ -693,9 +965,40 @@ const additionalBulkButtons = [ **Q10: How should we handle large bulk operations?** -- **Option A**: Synchronous processing (block until done) - - Pro: Simple, immediate feedback - - Con: Slow for large batches, request timeout risk +- **Option A**: Synchronous phone list upload + async campaign execution ✅ RECOMMENDED + - Pro: Simple, leverages Yemot's campaign execution + - Con: Upload may timeout for very large lists + - Implementation: Upload phone list synchronously (fast), Yemot runs campaign asynchronously + +- **Option B**: Job queue for phone list upload + campaign trigger + - Pro: Most robust, no timeouts + - Con: Requires Bull/BullMQ infrastructure + - Implementation: + ```typescript + @Processor('campaigns') + export class CampaignProcessor { + @Process('run-campaign') + async handleRunCampaign(job: Job) { + // Upload phones in batches + // Trigger campaign + // Poll for completion + } + } + ``` + +- **Option C**: Chunked upload with progress tracking + - Pro: Good UX for large lists + - Con: More complex implementation + - Implementation: Upload phones in chunks of 1000, show progress bar + +**Recommendation**: +- Phase 1: Option A (up to 1000 numbers) +- Phase 2: Option B or C for larger scale + +**Note**: Yemot's `UploadPhoneList` API handles the phone list upload efficiently. The campaign execution itself is handled asynchronously by Yemot, so our system only needs to: +1. Upload phone list (synchronous, but fast) +2. Trigger campaign (synchronous API call) +3. Poll for status (asynchronous background job or manual refresh) - **Option B**: Asynchronous with job queue ✅ RECOMMENDED - Pro: Fast response, no timeout, resilient @@ -770,52 +1073,59 @@ const additionalBulkButtons = [ **Timeline: 2-3 weeks** **Backend:** -- [ ] Create `PhoneMessage` entity and migration -- [ ] Create `PhoneCallHistory` entity and migration -- [ ] Implement `PhoneCallService` with basic operations -- [ ] Create Yemot API client service (TTS only) -- [ ] Add phone message entity module configuration -- [ ] Add bulk call endpoint +- [ ] Create `YemotCampaignTemplate` entity and migration +- [ ] Create `YemotCampaign` entity and migration +- [ ] Implement `YemotCampaignService` with basic operations +- [ ] Create `YemotTemplatesApiService` client +- [ ] Implement template creation via `CreateTemplate` API +- [ ] Implement phone list upload via `UploadPhoneList` API +- [ ] Implement campaign execution via `RunCampaign` API +- [ ] Implement status polling via `GetCampaignStatus` API +- [ ] Add entity module configurations **Frontend:** -- [ ] Create phone message CRUD UI (list, create, edit) -- [ ] Implement `BulkPhoneCallButton` component +- [ ] Create campaign template CRUD UI (list, create, edit) +- [ ] Implement template sync from Yemot (`GetTemplates`) +- [ ] Implement `BulkCampaignButton` component - [ ] Integrate button into `student-klasses-report` -- [ ] Create call history viewer (read-only) +- [ ] Create campaign history viewer (read-only) +- [ ] Display campaign status and results **Testing:** - [ ] Unit tests for services -- [ ] Integration test for bulk call flow -- [ ] Manual testing with Yemot sandbox +- [ ] Integration test for campaign flow +- [ ] Manual testing with Yemot account ### Phase 2: Enhanced Features **Timeline: 1-2 weeks** -- [ ] Audio file upload support -- [ ] Variable substitution in messages -- [ ] Retry logic for failed calls +- [ ] Campaign report download and parsing (`DownloadCampaignReport`) +- [ ] Detailed per-call results display +- [ ] Template content management via API (audio upload) - [ ] User-specific Yemot token management -- [ ] Call status webhook endpoint (if Yemot supports) +- [ ] Campaign cancellation (`CampaignAction`) +- [ ] Opt-out list management ### Phase 3: Advanced Features **Timeline: 2-3 weeks** -- [ ] Scheduled calls (job queue with Bull) -- [ ] Call templates with approval workflow -- [ ] Opt-out management UI +- [ ] Scheduled campaigns (`ScheduleCampaign` API) +- [ ] Job queue for large phone lists (Bull/BullMQ) +- [ ] Campaign templates approval workflow - [ ] Enhanced reporting and analytics -- [ ] Export call history to CSV +- [ ] Export campaign reports to CSV - [ ] Integration with additional entities (teachers, parents) +- [ ] Campaign retry for failed calls ### Phase 4: Production Hardening **Timeline: 1 week** -- [ ] Performance optimization +- [ ] Performance optimization (batch phone uploads) - [ ] Enhanced error handling -- [ ] Monitoring dashboard -- [ ] Load testing +- [ ] Monitoring dashboard for campaign metrics +- [ ] Load testing with large phone lists - [ ] Security audit -- [ ] Documentation +- [ ] Documentation (API docs, user guide) --- @@ -823,43 +1133,53 @@ const additionalBulkButtons = [ ### 15.1 Critical Decisions -**Q11: Yemot Account Setup** +**Q11: Yemot Account Setup** ⚠️ CRITICAL - Do we have an existing Yemot account with API access? - What is the account tier and associated rate limits? - Do we need separate accounts for staging/production? +- **NEW**: Does the account have Templates/Campaigns API access enabled? **Q12: User Workflow Priorities** -- Which entity lists should have bulk call buttons initially? +- Which entity lists should have bulk campaign buttons initially? - Student klasses report ✅ (specified in requirements) - Student list? - Teacher list (for staff notifications)? - Custom recipient lists? -**Q13: Message Content Validation** -- Should we validate message content (e.g., max length, forbidden words)? -- Should admins be able to preview/test messages before using in bulk? - +**Q13: Template Management Approach** +- Should we create templates via API or import existing templates from Yemot? +- Hybrid approach (both create and import)? +- Where should template content (audio/IVR) be configured: + - Yemot web interface (recommended for Phase 1)? + - Via API (Phase 2)? + **Q14: Billing & Cost Control** -- How should we track Yemot call costs? +- How should we track Yemot campaign costs? - Should we implement quota limits per user/organization? - Should we require additional approval for large bulk operations (e.g., >100 calls)? ### 15.2 Future Considerations -**Q15: Interactive Calls (IVR)** -- Current design focuses on one-way outbound messages -- Should we plan for interactive responses in the future? -- Would require webhook endpoint for Yemot to send responses back +**Q15: Campaign Monitoring** +- Should we implement real-time campaign status polling? +- Webhook endpoint for campaign completion notifications? +- How frequently should we check campaign status? **Q16: Multi-Language Support** -- Should TTS support multiple languages? +- Should TTS in templates support multiple languages? - Yemot supports Hebrew TTS by default - Consider adding language field to template if needed **Q17: Integration with Existing Yemot Functionality** - The system has existing inbound call handling (`YemotHandlerService`, chains) -- Should outbound calls be integrated with the existing call tracking? -- Current `YemotCall` entity tracks inbound calls - should outbound use the same or separate entity? +- Should outbound campaigns integrate with existing call tracking? +- Current `YemotCall` entity tracks inbound calls - keep separate for campaigns? +- **Recommendation**: Keep separate - campaigns are fundamentally different from interactive IVR calls + +**Q18: Template Synchronization** +- How often should we sync templates from Yemot? +- Manual sync only or automatic periodic sync? +- What happens if template is deleted in Yemot but referenced locally? --- @@ -920,9 +1240,23 @@ const additionalBulkButtons = [ ## 19. References ### 19.1 Yemot API Documentation -- Main API Documentation: https://f2.freeivr.co.il/topic/55/api-גישת-מפתחים-למערכות/5 -- Outbound Calls: (Follow links from main page to ApiOutCallV2 and ApiOutCall) -- File Upload: (Follow links from main page to ApiUploadFile) +- **Main API Documentation**: https://f2.freeivr.co.il/topic/55/api-גישת-מפתחים-למערכות/5 +- **Templates/Campaigns** (חלק 2 ניהול קמפיינים): + - `GetTemplates`: https://f2.freeivr.co.il/post/32033 + - `CreateTemplate`: https://f2.freeivr.co.il/post/32037 + - `UpdateTemplate`: https://f2.freeivr.co.il/post/32034 + - `DeleteTemplate`: https://f2.freeivr.co.il/post/32038 + - `UploadPhoneList`: https://f2.freeivr.co.il/post/32043 + - `GetTemplateEntries`: https://f2.freeivr.co.il/post/32039 + - `RunCampaign`: https://f2.freeivr.co.il/post/32044 + - `GetCampaignStatus`: https://f2.freeivr.co.il/post/32045 + - `DownloadCampaignReport`: https://f2.freeivr.co.il/post/32046 + - `GetActiveCampaigns`: https://f2.freeivr.co.il/post/32047 + - `CampaignAction`: https://f2.freeivr.co.il/post/32048 + - `ScheduleCampaign`: https://f2.freeivr.co.il/post/32049 +- **File Management**: + - `UploadFile`: https://f2.freeivr.co.il/post/32031 + - Audio file management: https://f2.freeivr.co.il/post/32035 ### 19.2 Existing Code Patterns - Entity Module Configuration: `/server/src/entity-modules/*.config.ts` @@ -942,47 +1276,48 @@ const additionalBulkButtons = [ ### Appendix A: Database Migration Script (Skeleton) ```sql --- Phone Message Templates -CREATE TABLE phone_messages ( +-- Yemot Campaign Templates +CREATE TABLE yemot_campaign_templates ( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, user_id INT NOT NULL, - name VARCHAR(100) NOT NULL UNIQUE, + name VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL, - message_type ENUM('tts', 'audio_file') NOT NULL, - text_content TEXT NULL, - audio_file_path VARCHAR(255) NULL, - is_active BOOLEAN DEFAULT FALSE, - yemot_instance_id VARCHAR(100) NULL, - additional_params JSON NULL, + yemot_template_id VARCHAR(50) NOT NULL, + yemot_template_name VARCHAR(100) NULL, + is_active BOOLEAN DEFAULT TRUE, + is_synced BOOLEAN DEFAULT FALSE, + template_config JSON NULL, + last_synced_at DATETIME NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, INDEX idx_user_id (user_id), - INDEX idx_name (name), - INDEX idx_is_active (is_active) + INDEX idx_yemot_template_id (yemot_template_id), + INDEX idx_is_active (is_active), + UNIQUE KEY unique_user_yemot_template (user_id, yemot_template_id) ); --- Phone Call History -CREATE TABLE phone_call_history ( +-- Yemot Campaigns +CREATE TABLE yemot_campaigns ( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, user_id INT NOT NULL, - phone_message_id INT UNSIGNED NOT NULL, - phone_number VARCHAR(20) NOT NULL, - recipient_name VARCHAR(100) NULL, - recipient_type VARCHAR(100) NULL, - recipient_reference_id INT NULL, - status ENUM('pending', 'queued', 'calling', 'completed', 'failed', 'cancelled') NOT NULL, - yemot_call_id VARCHAR(255) NULL, - call_result JSON NULL, + campaign_template_id INT UNSIGNED NOT NULL, + yemot_campaign_id VARCHAR(50) NOT NULL, + recipient_source VARCHAR(100) NULL, + recipient_ids JSON NULL, + total_recipients INT DEFAULT 0, + status ENUM('pending', 'uploading_phones', 'ready', 'running', 'completed', 'failed', 'cancelled') DEFAULT 'pending', + campaign_report JSON NULL, error_message TEXT NULL, - initiated_at DATETIME NULL, + started_at DATETIME NULL, completed_at DATETIME NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, INDEX idx_user_id (user_id), - INDEX idx_phone_message_id (phone_message_id), + INDEX idx_campaign_template_id (campaign_template_id), + INDEX idx_yemot_campaign_id (yemot_campaign_id), INDEX idx_status (status), INDEX idx_created_at (created_at), - FOREIGN KEY (phone_message_id) REFERENCES phone_messages(id) ON DELETE CASCADE + FOREIGN KEY (campaign_template_id) REFERENCES yemot_campaign_templates(id) ON DELETE CASCADE ); ``` @@ -990,7 +1325,7 @@ CREATE TABLE phone_call_history ( ```typescript // DTOs -export class CreatePhoneMessageDto { +export class CreateCampaignTemplateDto { @IsString() @MaxLength(100) name: string; @@ -999,79 +1334,138 @@ export class CreatePhoneMessageDto { @MaxLength(500) description: string; - @IsEnum(['tts', 'audio_file']) - messageType: 'tts' | 'audio_file'; - - @IsOptional() - @IsString() - textContent?: string; - @IsOptional() @IsString() - @MaxLength(255) - audioFilePath?: string; - - @IsBoolean() - @IsOptional() - isActive?: boolean; + text?: string; // TTS text for template (optional) } -export class BulkInitiateCallDto { +export class RunCampaignDto { @IsNumber() - phoneMessageId: number; + campaignTemplateId: number; @IsArray() @IsNumber({}, { each: true }) recipientIds: number[]; - @IsEnum(['student', 'teacher', 'parent']) - recipientType: 'student' | 'teacher' | 'parent'; - - @IsOptional() @IsString() - phoneNumberField?: string; // e.g., 'phone', 'parentPhone' + recipientSource: string; // e.g., 'student_klasses_report' + + @IsEnum(['phone', 'parentPhone']) + phoneNumberField: string; } -// Yemot API Types -export interface YemotOutboundCallParams { +// Yemot Templates API Types +export interface YemotTemplate { + id: string; + name: string; + status: string; + phoneCount: number; + lastModified: Date; +} + +export interface CreateTemplateParams { token: string; - phoneNumber: string; - messageType: 'tts' | 'audio_file'; - textContent?: string; - audioFileUrl?: string; - callerId?: string; + templateName: string; + text?: string; } -export interface YemotCallResponse { - success: boolean; - callId?: string; - errorMessage?: string; +export interface UploadPhoneListParams { + token: string; + templateId: string; + phoneNumbers: string[]; + mode: 'add' | 'replace'; +} + +export interface RunCampaignParams { + token: string; + templateId: string; +} + +export interface CampaignStatus { + campaignId: string; + status: 'running' | 'completed' | 'paused' | 'stopped'; + totalCalls: number; + completedCalls: number; + successfulCalls: number; + failedCalls: number; + averageDuration: number; +} + +export interface CampaignReport { + campaignId: string; + templateId: string; + startTime: Date; + endTime: Date; + totalNumbers: number; + calls: CampaignCallDetail[]; + summary: { + successful: number; + failed: number; + noAnswer: number; + busy: number; + totalDuration: number; + }; +} + +export interface CampaignCallDetail { + phoneNumber: string; + status: 'answered' | 'no_answer' | 'busy' | 'failed'; + duration: number; + timestamp: Date; + hangupCause?: string; +} } ``` ### Appendix C: Sample UI Mockups (Descriptions) -**Phone Message Template List:** +**Campaign Template List:** ``` -+--------------------------------------------------+ -| Phone Message Templates [+ New] | -+--------------------------------------------------+ -| Name | Type | Active | Actions| -|-------------------|-------------|--------|--------| -| Welcome Message | TTS | ✓ | Edit | -| Absence Reminder | Audio File | ✓ | Edit | -| Payment Due | TTS | ✗ | Edit | -+--------------------------------------------------+ ++---------------------------------------------------------------+ +| Campaign Templates [+ New] [Sync from Yemot]| ++---------------------------------------------------------------+ +| Name | Yemot ID | Synced | Active | Actions | +|-------------------|----------|--------|--------|--------------| +| Welcome Message | 1234 | ✓ | ✓ | Edit | View | +| Absence Reminder | 1235 | ✓ | ✓ | Edit | View | +| Payment Due | 1236 | ✓ | ✗ | Edit | View | ++---------------------------------------------------------------+ ``` -**Bulk Call Dialog:** +**Run Campaign Dialog:** ``` +-----------------------------------------------+ -| Send Phone Calls to 15 Students | +| Run Campaign for 15 Students | +-----------------------------------------------+ -| Select Message Template: | -| [▼ Welcome Message ] | +| Select Campaign Template: | +| [▼ Welcome Message (ID: 1234) ] | +| | +| Phone Number Field: | +| [▼ Student Phone ] | +| | +| Recipients: 15 students selected | | | +| Note: Campaign will be created in Yemot | +| and executed immediately. | +| | +| [Cancel] [Run Campaign]| ++-----------------------------------------------+ +``` + +**Campaign History:** +``` ++-----------------------------------------------------------------------+ +| Campaigns [Refresh]| ++-----------------------------------------------------------------------+ +| Template | Campaign ID | Status | Recipients | Success Rate | +|------------------|-------------|-----------|------------|--------------| +| Welcome Message | CAMP123 | Completed | 15 | 93% (14/15) | +| Absence Reminder | CAMP124 | Running | 25 | 60% (15/25) | +| Payment Due | CAMP125 | Failed | 10 | 0% | ++-----------------------------------------------------------------------+ +| Actions: [View Report] [Download CSV] [Cancel Campaign] | ++-----------------------------------------------------------------------+ +``` | Phone Number Field: | | [▼ Student Phone ] | | | @@ -1089,14 +1483,25 @@ export interface YemotCallResponse { | Version | Date | Author | Changes | |---------|------|--------|---------| | 1.0 | 2026-01-15 | GitHub Copilot | Initial HLD draft | +| 2.0 | 2026-01-15 | GitHub Copilot | **Major revision** - Shifted from generic outbound calls to Yemot Templates/Campaigns API approach per stakeholder feedback | + +**Version 2.0 Changes:** +- Updated to use Yemot Templates/Campaigns API (חלק 2 ניהול קמפיינים) +- Changed entities: `YemotCampaignTemplate` and `YemotCampaign` (instead of PhoneMessage/PhoneCallHistory) +- Workflow: Create template → Upload phone list → Run campaign → Monitor status → Download report +- Added template sync capability from Yemot +- Focused on campaign-based approach vs individual call approach +- Updated all API endpoints, services, and frontend components accordingly +- Added comprehensive Yemot Templates API method references --- -**Document Status**: Draft - Awaiting stakeholder review and decision on open questions. +**Document Status**: Draft v2.0 - Awaiting stakeholder review and decision on open questions. **Next Steps**: -1. Review and answer open questions (Q1-Q17) -2. Approve/modify database schema -3. Confirm Yemot API approach and obtain credentials -4. Approve implementation phases and timeline -5. Begin Phase 1 development +1. Review and answer open questions (Q1-Q18) +2. Confirm Yemot account has Templates/Campaigns API access +3. Approve/modify database schema +4. Obtain Yemot credentials and test API access +5. Approve implementation phases and timeline +6. Begin Phase 1 development