From aedb0143296f1d8805ff5bd216e25e3d60394489 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 15:34:10 +0000 Subject: [PATCH 1/5] Initial plan From b6833992b37e458ec3bd7de23a997ddc6ffc4f1e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 15:43:58 +0000 Subject: [PATCH 2/5] Add comprehensive HLD for Automated Phone Calls feature - Complete 715-line HLD document covering all aspects - Data models for PhoneTemplate and PhoneCampaign entities - Backend services (YemotApiService, PhoneTemplateService, PhoneCampaignService) - Frontend components and bulk action integration - Yemot API integration details - 8 design questions with alternatives and recommendations - Implementation phases (MVP, Enhanced, Advanced) - Security, testing, monitoring, and deployment considerations - 10 open questions for product owner - Summary document for quick reference Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-Automated-Phone-Calls.md | 715 ++++++++++++++++++++++++++++++ docs/HLD-Summary.md | 101 +++++ 2 files changed, 816 insertions(+) create mode 100644 docs/HLD-Automated-Phone-Calls.md create mode 100644 docs/HLD-Summary.md diff --git a/docs/HLD-Automated-Phone-Calls.md b/docs/HLD-Automated-Phone-Calls.md new file mode 100644 index 00000000..7bbfafab --- /dev/null +++ b/docs/HLD-Automated-Phone-Calls.md @@ -0,0 +1,715 @@ +# High-Level Design: Automated Phone Calls Feature + +## 1. Executive Summary + +This HLD describes a new feature for sending automated phone calls to students, parents, or other stakeholders using the Yemot phone system API. The feature will allow users to create reusable phone message templates and trigger them via bulk actions in the admin interface. + +**Key Terminology:** We will use the term **"Phone Message Template"** or **"Phone Template"** to refer to configurable phone message definitions. + +## 2. Feature Overview + +### 2.1 Core Capabilities +1. **Template Management**: Create, configure, and manage reusable phone message templates +2. **Message Configuration**: Define message content as text (TTS) or recorded audio files +3. **Bulk Actions**: Trigger phone campaigns from entity lists (e.g., student classes) +4. **Campaign Tracking**: Monitor and track phone call campaigns +5. **Integration**: Seamless integration with existing Yemot system infrastructure + +### 2.2 User Workflow +1. Admin creates a phone message template with configuration +2. Admin uploads or configures message content (text or audio file) +3. Admin navigates to an entity list (e.g., student classes report) +4. Admin selects records using checkboxes +5. Admin clicks bulk action button and selects a phone template +6. System triggers Yemot campaign to selected phone numbers +7. Admin can view campaign status and results + +## 3. Architecture Components + +### 3.1 Data Model + +#### 3.1.1 PhoneTemplate Entity (Backend) +**Location:** `/server/src/db/entities/PhoneTemplate.entity.ts` + +```typescript +@Entity() +export class PhoneTemplate { + @PrimaryGeneratedColumn() + id: number; + + @Column() + userId: number; + + @ManyToOne(() => User) + user: User; + + @Column({ length: 100 }) + name: string; // Template identifier/name + + @Column({ length: 500 }) + description: string; // Human-readable description + + @Column({ nullable: true }) + yemotTemplateId: string; // Yemot campaign template ID + + @Column({ type: 'enum', enum: ['text', 'file'] }) + messageType: 'text' | 'file'; // TTS text or audio file + + @Column({ type: 'text', nullable: true }) + messageText: string; // TTS text content + + @Column({ length: 255, nullable: true }) + messageFilePath: string; // Path to audio file in Yemot system + + @Column({ default: false }) + isActive: boolean; // Enable/disable template + + @Column({ nullable: true }) + callerId: string; // Outgoing caller ID + + @Column('simple-json', nullable: true }) + settings: PhoneTemplateSettings; // Additional settings + + @CreateDateColumn() + createdAt: Date; + + @UpdateDateColumn() + updatedAt: Date; +} + +interface PhoneTemplateSettings { + maxRetries?: number; + retryDelay?: number; + timeWindow?: { start: string; end: string }; + // Future extensibility +} +``` + +#### 3.1.2 PhoneCampaign Entity (Backend) +**Location:** `/server/src/db/entities/PhoneCampaign.entity.ts` + +```typescript +@Entity() +export class PhoneCampaign { + @PrimaryGeneratedColumn() + id: number; + + @Column() + userId: number; + + @ManyToOne(() => User) + user: User; + + @Column() + phoneTemplateId: number; + + @ManyToOne(() => PhoneTemplate) + phoneTemplate: PhoneTemplate; + + @Column({ nullable: true }) + yemotCampaignId: string; // Yemot campaign execution ID + + @Column({ type: 'enum', enum: ['pending', 'running', 'completed', 'failed', 'cancelled'] }) + status: string; + + @Column({ type: 'int', default: 0 }) + totalPhones: number; + + @Column({ type: 'int', default: 0 }) + successfulCalls: number; + + @Column({ type: 'int', default: 0 }) + failedCalls: number; + + @Column('simple-json') + phoneNumbers: PhoneEntry[]; // Phone numbers in campaign + + @Column({ type: 'text', nullable: true }) + errorMessage: string; + + @CreateDateColumn() + createdAt: Date; + + @UpdateDateColumn() + updatedAt: Date; + + @Column({ nullable: true }) + completedAt: Date; +} + +interface PhoneEntry { + phone: string; + name?: string; + metadata?: Record; +} +``` + +### 3.2 Backend Components + +#### 3.2.1 YemotApiService +**Location:** `/server/src/yemot/yemot-api.service.ts` + +New service to interact with Yemot REST API for campaigns/templates. + +**Key Methods:** +- `createTemplate(userId, description)` - Create Yemot template +- `uploadAudioFile(templateId, filePath, audioData)` - Upload audio file +- `updateTemplateSettings(templateId, settings)` - Update template configuration +- `uploadPhoneList(templateId, phones)` - Upload phone numbers +- `runCampaign(templateId, phones, options)` - Execute campaign +- `getCampaignStatus(campaignId)` - Get campaign status +- `downloadCampaignReport(campaignId)` - Get campaign results + +**Authentication:** Uses API KEY token stored in environment variables per user. + + +#### 3.2.2 PhoneTemplateService +**Location:** `/server/src/phone-template/phone-template.service.ts` + +Business logic for managing phone templates. + +**Key Methods:** +- `createTemplate(data)` - Create template and Yemot template +- `configureMessage(templateId, messageConfig)` - Set up message content +- `syncWithYemot(templateId)` - Sync local template with Yemot +- `validateTemplate(templateId)` - Validate template is ready to use + +#### 3.2.3 PhoneCampaignService +**Location:** `/server/src/phone-campaign/phone-campaign.service.ts` + +Business logic for executing and tracking campaigns. + +**Key Methods:** +- `executeCampaign(templateId, phoneNumbers, metadata)` - Start campaign +- `trackCampaignStatus(campaignId)` - Update campaign status +- `getCampaignResults(campaignId)` - Fetch results from Yemot +- `cancelCampaign(campaignId)` - Cancel running campaign + +#### 3.2.4 Entity Modules +- `/server/src/entity-modules/phone-template.config.ts` +- `/server/src/entity-modules/phone-campaign.config.ts` + +Standard CRUD configuration following existing patterns. + +### 3.3 Frontend Components + +#### 3.3.1 Phone Template Entity +**Location:** `/client/src/entities/phone-template.jsx` + +React-Admin resource for managing templates. + +**Features:** +- List view with filters (name, messageType, isActive) +- Create/Edit forms with message configuration +- File upload for audio messages +- Test campaign button +- Sync with Yemot button + +#### 3.3.2 Phone Campaign Entity +**Location:** `/client/src/entities/phone-campaign.jsx` + +React-Admin resource for viewing campaign history. + +**Features:** +- List view with filters (status, date range, template) +- Show view with detailed results +- Real-time status updates +- Download report button + +#### 3.3.3 Bulk Action Button Component +**Location:** `/client/src/components/PhoneTemplateBulkButton.jsx` + +Reusable bulk action button for triggering campaigns. + +**Props:** +- `entityType` - Type of entity being acted upon +- `phoneNumberExtractor` - Function to extract phone numbers from records +- `metadataExtractor` - Optional function to extract additional data + +**Behavior:** +1. Opens dialog showing available active phone templates +2. User selects template +3. Confirms action with phone count +4. Triggers API call to execute campaign +5. Shows success/error notification + +#### 3.3.4 Integration Example +**Location:** `/client/src/entities/student-klasses-report.jsx` + +Add bulk button to existing entity: + +```jsx +import PhoneTemplateBulkButton from 'src/components/PhoneTemplateBulkButton'; + +const additionalBulkButtons = [ + record.student?.phoneNumber} + metadataExtractor={(record) => ({ + studentName: record.student?.name, + klassName: record.klass?.name + })} + /> +]; +``` + +### 3.4 API Endpoints + +#### 3.4.1 Phone Templates +- `GET /api/phone-template` - List templates +- `POST /api/phone-template` - Create template +- `GET /api/phone-template/:id` - Get template details +- `PATCH /api/phone-template/:id` - Update template +- `DELETE /api/phone-template/:id` - Delete template +- `POST /api/phone-template/:id/upload-audio` - Upload audio file +- `POST /api/phone-template/:id/sync` - Sync with Yemot +- `POST /api/phone-template/:id/test` - Send test call + +#### 3.4.2 Phone Campaigns +- `GET /api/phone-campaign` - List campaigns +- `POST /api/phone-campaign/execute` - Execute new campaign +- `GET /api/phone-campaign/:id` - Get campaign details +- `POST /api/phone-campaign/:id/refresh-status` - Update from Yemot +- `POST /api/phone-campaign/:id/cancel` - Cancel campaign +- `GET /api/phone-campaign/:id/report` - Download report + +## 4. Yemot API Integration + +### 4.1 Required Yemot API Methods + +#### 4.1.1 Template Management +- **CreateTemplate** - Create new campaign template + - Returns: `templateId` +- **GetTemplates** - List all templates + - Returns: Array of template configurations +- **UpdateTemplate** - Update template settings +- **DeleteTemplate** - Remove template (if needed) + +#### 4.1.2 Content Management +- **UploadFile** - Upload audio files to Yemot + - Endpoint: `ivr2:` path format + - Supports: WAV files, automatic conversion + - Template file naming: `${templateId}.wav` + +#### 4.1.3 Campaign Execution +- **UploadPhoneList** - Upload phone numbers to template + - Format: JSON or colon-separated + - Supports: name, moreinfo, blocked status +- **RunCampaign** - Execute campaign + - Parameters: templateId, phones, callerId, options + - Returns: `campaignId` + - Options: TTS mode, SMS hybrid, scheduled execution + +#### 4.1.4 Campaign Monitoring +- **GetCampaignStatus** - Get real-time status + - Returns: status, progress, statistics +- **GetActiveCampaigns** - List running campaigns +- **CampaignAction** - Control running campaigns (pause/resume/stop) +- **DownloadCampaignReport** - Get detailed results +- **GetTransactions** - View campaign billing history + +### 4.2 Authentication Strategy + +**Recommended Approach: API KEY (Static Token)** + +**Rationale:** +- ✅ Persistent, never expires +- ✅ No MFA requirements +- ✅ Can restrict by IP, services, permissions +- ✅ Suitable for server-to-server communication +- ❌ Requires manual setup per user/organization + +**Implementation:** +1. Store API KEY in User entity or separate YemotConfig entity +2. Use environment variable for default/system-level key +3. Pass token in `authorization` header (recommended) or as `token` parameter + +**Alternative: Login Token (Dynamic)** +- Only if per-user authentication is required +- Requires MFA handling or IP whitelisting +- Adds complexity with token expiration + +### 4.3 File Storage Strategy + +**Option A: Direct Yemot Storage (Recommended)** +- Upload audio files directly to Yemot using UploadFile API +- Store only Yemot file path in PhoneTemplate entity +- Pros: No local storage needed, files managed by Yemot +- Cons: Requires Yemot API for all file operations + +**Option B: Hybrid Storage** +- Store files locally and sync to Yemot when template activated +- Keep local backup for redundancy +- Pros: Offline access, backup capability +- Cons: Increased storage requirements, sync complexity + +**Option C: Reference Existing Text/Audio System** +- Extend existing Text entity pattern with filepath field +- Leverage TextByUser view for per-user overrides +- Pros: Consistent with existing patterns, reusable infrastructure +- Cons: May not fit all use cases + + +## 5. Design Questions and Alternatives + +### 5.1 Template Creation Timing + +**Question:** When should the Yemot template be created? + +**Option A: On PhoneTemplate Creation (Recommended)** +- Create Yemot template immediately when PhoneTemplate is saved +- Store yemotTemplateId immediately +- Pros: Simple, predictable, ready to use +- Cons: Creates Yemot resources that may never be used + +**Option B: On First Use** +- Create Yemot template lazily when first campaign is executed +- Pros: Only creates resources when actually needed +- Cons: Added complexity, potential failure point during campaign execution + +**Option C: Manual Activation** +- Require explicit "Activate" action to create Yemot template +- Pros: User control, clear activation status +- Cons: Extra user step, potential confusion + +**Recommendation:** Option A for simplicity, with Option C as enhancement for advanced users. + +### 5.2 Message Content Management + +**Question:** How should message content (text/audio) be managed? + +**Option A: Embedded in PhoneTemplate (Recommended)** +- Store messageText and messageFilePath directly in PhoneTemplate entity +- Upload audio to Yemot using template naming convention +- Pros: Self-contained, simple to implement +- Cons: Less reusable across templates + +**Option B: Separate PhoneMessage Entity** +- Create separate entity for message content +- PhoneTemplate references PhoneMessage +- Pros: Reusable messages, versioning capability +- Cons: Added complexity, more entities to manage + +**Option C: Reuse Text Entity** +- Extend existing Text entity pattern +- Use TextByUser view for per-user overrides +- Pros: Consistent with existing system, leverages existing infrastructure +- Cons: Text entity may not fit all phone message needs + +**Option D: Hybrid Approach** +- Allow both embedded content AND reference to Text entity +- User chooses which to use +- Pros: Maximum flexibility +- Cons: Most complex, potential confusion + +**Recommendation:** Start with Option A for MVP, consider Option C for consistency if Text entity fits requirements well. + +### 5.3 Phone Number Extraction + +**Question:** How should phone numbers be extracted from entity records? + +**Option A: Configuration-Based (Recommended)** +- Define phone number field path in bulk button configuration +- Use accessor function: `phoneNumberExtractor={(record) => record.student.phoneNumber}` +- Pros: Flexible, works with any entity, explicit +- Cons: Requires configuration per entity + +**Option B: Convention-Based** +- Follow naming convention (e.g., look for `phoneNumber`, `phone`, `mobile` fields) +- Auto-detect phone numbers in records +- Pros: Less configuration, automatic +- Cons: May miss edge cases, less explicit + +**Option C: Entity-Specific Services** +- Create service methods per entity type +- E.g., `StudentKlassService.getPhoneNumbers(recordIds)` +- Pros: Business logic encapsulation, type-safe +- Cons: More code, harder to extend + +**Recommendation:** Option A for flexibility, with Option C as enhancement for complex cases. + +### 5.4 Campaign Execution Flow + +**Question:** Should campaigns be executed synchronously or asynchronously? + +**Option A: Asynchronous with Job Queue (Recommended)** +- Create PhoneCampaign record immediately +- Execute Yemot API call asynchronously +- Poll for status updates +- Pros: Non-blocking UI, handles failures gracefully +- Cons: Requires job queue or background worker + +**Option B: Synchronous Execution** +- Wait for Yemot API response before returning +- Pros: Simple, immediate feedback +- Cons: Slow UI, timeout risks, poor UX for large campaigns + +**Option C: Hybrid - Quick Validation Then Async** +- Validate template and phones synchronously +- Execute campaign asynchronously +- Pros: Fast failure feedback, non-blocking execution +- Cons: Medium complexity + +**Recommendation:** Option C for best UX, or Option A for production-grade implementation. + +### 5.5 Result Tracking + +**Question:** How granular should campaign result tracking be? + +**Option A: Summary Only (Recommended for MVP)** +- Track only aggregate statistics (total, success, failed) +- Store in PhoneCampaign entity +- Pros: Simple, low storage overhead +- Cons: No per-call details + +**Option B: Per-Call Tracking** +- Create PhoneCampaignCall entity for each call +- Store individual call results +- Pros: Detailed insights, troubleshooting capability +- Cons: High storage overhead, more complex + +**Option C: Summary + Error Details** +- Store aggregate stats plus failed call details +- Pros: Balance of simplicity and troubleshooting +- Cons: Medium complexity + +**Recommendation:** Option A for MVP, enhance to Option C based on user needs. + +### 5.6 Template vs Campaign Naming + +**Question:** What terminology should we use in the UI? + +**Option A: "Phone Template" + "Campaign" (Recommended)** +- Template = reusable configuration +- Campaign = specific execution instance +- Pros: Clear distinction, industry standard +- Cons: More concepts to explain + +**Option B: "Phone Message" + "Broadcast"** +- Message = content to send +- Broadcast = sending action +- Pros: Simpler language, user-friendly +- Cons: Less technical precision + +**Option C: "Call Template" + "Call Job"** +- Technical naming +- Pros: Precise, developer-friendly +- Cons: Less user-friendly + +**Recommendation:** Option A for clarity, use "Phone Message Template" in UI for user-friendliness. + +### 5.7 Multi-Tenancy Considerations + +**Question:** Should templates be global or per-user? + +**Option A: Per-User Templates (Recommended)** +- Each user has their own templates +- Managed via userId foreign key +- Pros: Isolation, security, aligns with existing pattern +- Cons: No template sharing + +**Option B: Global Templates with Permissions** +- System-level templates (userId = 0) +- User-level templates (userId > 0) +- Pros: Template reuse, admin control +- Cons: More complex permissions + +**Option C: Organization-Level Templates** +- Templates scoped to organization +- Requires organization entity +- Pros: Team collaboration +- Cons: Requires organization model + +**Recommendation:** Option A for MVP, enhance to Option B following Text entity pattern. + +### 5.8 Error Handling Strategy + +**Question:** How should Yemot API errors be handled? + +**Option A: Retry with Exponential Backoff** +- Automatically retry failed API calls +- Implement exponential backoff +- Pros: Resilient to transient failures +- Cons: Delayed error reporting + +**Option B: Fail Fast** +- Report errors immediately +- User manually retries +- Pros: Simple, clear failures +- Cons: Poor UX for temporary issues + +**Option C: Smart Retry** +- Retry on specific error codes (timeout, rate limit) +- Fail fast on others (auth, invalid data) +- Pros: Best UX, balanced approach +- Cons: Most complex + +**Recommendation:** Option C for production quality. + +## 6. Implementation Phases + +### Phase 1: MVP (Minimum Viable Product) +- PhoneTemplate entity with basic fields +- PhoneCampaign entity with status tracking +- YemotApiService with core methods +- Basic template CRUD in admin +- Single bulk action button implementation +- Text-based messages (TTS) only +- Summary-level campaign tracking + +### Phase 2: Enhanced Features +- Audio file upload support +- Advanced template settings +- Multiple bulk action integrations +- Real-time status updates +- Campaign report download + +### Phase 3: Advanced Features +- Scheduled campaigns +- Template sharing (global templates) +- Per-call result tracking +- Campaign analytics dashboard +- Message personalization (name insertion) +- Retry failed calls + +## 7. Security and Compliance + +### 7.1 Security Measures +- API KEY stored securely (encrypted environment variables) +- User-level template isolation +- Phone number validation and sanitization +- Rate limiting on campaign execution +- Audit logging for all operations + +### 7.2 Compliance Considerations +- GDPR: Phone numbers are personal data - ensure proper consent +- Do-not-call lists: Check against blocked numbers +- Time restrictions: Respect calling hours +- Opt-out mechanism: Provide way to unsubscribe + +### 7.3 Data Privacy +- Minimize phone number retention +- Encrypt phone numbers at rest +- Audit trail for phone number access +- Automatic cleanup of old campaign data + +## 8. Testing Strategy + +### 8.1 Unit Tests +- Entity validation +- Service business logic +- Phone number extraction +- Yemot API client (mocked) + +### 8.2 Integration Tests +- Full campaign execution flow +- Yemot API integration (sandbox) +- Error handling scenarios + +### 8.3 E2E Tests +- Template creation workflow +- Bulk action execution +- Campaign status tracking + +## 9. Monitoring and Observability + +### 9.1 Logging +- Campaign execution events +- Yemot API calls and responses +- Error conditions +- Performance metrics + +### 9.2 Metrics +- Campaign success rate +- Average campaign size +- API response times +- Error rates by type + +### 9.3 Alerts +- Campaign failures +- API rate limit approaching +- Unusual call volumes + +## 10. Migration and Deployment + +### 10.1 Database Migrations +- Create PhoneTemplate table +- Create PhoneCampaign table +- Add indexes for performance + +### 10.2 Configuration +- Environment variables for Yemot API credentials +- Feature flags for gradual rollout +- Default settings configuration + +### 10.3 Rollout Strategy +- Deploy backend entities and APIs first +- Test with admin users +- Add bulk buttons to selected entities +- Monitor and iterate +- Expand to all applicable entities + +## 11. Open Questions for Product Owner + +1. **Authentication**: Do we have Yemot API KEY for each user, or single system-level key? +2. **Billing**: Who pays for phone calls? Is there a budget/limit per user? +3. **Priority**: Which entity should get bulk action button first? (student-klasses-report?) +4. **Content**: Should we support both TTS and audio files in MVP, or TTS only? +5. **Scheduling**: Is scheduled campaign execution needed in MVP? +6. **Reporting**: What level of detail is needed in campaign reports? +7. **Permissions**: Should all users be able to create templates, or admin-only? +8. **Compliance**: Are there legal requirements for calling hours, opt-out, etc.? +9. **Integration**: Should this integrate with existing YemotCall entity for incoming calls? +10. **Naming**: Do we prefer "Phone Template", "Phone Message", or "Call Template" in UI? + +## 12. Dependencies and Risks + +### 12.1 External Dependencies +- Yemot API availability and stability +- Yemot account with sufficient units/credits +- Network connectivity to Yemot servers + +### 12.2 Technical Risks +- Yemot API rate limits or quotas +- Large campaign execution times +- Phone number data quality +- Audio file format compatibility + +### 12.3 Mitigation Strategies +- Implement robust error handling and retries +- Add campaign size limits +- Validate phone numbers before sending +- Test with small campaigns first +- Provide clear user feedback and status + +## 13. Success Criteria + +1. **Functionality**: Users can create templates and execute campaigns successfully +2. **Performance**: Campaigns with 100+ numbers execute within acceptable time +3. **Reliability**: 95%+ success rate for valid phone numbers +4. **Usability**: Users can complete workflow without training +5. **Adoption**: At least 50% of active users create phone templates + +## 14. Future Enhancements + +- SMS/voice hybrid campaigns +- Voice message recording in UI +- Campaign A/B testing +- Response tracking (key presses) +- Integration with calendar for automated campaigns +- Message templates with dynamic variables +- Multi-language support +- Voice mail detection and handling +- Custom retry logic per template +- Campaign cost estimation + +## 15. Conclusion + +This HLD provides a comprehensive design for the Automated Phone Calls feature, covering all architectural aspects from data models to API integration. The phased implementation approach allows for incremental delivery and validation. The design questions highlight key decision points that should be discussed with stakeholders before detailed implementation begins. + +**Next Steps:** +1. Review and approve HLD +2. Answer open questions +3. Create detailed implementation tickets +4. Set up Yemot API test environment +5. Begin Phase 1 implementation diff --git a/docs/HLD-Summary.md b/docs/HLD-Summary.md new file mode 100644 index 00000000..4ca397bd --- /dev/null +++ b/docs/HLD-Summary.md @@ -0,0 +1,101 @@ +# Automated Phone Calls - HLD Summary + +## Quick Reference + +**Full Document:** [HLD-Automated-Phone-Calls.md](./HLD-Automated-Phone-Calls.md) + +## Feature Overview + +Create reusable phone message templates and trigger automated phone campaigns via bulk actions using Yemot API. + +## Key Components + +### Entities +1. **PhoneTemplate** - Reusable message template configuration +2. **PhoneCampaign** - Campaign execution tracking + +### Services +1. **YemotApiService** - Yemot REST API integration +2. **PhoneTemplateService** - Template business logic +3. **PhoneCampaignService** - Campaign execution logic + +### Frontend +1. **phone-template.jsx** - Template management UI +2. **phone-campaign.jsx** - Campaign history UI +3. **PhoneTemplateBulkButton.jsx** - Reusable bulk action component + +## Yemot API Methods Required + +**Template Management:** +- CreateTemplate, GetTemplates, UpdateTemplate, DeleteTemplate + +**Content Management:** +- UploadFile (audio files) + +**Campaign Execution:** +- UploadPhoneList, RunCampaign + +**Campaign Monitoring:** +- GetCampaignStatus, GetActiveCampaigns, DownloadCampaignReport + +## Key Design Decisions (Recommended) + +1. **Template Creation Timing**: On PhoneTemplate creation (immediate) +2. **Message Content**: Embedded in PhoneTemplate entity +3. **Phone Extraction**: Configuration-based with accessor functions +4. **Execution Flow**: Hybrid - validate sync, execute async +5. **Result Tracking**: Summary only for MVP +6. **Terminology**: "Phone Template" + "Campaign" +7. **Multi-Tenancy**: Per-user templates +8. **Error Handling**: Smart retry (retry on specific errors) + +## Implementation Phases + +### Phase 1: MVP +- Basic entities and CRUD +- YemotApiService core methods +- Single bulk action button +- TTS messages only +- Summary tracking + +### Phase 2: Enhanced +- Audio file support +- Multiple integrations +- Real-time updates +- Report download + +### Phase 3: Advanced +- Scheduled campaigns +- Template sharing +- Per-call tracking +- Analytics dashboard +- Personalization + +## Open Questions for PO + +1. Do we have Yemot API KEY for each user or system-level? +2. Who pays for calls? Budget/limits per user? +3. Which entity gets bulk button first? +4. TTS only or audio files in MVP? +5. Scheduled campaigns needed in MVP? +6. Detail level for campaign reports? +7. Who can create templates - all users or admin only? +8. Legal requirements for calling hours, opt-out? +9. Integrate with existing YemotCall entity? +10. Preferred UI terminology? + +## Success Criteria + +- Users can create templates and execute campaigns +- Campaigns with 100+ numbers execute timely +- 95%+ success rate for valid numbers +- Users need no training +- 50%+ adoption rate + +## Next Steps + +1. Review and approve HLD +2. Answer open questions +3. Create implementation tickets +4. Set up Yemot API test environment +5. Begin Phase 1 implementation From 261716a40349b9e985948527abf777935f56dbde Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 15:46:49 +0000 Subject: [PATCH 3/5] Add architecture diagrams and documentation index for Phone Calls HLD - Comprehensive ASCII diagrams showing system architecture - Data flow diagrams for template creation and campaign execution - Component interaction matrix - Authentication and error handling flows - Deployment architecture visualization - README for docs directory with reading order and quick reference Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-Architecture-Diagram.md | 285 +++++++++++++++++++++++++++++++ docs/README.md | 103 +++++++++++ 2 files changed, 388 insertions(+) create mode 100644 docs/HLD-Architecture-Diagram.md create mode 100644 docs/README.md diff --git a/docs/HLD-Architecture-Diagram.md b/docs/HLD-Architecture-Diagram.md new file mode 100644 index 00000000..aaa32ffd --- /dev/null +++ b/docs/HLD-Architecture-Diagram.md @@ -0,0 +1,285 @@ +# Automated Phone Calls - Architecture Diagram + +## System Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ Frontend (React Admin) │ +├─────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ phone-template │ │ phone-campaign │ │ Student Klasses │ │ +│ │ .jsx │ │ .jsx │ │ Report │ │ +│ │ │ │ │ │ │ │ +│ │ - List/Create │ │ - View History │ │ - Bulk Actions │ │ +│ │ - Edit/Delete │ │ - Status Track │ │ - Select Records│ │ +│ │ - Upload Audio │ │ - Download Rpt │ │ │ │ +│ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │ +│ │ │ │ │ +│ └─────────────────────┼─────────────────────┘ │ +│ │ │ +│ ┌─────────────────────▼─────────────────────┐ │ +│ │ PhoneTemplateBulkButton.jsx │ │ +│ │ - Select Template │ │ +│ │ - Extract Phone Numbers │ │ +│ │ - Confirm & Execute │ │ +│ └─────────────────────┬─────────────────────┘ │ +└───────────────────────────────┬─┴──────────────────────────────────┘ + │ + API Calls │ (REST) + │ +┌───────────────────────────────▼──────────────────────────────────────┐ +│ Backend (NestJS) │ +├──────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌────────────────────────────────────────────────────────────────┐ │ +│ │ API Endpoints │ │ +│ │ /api/phone-template/* | /api/phone-campaign/* │ │ +│ └────────────┬────────────────────────┬──────────────────────────┘ │ +│ │ │ │ +│ ┌────────────▼────────────┐ ┌───────▼──────────────┐ │ +│ │ PhoneTemplateService │ │ PhoneCampaignService │ │ +│ │ │ │ │ │ +│ │ - createTemplate() │ │ - executeCampaign() │ │ +│ │ - configureMessage() │ │ - trackStatus() │ │ +│ │ - syncWithYemot() │ │ - getCampaignResults()│ │ +│ │ - validateTemplate() │ │ - cancelCampaign() │ │ +│ └────────────┬────────────┘ └───────┬──────────────┘ │ +│ │ │ │ +│ └───────────┬───────────┘ │ +│ │ │ +│ ┌───────────▼────────────┐ │ +│ │ YemotApiService │ │ +│ │ │ │ +│ │ - createTemplate() │ │ +│ │ - uploadAudioFile() │ │ +│ │ - uploadPhoneList() │ │ +│ │ - runCampaign() │ │ +│ │ - getCampaignStatus() │ │ +│ │ - downloadReport() │ │ +│ └───────────┬────────────┘ │ +│ │ │ +│ ┌────────────────────────┼──────────────────────────────┐ │ +│ │ Database (TypeORM) │ │ +│ │ │ │ │ +│ │ ┌─────────────────┐ │ ┌────────────────────────┐ │ │ +│ │ │ PhoneTemplate │ │ │ PhoneCampaign │ │ │ +│ │ ├─────────────────┤ │ ├────────────────────────┤ │ │ +│ │ │ id │ │ │ id │ │ │ +│ │ │ userId │ │ │ userId │ │ │ +│ │ │ name │ │ │ phoneTemplateId │ │ │ +│ │ │ description │ │ │ yemotCampaignId │ │ │ +│ │ │ yemotTemplateId │ │ │ status │ │ │ +│ │ │ messageType │ │ │ totalPhones │ │ │ +│ │ │ messageText │ │ │ successfulCalls │ │ │ +│ │ │ messageFilePath │ │ │ failedCalls │ │ │ +│ │ │ isActive │ │ │ phoneNumbers │ │ │ +│ │ │ callerId │ │ │ errorMessage │ │ │ +│ │ │ settings │ │ │ createdAt │ │ │ +│ │ │ createdAt │ │ │ completedAt │ │ │ +│ │ │ updatedAt │ │ └────────────────────────┘ │ │ +│ │ └─────────────────┘ │ │ +│ └──────────────────────────────────────────────────────┘ │ +└───────────────────────────────┬──────────────────────────────────────┘ + │ + │ HTTPS API Calls + │ (with API KEY) + │ +┌───────────────────────────────▼──────────────────────────────────────┐ +│ Yemot Phone System API │ +│ (https://www.call2all.co.il/ym/api/) │ +├──────────────────────────────────────────────────────────────────────┤ +│ │ +│ Template Management Campaign Execution │ +│ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ CreateTemplate │ │ UploadPhoneList │ │ +│ │ GetTemplates │ │ RunCampaign │ │ +│ │ UpdateTemplate │ │ GetCampaignStatus│ │ +│ │ DeleteTemplate │ │ GetActiveCampaigns│ │ +│ └──────────────────┘ │ DownloadReport │ │ +│ └──────────────────┘ │ +│ Content Management │ +│ ┌──────────────────┐ │ +│ │ UploadFile │ │ +│ │ DownloadFile │ │ +│ │ (Audio/TTS) │ │ +│ └──────────────────┘ │ +│ │ +└───────────────────────────────────────────────────────────────────────┘ + │ + │ Actual Phone Calls + ▼ + [ Phone Network ] + │ + ▼ + [ Student Phones ] +``` + +## Data Flow Diagram + +### Creating a Phone Template + +``` +User → Frontend → POST /api/phone-template + │ + ▼ + PhoneTemplateService.createTemplate() + │ + ├─► Save to PhoneTemplate table + │ + └─► YemotApiService.createTemplate() + │ + ▼ + Yemot API (CreateTemplate) + │ + └─► Returns yemotTemplateId + │ + ▼ + Update PhoneTemplate record +``` + +### Executing a Campaign + +``` +User selects records → Clicks Bulk Button → PhoneTemplateBulkButton + │ + ├─► Extracts phone numbers + ├─► Shows template selector + └─► POST /api/phone-campaign/execute + │ + ▼ + PhoneCampaignService.executeCampaign() + │ + ├─► Create PhoneCampaign record (status: pending) + │ + ├─► YemotApiService.uploadPhoneList() + │ │ + │ ▼ + │ Yemot API (UploadPhoneList) + │ + ├─► YemotApiService.runCampaign() + │ │ + │ ▼ + │ Yemot API (RunCampaign) + │ │ + │ └─► Returns campaignId + │ + └─► Update PhoneCampaign (status: running, yemotCampaignId) + │ + ▼ + Background polling: + YemotApiService.getCampaignStatus() + │ + ▼ + Update PhoneCampaign with results +``` + +## Component Interaction Matrix + +| Component | Interacts With | Purpose | +|-----------|----------------|---------| +| **Frontend Components** | +| phone-template.jsx | PhoneTemplateService | CRUD operations on templates | +| phone-campaign.jsx | PhoneCampaignService | View campaign history | +| PhoneTemplateBulkButton | PhoneCampaignService | Trigger campaigns | +| **Backend Services** | +| PhoneTemplateService | YemotApiService, PhoneTemplate entity | Template management | +| PhoneCampaignService | YemotApiService, PhoneCampaign entity | Campaign execution | +| YemotApiService | Yemot External API | API integration | +| **Data Layer** | +| PhoneTemplate | PhoneTemplateService, User entity | Store template config | +| PhoneCampaign | PhoneCampaignService, PhoneTemplate, User | Store campaign status | + +## Authentication Flow + +``` +Backend Service + │ + ├─► Get User's Yemot API KEY + │ (from User entity or env variable) + │ + ▼ +YemotApiService + │ + ├─► Add to request header: + │ "authorization: {API_KEY}" + │ + ▼ +HTTPS Request to Yemot API + │ + ▼ +Yemot validates API KEY + │ + ├─► Valid → Process request + │ + └─► Invalid → Return 401/403 error +``` + +## Error Handling Flow + +``` +API Call to Yemot + │ + ▼ + ┌─────────┐ + │ Success?│ + └────┬────┘ + │ + ├─── Yes ──► Return result + │ + └─── No ──► Check error type + │ + ├─► Timeout/Rate Limit + │ │ + │ └─► Retry with backoff + │ + ├─► Auth Error + │ │ + │ └─► Fail immediately + │ │ + │ └─► Log error + │ │ + │ └─► Update campaign status: failed + │ + └─► Network Error + │ + └─► Retry up to 3 times + │ + └─► If all fail → Mark failed +``` + +## Deployment Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Docker Environment │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌───────────────┐ ┌───────────────┐ ┌────────────────┐ │ +│ │ Frontend │ │ Backend │ │ MySQL │ │ +│ │ (Vite) │ │ (NestJS) │ │ Database │ │ +│ │ Port 3000 │ │ Port 3001 │ │ Port 3306 │ │ +│ │ │ │ │ │ │ │ +│ │ + phone- │ │ + Yemot │ │ + phone_ │ │ +│ │ template │ │ ApiService │ │ templates │ │ +│ │ .jsx │ │ │ │ + phone_ │ │ +│ │ │ │ + Phone │ │ campaigns │ │ +│ │ + Phone │ │ Template │ │ │ │ +│ │ Template │ │ Service │ │ │ │ +│ │ BulkButton │ │ │ │ │ │ +│ │ │ │ + Phone │ │ │ │ +│ │ + phone- │ │ Campaign │ │ │ │ +│ │ campaign │ │ Service │ │ │ │ +│ │ .jsx │ │ │ │ │ │ +│ └───────┬───────┘ └───────┬───────┘ └────────────────┘ │ +│ │ │ │ +│ └──────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ + │ + │ HTTPS + │ + ┌─────────▼──────────┐ + │ Yemot API Server │ + │ (External) │ + └────────────────────┘ +``` diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..e6ccbb9e --- /dev/null +++ b/docs/README.md @@ -0,0 +1,103 @@ +# Automated Phone Calls Feature Documentation + +This directory contains the High-Level Design (HLD) documentation for the Automated Phone Calls feature. + +## Documents + +### 1. [HLD-Automated-Phone-Calls.md](./HLD-Automated-Phone-Calls.md) +**Complete High-Level Design Document** (715 lines) + +Comprehensive design covering: +- Executive Summary +- Feature Overview & User Workflow +- Architecture Components (Data Models, Backend Services, Frontend Components) +- API Endpoints +- Yemot API Integration (Methods, Authentication, File Storage) +- Design Questions & Alternatives (8 key decision points) +- Implementation Phases (MVP, Enhanced, Advanced) +- Security, Compliance & Data Privacy +- Testing Strategy +- Monitoring & Observability +- Migration & Deployment +- Open Questions for Product Owner (10 questions) +- Dependencies, Risks & Mitigation +- Success Criteria +- Future Enhancements + +### 2. [HLD-Summary.md](./HLD-Summary.md) +**Quick Reference Guide** + +Condensed version highlighting: +- Key components overview +- Yemot API methods required +- Recommended design decisions +- Implementation phases summary +- Open questions for stakeholders +- Success criteria +- Next steps + +### 3. [HLD-Architecture-Diagram.md](./HLD-Architecture-Diagram.md) +**Visual Architecture Documentation** + +ASCII diagrams showing: +- System Architecture Overview +- Data Flow Diagrams (Template Creation, Campaign Execution) +- Component Interaction Matrix +- Authentication Flow +- Error Handling Flow +- Deployment Architecture + +## Key Terminology + +- **Phone Template** / **Phone Message Template**: Reusable configuration for automated phone messages +- **Campaign**: Specific execution instance of a phone template to multiple recipients +- **TTS**: Text-to-Speech, automated voice generation from text +- **Yemot**: Israeli phone system service providing IVR and campaign capabilities + +## Technology Stack + +- **Frontend**: React 18, React-Admin 5.3, Vite 2.7 +- **Backend**: NestJS 9, TypeORM +- **Database**: MySQL +- **External API**: Yemot Phone System (https://www.call2all.co.il/ym/api/) +- **HTTP Client**: Axios (@nestjs/axios) + +## Reading Order + +1. Start with **HLD-Summary.md** for a quick overview +2. Review **HLD-Architecture-Diagram.md** for visual understanding +3. Read **HLD-Automated-Phone-Calls.md** for complete details + +## Implementation Status + +📝 **Status**: Design Complete - Awaiting Approval + +**Next Steps**: +1. Review HLD with stakeholders +2. Answer open questions (see Section 11 in main HLD) +3. Approve design decisions +4. Create implementation tickets +5. Set up Yemot API test environment +6. Begin Phase 1 (MVP) implementation + +## Questions or Feedback + +For questions about this design or to provide feedback, please: +1. Review the "Open Questions for Product Owner" section in the main HLD +2. Consider the design alternatives presented for each decision point +3. Provide answers and preferences for implementation + +## Related Documentation + +- Project Root: `/README.md` +- Project Index: `/project-index.md` +- Agent Workflow: `/AGENTS.md` +- New Project Checklist: `/new-project-checklist.md` + +## Document Maintenance + +- **Created**: 2026-01-15 +- **Last Updated**: 2026-01-15 +- **Version**: 1.0 +- **Author**: GitHub Copilot Agent +- **Approved By**: [Pending] From 858ac3cd42f0903b29dcfe964ffce61bdff00f2d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 16:09:29 +0000 Subject: [PATCH 4/5] Update HLD based on product owner feedback - finalize all design decisions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Per-user API KEY in User.yemotApiKey field (user settings) - Each user pays via their own Yemot account (no shared billing) - First integration: Student attendance report pivot table - MVP: TTS messages only, audio files deferred to Phase 2 - No scheduled campaigns in MVP - Summary-level campaign tracking (MVP detail level) - Per-user templates and campaigns with CrudAuth isolation - Compliance handled by Yemot system - No integration with YemotCall (incoming calls) - Terminology: EN "Phone Template"/"Campaign", HE "תבנית שיחה"/"מסע פרסום" - Standard BaseEntityModule endpoints with doAction for custom operations - Backend extracts phone numbers from selected IDs - User-triggered status refresh (no background polling) - Updated all design questions with final decisions - Converted "Open Questions" to "Decisions Made" section Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-Automated-Phone-Calls.md | 539 ++++++++++++++---------------- docs/HLD-Summary.md | 117 ++++--- 2 files changed, 313 insertions(+), 343 deletions(-) diff --git a/docs/HLD-Automated-Phone-Calls.md b/docs/HLD-Automated-Phone-Calls.md index 7bbfafab..a4cf62be 100644 --- a/docs/HLD-Automated-Phone-Calls.md +++ b/docs/HLD-Automated-Phone-Calls.md @@ -4,25 +4,28 @@ This HLD describes a new feature for sending automated phone calls to students, parents, or other stakeholders using the Yemot phone system API. The feature will allow users to create reusable phone message templates and trigger them via bulk actions in the admin interface. -**Key Terminology:** We will use the term **"Phone Message Template"** or **"Phone Template"** to refer to configurable phone message definitions. +**Key Terminology:** +- **English**: "Phone Template" for template, "Campaign" for execution +- **Hebrew**: "תבנית שיחה" (Tavnit Sicha) for template, "מסע פרסום" (Masa Parsum) or "משלוח שיחות" (Mishloach Sichot) for campaign ## 2. Feature Overview ### 2.1 Core Capabilities -1. **Template Management**: Create, configure, and manage reusable phone message templates -2. **Message Configuration**: Define message content as text (TTS) or recorded audio files -3. **Bulk Actions**: Trigger phone campaigns from entity lists (e.g., student classes) -4. **Campaign Tracking**: Monitor and track phone call campaigns -5. **Integration**: Seamless integration with existing Yemot system infrastructure +1. **Template Management**: Each user creates and manages their own phone message templates +2. **Message Configuration**: Define message content as text (TTS) in MVP, audio files in future phases +3. **Bulk Actions**: Trigger phone campaigns from student attendance report (pivot table) +4. **Campaign Tracking**: Monitor campaigns with user-triggered status refresh button +5. **Integration**: Each user uses their own Yemot API KEY stored in user settings ### 2.2 User Workflow -1. Admin creates a phone message template with configuration -2. Admin uploads or configures message content (text or audio file) -3. Admin navigates to an entity list (e.g., student classes report) -4. Admin selects records using checkboxes -5. Admin clicks bulk action button and selects a phone template -6. System triggers Yemot campaign to selected phone numbers -7. Admin can view campaign status and results +1. User configures their Yemot API KEY in user settings +2. User creates a phone message template with TTS text +3. User navigates to student attendance report (pivot table) +4. User selects records using checkboxes +5. User clicks bulk action button and selects a phone template +6. Backend extracts phone numbers and triggers Yemot campaign using user's API KEY +7. User can refresh campaign status manually using a button +8. User views campaign results in campaign history ## 3. Architecture Components @@ -52,14 +55,14 @@ export class PhoneTemplate { @Column({ nullable: true }) yemotTemplateId: string; // Yemot campaign template ID - @Column({ type: 'enum', enum: ['text', 'file'] }) - messageType: 'text' | 'file'; // TTS text or audio file + @Column({ type: 'enum', enum: ['text'] }) + messageType: 'text'; // TTS text only in MVP - @Column({ type: 'text', nullable: true }) - messageText: string; // TTS text content + @Column({ type: 'text' }) + messageText: string; // TTS text content (required in MVP) @Column({ length: 255, nullable: true }) - messageFilePath: string; // Path to audio file in Yemot system + messageFilePath: string; // Reserved for future audio file support @Column({ default: false }) isActive: boolean; // Enable/disable template @@ -152,44 +155,47 @@ interface PhoneEntry { New service to interact with Yemot REST API for campaigns/templates. **Key Methods:** -- `createTemplate(userId, description)` - Create Yemot template -- `uploadAudioFile(templateId, filePath, audioData)` - Upload audio file -- `updateTemplateSettings(templateId, settings)` - Update template configuration -- `uploadPhoneList(templateId, phones)` - Upload phone numbers -- `runCampaign(templateId, phones, options)` - Execute campaign -- `getCampaignStatus(campaignId)` - Get campaign status -- `downloadCampaignReport(campaignId)` - Get campaign results +- `createTemplate(apiKey, description)` - Create Yemot template +- `updateTemplateSettings(apiKey, templateId, settings)` - Update template configuration +- `uploadPhoneList(apiKey, templateId, phones)` - Upload phone numbers +- `runCampaign(apiKey, templateId, phones, options)` - Execute campaign with TTS mode +- `getCampaignStatus(apiKey, campaignId)` - Get campaign status (user-triggered) +- `downloadCampaignReport(apiKey, campaignId)` - Get campaign results -**Authentication:** Uses API KEY token stored in environment variables per user. +**Authentication:** Uses per-user API KEY from User.yemotApiKey field, passed in authorization header. #### 3.2.2 PhoneTemplateService -**Location:** `/server/src/phone-template/phone-template.service.ts` +**Location:** Extends `BaseEntityService` -Business logic for managing phone templates. +Business logic for managing phone templates, integrated with BaseEntityModule. **Key Methods:** -- `createTemplate(data)` - Create template and Yemot template -- `configureMessage(templateId, messageConfig)` - Set up message content +- `createTemplate(data)` - Create local template and Yemot template via YemotApiService - `syncWithYemot(templateId)` - Sync local template with Yemot - `validateTemplate(templateId)` - Validate template is ready to use +**Note:** Standard CRUD operations (list, get, update, delete) are handled by BaseEntityController. + #### 3.2.3 PhoneCampaignService -**Location:** `/server/src/phone-campaign/phone-campaign.service.ts` +**Location:** Extends `BaseEntityService` Business logic for executing and tracking campaigns. **Key Methods:** -- `executeCampaign(templateId, phoneNumbers, metadata)` - Start campaign -- `trackCampaignStatus(campaignId)` - Update campaign status -- `getCampaignResults(campaignId)` - Fetch results from Yemot -- `cancelCampaign(campaignId)` - Cancel running campaign +- `executeCampaign(req, body)` - Handles bulk action, extracts phone numbers from selected records, creates campaign, and executes via YemotApiService +- `refreshCampaignStatus(campaignId)` - User-triggered status update from Yemot +- `getCampaignResults(campaignId)` - Fetch detailed results from Yemot + +**Custom Endpoint:** +- `POST /phone-campaign/execute` - Bulk action endpoint (added to BaseEntityController via doAction method) +- `POST /phone-campaign/:id/refresh-status` - Manual status refresh button #### 3.2.4 Entity Modules -- `/server/src/entity-modules/phone-template.config.ts` -- `/server/src/entity-modules/phone-campaign.config.ts` +- `/server/src/entity-modules/phone-template.config.ts` - Uses BaseEntityModule.register() +- `/server/src/entity-modules/phone-campaign.config.ts` - Uses BaseEntityModule.register() -Standard CRUD configuration following existing patterns. +Standard CRUD configuration following existing patterns, registered in entities.module.ts. ### 3.3 Frontend Components @@ -199,11 +205,10 @@ Standard CRUD configuration following existing patterns. React-Admin resource for managing templates. **Features:** -- List view with filters (name, messageType, isActive) -- Create/Edit forms with message configuration -- File upload for audio messages -- Test campaign button -- Sync with Yemot button +- List view with filters (name, isActive) +- Create/Edit forms with TTS message configuration +- Test campaign button (single phone number) +- Each user sees only their own templates #### 3.3.2 Phone Campaign Entity **Location:** `/client/src/entities/phone-campaign.jsx` @@ -212,66 +217,68 @@ React-Admin resource for viewing campaign history. **Features:** - List view with filters (status, date range, template) -- Show view with detailed results -- Real-time status updates -- Download report button +- Show view with summary-level results (total, success, failed counts) +- Manual "Refresh Status" button on each campaign row +- Each user sees only their own campaigns #### 3.3.3 Bulk Action Button Component **Location:** `/client/src/components/PhoneTemplateBulkButton.jsx` -Reusable bulk action button for triggering campaigns. +Reusable bulk action button integrated with student attendance report pivot table. **Props:** -- `entityType` - Type of entity being acted upon -- `phoneNumberExtractor` - Function to extract phone numbers from records -- `metadataExtractor` - Optional function to extract additional data +- `phoneNumberExtractor` - Function to extract phone numbers from pivot records **Behavior:** -1. Opens dialog showing available active phone templates +1. Opens dialog showing user's active phone templates 2. User selects template 3. Confirms action with phone count -4. Triggers API call to execute campaign -5. Shows success/error notification +4. Triggers POST to /phone-campaign/execute with selected IDs and templateId +5. Backend extracts phone numbers and executes campaign +6. Shows success/error notification #### 3.3.4 Integration Example -**Location:** `/client/src/entities/student-klasses-report.jsx` +**Location:** Student attendance report pivot table -Add bulk button to existing entity: +Add bulk button to attendance report: ```jsx import PhoneTemplateBulkButton from 'src/components/PhoneTemplateBulkButton'; +// In the pivot table component (StudentAttendanceList.jsx) const additionalBulkButtons = [ record.student?.phoneNumber} - metadataExtractor={(record) => ({ - studentName: record.student?.name, - klassName: record.klass?.name - })} + phoneNumberExtractor={(record) => record.phoneNumber} // Pivot records have phone numbers /> ]; ``` +**Note:** The pivot table format differs from standard entity lists, so phone number extraction will need to be tailored to the pivot data structure. + ### 3.4 API Endpoints -#### 3.4.1 Phone Templates -- `GET /api/phone-template` - List templates -- `POST /api/phone-template` - Create template -- `GET /api/phone-template/:id` - Get template details -- `PATCH /api/phone-template/:id` - Update template -- `DELETE /api/phone-template/:id` - Delete template -- `POST /api/phone-template/:id/upload-audio` - Upload audio file -- `POST /api/phone-template/:id/sync` - Sync with Yemot -- `POST /api/phone-template/:id/test` - Send test call - -#### 3.4.2 Phone Campaigns -- `GET /api/phone-campaign` - List campaigns -- `POST /api/phone-campaign/execute` - Execute new campaign -- `GET /api/phone-campaign/:id` - Get campaign details -- `POST /api/phone-campaign/:id/refresh-status` - Update from Yemot -- `POST /api/phone-campaign/:id/cancel` - Cancel campaign -- `GET /api/phone-campaign/:id/report` - Download report +**All endpoints use standard BaseEntityController endpoints with CrudAuth filtering.** + +#### 3.4.1 Phone Templates (via BaseEntityModule) +- `GET /api/phone_template` - List user's templates (auto-filtered by userId) +- `POST /api/phone_template` - Create template +- `GET /api/phone_template/:id` - Get template details +- `PATCH /api/phone_template/:id` - Update template +- `DELETE /api/phone_template/:id` - Delete template + +**Custom actions via POST /api/phone_template/action:** +- `{ action: 'test', templateId, phoneNumber }` - Send test call to single number + +#### 3.4.2 Phone Campaigns (via BaseEntityModule) +- `GET /api/phone_campaign` - List user's campaigns (auto-filtered by userId) +- `GET /api/phone_campaign/:id` - Get campaign details +- `GET /api/phone_campaign/export` - Export campaign list +- `GET /api/phone_campaign/report` - Get campaign report data + +**Custom actions via POST /api/phone_campaign/action:** +- `{ action: 'execute', templateId, selectedIds: [] }` - Execute new campaign (bulk action) +- `{ action: 'refresh-status', campaignId }` - Manually refresh campaign status from Yemot ## 4. Yemot API Integration @@ -310,45 +317,48 @@ const additionalBulkButtons = [ ### 4.2 Authentication Strategy -**Recommended Approach: API KEY (Static Token)** - -**Rationale:** -- ✅ Persistent, never expires -- ✅ No MFA requirements -- ✅ Can restrict by IP, services, permissions -- ✅ Suitable for server-to-server communication -- ❌ Requires manual setup per user/organization +**Approach: Per-User API KEY in User Settings** **Implementation:** -1. Store API KEY in User entity or separate YemotConfig entity -2. Use environment variable for default/system-level key -3. Pass token in `authorization` header (recommended) or as `token` parameter - -**Alternative: Login Token (Dynamic)** -- Only if per-user authentication is required -- Requires MFA handling or IP whitelisting -- Adds complexity with token expiration +1. Add `yemotApiKey` field to User entity (varchar, nullable, encrypted at rest) +2. User configures their own Yemot API KEY in user settings/profile page +3. Backend retrieves user's API KEY when executing Yemot API calls +4. Pass API KEY in `authorization` header to Yemot API +5. Each user pays for their own calls via their Yemot account + +**Benefits:** +- ✅ User manages their own Yemot account and billing +- ✅ No shared system-level credentials +- ✅ User-level quota and budget management (handled by Yemot) +- ✅ Secure per-user isolation +- ✅ Users can set different Yemot accounts if needed + +**Security:** +- Store API KEY encrypted in database +- Validate API KEY on first use (test API call) +- Show masked API KEY in UI (e.g., `077***1234`) +- Audit log for all Yemot API calls with userId + +**User Entity Addition:** +```typescript +@Column({ type: 'varchar', length: 255, nullable: true }) +yemotApiKey: string; // Encrypted Yemot API KEY +``` ### 4.3 File Storage Strategy -**Option A: Direct Yemot Storage (Recommended)** +**MVP: TTS Only - No File Storage Needed** + +For MVP, only TTS (text-to-speech) messages are supported, so no audio file storage is required. + +**Future Enhancement (Audio Files):** +When audio file support is added in Phase 2: - Upload audio files directly to Yemot using UploadFile API -- Store only Yemot file path in PhoneTemplate entity +- Store only Yemot file path in PhoneTemplate.messageFilePath +- Use template naming convention: `tpl:${yemotTemplateId}` - Pros: No local storage needed, files managed by Yemot - Cons: Requires Yemot API for all file operations -**Option B: Hybrid Storage** -- Store files locally and sync to Yemot when template activated -- Keep local backup for redundancy -- Pros: Offline access, backup capability -- Cons: Increased storage requirements, sync complexity - -**Option C: Reference Existing Text/Audio System** -- Extend existing Text entity pattern with filepath field -- Leverage TextByUser view for per-user overrides -- Pros: Consistent with existing patterns, reusable infrastructure -- Cons: May not fit all use cases - ## 5. Design Questions and Alternatives @@ -356,221 +366,151 @@ const additionalBulkButtons = [ **Question:** When should the Yemot template be created? -**Option A: On PhoneTemplate Creation (Recommended)** +**Decision: On PhoneTemplate Creation (Immediate)** - Create Yemot template immediately when PhoneTemplate is saved - Store yemotTemplateId immediately -- Pros: Simple, predictable, ready to use -- Cons: Creates Yemot resources that may never be used - -**Option B: On First Use** -- Create Yemot template lazily when first campaign is executed -- Pros: Only creates resources when actually needed -- Cons: Added complexity, potential failure point during campaign execution - -**Option C: Manual Activation** -- Require explicit "Activate" action to create Yemot template -- Pros: User control, clear activation status -- Cons: Extra user step, potential confusion - -**Recommendation:** Option A for simplicity, with Option C as enhancement for advanced users. +- Rationale: Simple, predictable, ready to use. User already has API KEY configured. +- Consequence: May create Yemot resources that are never used, but user pays only for actual calls. ### 5.2 Message Content Management **Question:** How should message content (text/audio) be managed? -**Option A: Embedded in PhoneTemplate (Recommended)** -- Store messageText and messageFilePath directly in PhoneTemplate entity -- Upload audio to Yemot using template naming convention -- Pros: Self-contained, simple to implement -- Cons: Less reusable across templates - -**Option B: Separate PhoneMessage Entity** -- Create separate entity for message content -- PhoneTemplate references PhoneMessage -- Pros: Reusable messages, versioning capability -- Cons: Added complexity, more entities to manage - -**Option C: Reuse Text Entity** -- Extend existing Text entity pattern -- Use TextByUser view for per-user overrides -- Pros: Consistent with existing system, leverages existing infrastructure -- Cons: Text entity may not fit all phone message needs - -**Option D: Hybrid Approach** -- Allow both embedded content AND reference to Text entity -- User chooses which to use -- Pros: Maximum flexibility -- Cons: Most complex, potential confusion - -**Recommendation:** Start with Option A for MVP, consider Option C for consistency if Text entity fits requirements well. +**Decision: Embedded in PhoneTemplate (TTS only for MVP)** +- Store messageText directly in PhoneTemplate entity +- Simple, self-contained approach +- Rationale: MVP focuses on TTS only, embedded content is simplest +- Consequence: Each template has its own message text. Audio files reserved for future. ### 5.3 Phone Number Extraction **Question:** How should phone numbers be extracted from entity records? -**Option A: Configuration-Based (Recommended)** -- Define phone number field path in bulk button configuration -- Use accessor function: `phoneNumberExtractor={(record) => record.student.phoneNumber}` -- Pros: Flexible, works with any entity, explicit -- Cons: Requires configuration per entity - -**Option B: Convention-Based** -- Follow naming convention (e.g., look for `phoneNumber`, `phone`, `mobile` fields) -- Auto-detect phone numbers in records -- Pros: Less configuration, automatic -- Cons: May miss edge cases, less explicit - -**Option C: Entity-Specific Services** -- Create service methods per entity type -- E.g., `StudentKlassService.getPhoneNumbers(recordIds)` -- Pros: Business logic encapsulation, type-safe -- Cons: More code, harder to extend +**Decision: Backend Extraction via doAction** +- Frontend sends selected record IDs via bulk action +- Backend receives IDs and PhoneTemplate ID +- Backend queries the pivot table/entity to extract phone numbers +- Rationale: Backend has direct database access and can handle complex pivot queries +- Consequence: Frontend just passes IDs, backend handles all extraction logic + +**Frontend:** +```jsx +// Bulk button just sends selected IDs + +``` -**Recommendation:** Option A for flexibility, with Option C as enhancement for complex cases. +**Backend (PhoneCampaignService.executeCampaign):** +```typescript +async executeCampaign(req: CrudRequest, body: { templateId: number }) { + const selectedIds = body.selectedIds; // From bulk selection + const phoneNumbers = await this.extractPhoneNumbersFromPivot(selectedIds); + // ... create campaign and execute +} +``` ### 5.4 Campaign Execution Flow **Question:** Should campaigns be executed synchronously or asynchronously? -**Option A: Asynchronous with Job Queue (Recommended)** -- Create PhoneCampaign record immediately -- Execute Yemot API call asynchronously -- Poll for status updates -- Pros: Non-blocking UI, handles failures gracefully -- Cons: Requires job queue or background worker - -**Option B: Synchronous Execution** -- Wait for Yemot API response before returning -- Pros: Simple, immediate feedback -- Cons: Slow UI, timeout risks, poor UX for large campaigns - -**Option C: Hybrid - Quick Validation Then Async** -- Validate template and phones synchronously -- Execute campaign asynchronously -- Pros: Fast failure feedback, non-blocking execution -- Cons: Medium complexity - -**Recommendation:** Option C for best UX, or Option A for production-grade implementation. +**Decision: Synchronous for MVP (Simple Approach)** +- Frontend waits for backend to complete campaign execution +- Backend creates campaign, calls Yemot API, saves yemotCampaignId, returns success +- Status updates are user-triggered via "Refresh Status" button +- Rationale: Simpler implementation for MVP, no background job queue needed +- Consequence: User may wait a few seconds for large campaigns, but gets immediate feedback + +**Flow:** +1. User clicks bulk action → POST /phone-campaign/action { action: 'execute', templateId, selectedIds } +2. Backend extracts phone numbers, creates PhoneCampaign record (status: pending) +3. Backend calls Yemot RunCampaign API (waits for response) +4. Backend updates PhoneCampaign with yemotCampaignId (status: running) +5. User gets success message +6. User manually clicks "Refresh Status" button later to get results ### 5.5 Result Tracking **Question:** How granular should campaign result tracking be? -**Option A: Summary Only (Recommended for MVP)** +**Decision: Summary Only (MVP Level)** - Track only aggregate statistics (total, success, failed) - Store in PhoneCampaign entity -- Pros: Simple, low storage overhead -- Cons: No per-call details - -**Option B: Per-Call Tracking** -- Create PhoneCampaignCall entity for each call -- Store individual call results -- Pros: Detailed insights, troubleshooting capability -- Cons: High storage overhead, more complex +- Rationale: Matches MVP requirement for "mvp level" detail +- Consequence: No per-call details in MVP. Can enhance in future phases if needed. -**Option C: Summary + Error Details** -- Store aggregate stats plus failed call details -- Pros: Balance of simplicity and troubleshooting -- Cons: Medium complexity - -**Recommendation:** Option A for MVP, enhance to Option C based on user needs. +**PhoneCampaign Fields:** +- totalPhones: number +- successfulCalls: number +- failedCalls: number +- status: 'pending' | 'running' | 'completed' | 'failed' ### 5.6 Template vs Campaign Naming **Question:** What terminology should we use in the UI? -**Option A: "Phone Template" + "Campaign" (Recommended)** -- Template = reusable configuration -- Campaign = specific execution instance -- Pros: Clear distinction, industry standard -- Cons: More concepts to explain - -**Option B: "Phone Message" + "Broadcast"** -- Message = content to send -- Broadcast = sending action -- Pros: Simpler language, user-friendly -- Cons: Less technical precision - -**Option C: "Call Template" + "Call Job"** -- Technical naming -- Pros: Precise, developer-friendly -- Cons: Less user-friendly - -**Recommendation:** Option A for clarity, use "Phone Message Template" in UI for user-friendliness. +**Decision: "Phone Template" + "Campaign" with Hebrew translations** +- English: "Phone Template" (template), "Campaign" (execution) +- Hebrew: "תבנית שיחה" (template), "מסע פרסום" or "משלוח שיחות" (campaign) +- Rationale: Clear, industry-standard terminology +- Consequence: User-friendly labels in both languages ### 5.7 Multi-Tenancy Considerations **Question:** Should templates be global or per-user? -**Option A: Per-User Templates (Recommended)** -- Each user has their own templates -- Managed via userId foreign key -- Pros: Isolation, security, aligns with existing pattern -- Cons: No template sharing - -**Option B: Global Templates with Permissions** -- System-level templates (userId = 0) -- User-level templates (userId > 0) -- Pros: Template reuse, admin control -- Cons: More complex permissions - -**Option C: Organization-Level Templates** -- Templates scoped to organization -- Requires organization entity -- Pros: Team collaboration -- Cons: Requires organization model - -**Recommendation:** Option A for MVP, enhance to Option B following Text entity pattern. +**Decision: Per-User Templates and Campaigns** +- Each user creates and manages their own templates +- Each user sees only their own campaigns +- Enforced via CrudAuth filter on userId +- Rationale: Matches requirement "each user can create its own template, and its own campaign" +- Consequence: No template sharing between users, strong isolation ### 5.8 Error Handling Strategy **Question:** How should Yemot API errors be handled? -**Option A: Retry with Exponential Backoff** -- Automatically retry failed API calls -- Implement exponential backoff -- Pros: Resilient to transient failures -- Cons: Delayed error reporting - -**Option B: Fail Fast** -- Report errors immediately -- User manually retries -- Pros: Simple, clear failures -- Cons: Poor UX for temporary issues - -**Option C: Smart Retry** -- Retry on specific error codes (timeout, rate limit) -- Fail fast on others (auth, invalid data) -- Pros: Best UX, balanced approach -- Cons: Most complex - -**Recommendation:** Option C for production quality. +**Decision: Smart Retry for MVP** +- Retry on timeout/network errors (up to 2 retries) +- Fail fast on authentication/validation errors +- Log all errors with userId for debugging +- Return user-friendly error messages +- Rationale: Balance between resilience and simplicity +- Consequence: Most transient errors will succeed, permanent errors fail quickly ## 6. Implementation Phases ### Phase 1: MVP (Minimum Viable Product) -- PhoneTemplate entity with basic fields -- PhoneCampaign entity with status tracking -- YemotApiService with core methods -- Basic template CRUD in admin -- Single bulk action button implementation -- Text-based messages (TTS) only -- Summary-level campaign tracking +**Target:** First working version with essential features + +- Add `yemotApiKey` field to User entity (varchar, nullable) +- User settings page to configure Yemot API KEY +- PhoneTemplate entity with TTS text only +- PhoneCampaign entity with summary-level tracking +- YemotApiService with core Yemot API methods (CreateTemplate, RunCampaign, GetCampaignStatus) +- PhoneTemplateService and PhoneCampaignService extending BaseEntityService +- Entity configs using BaseEntityModule.register() +- phone-template.jsx - Basic CRUD for templates +- phone-campaign.jsx - View campaign history with "Refresh Status" button +- PhoneTemplateBulkButton integrated with student attendance pivot table +- Execute campaign via doAction endpoint +- TTS messages only (no audio files) +- Summary-level campaign tracking (total, success, failed counts) +- Manual status refresh (user-triggered button) ### Phase 2: Enhanced Features -- Audio file upload support -- Advanced template settings -- Multiple bulk action integrations -- Real-time status updates -- Campaign report download +- Audio file upload support (messageFilePath) +- Advanced template settings (timeWindow, retry config) +- Additional bulk action integrations (other entities beyond attendance) +- Automatic status refresh (background polling) +- Campaign report export/download +- Template test functionality (send to single number) ### Phase 3: Advanced Features -- Scheduled campaigns -- Template sharing (global templates) -- Per-call result tracking +- Scheduled campaigns (time-delayed execution) +- Message personalization (insert student name, class name) - Campaign analytics dashboard -- Message personalization (name insertion) -- Retry failed calls +- Per-call result tracking (detailed logs) +- Retry failed calls functionality +- Campaign cost estimation before execution ## 7. Security and Compliance @@ -649,18 +589,27 @@ const additionalBulkButtons = [ - Monitor and iterate - Expand to all applicable entities -## 11. Open Questions for Product Owner +## 11. Decisions Made (Based on Product Owner Feedback) + +All major questions have been answered by the product owner: + +1. **Authentication**: ✅ Per-user API KEY stored in user settings (User.yemotApiKey field) +2. **Billing**: ✅ Each user pays via their own Yemot account/API KEY +3. **Priority**: ✅ Student attendance report pivot table gets bulk button first +4. **Content**: ✅ MVP supports TTS only, audio files in Phase 2 +5. **Scheduling**: ✅ Not needed in MVP, deferred to Phase 3 +6. **Reporting**: ✅ MVP level - summary statistics only (total, success, failed) +7. **Permissions**: ✅ Each user creates their own templates and campaigns (per-user isolation) +8. **Compliance**: ✅ Handled by Yemot system (calling hours, opt-out, etc.) +9. **Integration**: ✅ Not related to existing YemotCall entity (incoming calls), separate logic +10. **Terminology**: ✅ EN: "Phone Template" / "Campaign", HE: "תבנית שיחה" / "מסע פרסום" -1. **Authentication**: Do we have Yemot API KEY for each user, or single system-level key? -2. **Billing**: Who pays for phone calls? Is there a budget/limit per user? -3. **Priority**: Which entity should get bulk action button first? (student-klasses-report?) -4. **Content**: Should we support both TTS and audio files in MVP, or TTS only? -5. **Scheduling**: Is scheduled campaign execution needed in MVP? -6. **Reporting**: What level of detail is needed in campaign reports? -7. **Permissions**: Should all users be able to create templates, or admin-only? -8. **Compliance**: Are there legal requirements for calling hours, opt-out, etc.? -9. **Integration**: Should this integrate with existing YemotCall entity for incoming calls? -10. **Naming**: Do we prefer "Phone Template", "Phone Message", or "Call Template" in UI? +### Additional Implementation Details Confirmed: + +11. **API Endpoints**: ✅ Use standard BaseEntityModule endpoints with doAction for custom operations +12. **Campaign Execution**: ✅ Backend extracts phone numbers from selected IDs, creates and executes campaign +13. **Status Updates**: ✅ User-triggered "Refresh Status" button (no background polling in MVP) +14. **Phone Number Source**: ✅ Extract from student attendance pivot table records ## 12. Dependencies and Risks @@ -705,11 +654,25 @@ const additionalBulkButtons = [ ## 15. Conclusion -This HLD provides a comprehensive design for the Automated Phone Calls feature, covering all architectural aspects from data models to API integration. The phased implementation approach allows for incremental delivery and validation. The design questions highlight key decision points that should be discussed with stakeholders before detailed implementation begins. +This HLD provides a comprehensive design for the Automated Phone Calls feature with all decisions finalized based on product owner feedback. The design covers: + +- **Data Models**: PhoneTemplate and PhoneCampaign entities with per-user isolation +- **Services**: YemotApiService, PhoneTemplateService, PhoneCampaignService using BaseEntityModule +- **Frontend**: React-Admin entities and bulk action button for student attendance pivot +- **Yemot Integration**: Per-user API KEY authentication, TTS messages, manual status refresh +- **Implementation**: Clear 3-phase roadmap (MVP → Enhanced → Advanced) + +**Key Design Principles:** +- Per-user templates and campaigns with strong isolation +- Standard BaseEntityModule patterns for consistency +- TTS-only for MVP, audio files deferred +- Backend-driven phone number extraction +- User-triggered status updates (simple, no background polling) +- Each user manages their own Yemot account and billing **Next Steps:** -1. Review and approve HLD -2. Answer open questions -3. Create detailed implementation tickets -4. Set up Yemot API test environment +1. ✅ HLD approved with all decisions finalized +2. Create detailed implementation tickets for Phase 1 +3. Add User.yemotApiKey field migration +4. Set up Yemot API test environment with test API KEY 5. Begin Phase 1 implementation diff --git a/docs/HLD-Summary.md b/docs/HLD-Summary.md index 4ca397bd..cebb7180 100644 --- a/docs/HLD-Summary.md +++ b/docs/HLD-Summary.md @@ -6,96 +6,103 @@ ## Feature Overview -Create reusable phone message templates and trigger automated phone campaigns via bulk actions using Yemot API. +Create reusable phone message templates and trigger automated phone campaigns via bulk actions using Yemot API. Each user manages their own templates, campaigns, and Yemot API KEY. ## Key Components ### Entities -1. **PhoneTemplate** - Reusable message template configuration -2. **PhoneCampaign** - Campaign execution tracking +1. **PhoneTemplate** - Per-user reusable message template (TTS text in MVP) +2. **PhoneCampaign** - Per-user campaign execution tracking +3. **User.yemotApiKey** - New field for per-user Yemot API KEY ### Services -1. **YemotApiService** - Yemot REST API integration -2. **PhoneTemplateService** - Template business logic -3. **PhoneCampaignService** - Campaign execution logic +1. **YemotApiService** - Yemot REST API integration (uses per-user API KEY) +2. **PhoneTemplateService** - Extends BaseEntityService, template business logic +3. **PhoneCampaignService** - Extends BaseEntityService, campaign execution & tracking ### Frontend -1. **phone-template.jsx** - Template management UI -2. **phone-campaign.jsx** - Campaign history UI -3. **PhoneTemplateBulkButton.jsx** - Reusable bulk action component +1. **phone-template.jsx** - Template CRUD (per-user, TTS only) +2. **phone-campaign.jsx** - Campaign history with manual "Refresh Status" button +3. **PhoneTemplateBulkButton.jsx** - Bulk action for student attendance pivot table ## Yemot API Methods Required **Template Management:** -- CreateTemplate, GetTemplates, UpdateTemplate, DeleteTemplate - -**Content Management:** -- UploadFile (audio files) +- CreateTemplate, UpdateTemplate **Campaign Execution:** -- UploadPhoneList, RunCampaign +- UploadPhoneList, RunCampaign (with ttsMode=1) **Campaign Monitoring:** -- GetCampaignStatus, GetActiveCampaigns, DownloadCampaignReport - -## Key Design Decisions (Recommended) - -1. **Template Creation Timing**: On PhoneTemplate creation (immediate) -2. **Message Content**: Embedded in PhoneTemplate entity -3. **Phone Extraction**: Configuration-based with accessor functions -4. **Execution Flow**: Hybrid - validate sync, execute async -5. **Result Tracking**: Summary only for MVP -6. **Terminology**: "Phone Template" + "Campaign" -7. **Multi-Tenancy**: Per-user templates -8. **Error Handling**: Smart retry (retry on specific errors) +- GetCampaignStatus (user-triggered) + +## Key Design Decisions (Finalized) + +1. **Authentication**: Per-user API KEY in User.yemotApiKey field +2. **Billing**: Each user pays via their own Yemot account +3. **Template Creation Timing**: Immediate on save +4. **Message Content**: TTS text only (MVP), embedded in PhoneTemplate +5. **Phone Extraction**: Backend extracts from selected IDs (pivot table) +6. **Execution Flow**: Synchronous for MVP (simple, user waits for completion) +7. **Result Tracking**: Summary only (total, success, failed counts) +8. **Terminology**: EN: "Phone Template"/"Campaign", HE: "תבנית שיחה"/"מסע פרסום" +9. **Multi-Tenancy**: Per-user templates and campaigns (CrudAuth filtering) +10. **Status Updates**: User-triggered "Refresh Status" button (no background polling) +11. **API Endpoints**: Standard BaseEntityModule with doAction for custom operations +12. **First Integration**: Student attendance report pivot table ## Implementation Phases ### Phase 1: MVP -- Basic entities and CRUD +- User.yemotApiKey field & settings page +- PhoneTemplate & PhoneCampaign entities - YemotApiService core methods -- Single bulk action button +- BaseEntityModule configs - TTS messages only -- Summary tracking +- Bulk button on student attendance pivot +- Execute campaign via doAction +- Manual status refresh +- Summary-level tracking ### Phase 2: Enhanced - Audio file support -- Multiple integrations -- Real-time updates -- Report download +- Additional bulk actions +- Automatic status refresh +- Report export +- Template testing ### Phase 3: Advanced - Scheduled campaigns -- Template sharing +- Message personalization - Per-call tracking - Analytics dashboard -- Personalization +- Cost estimation -## Open Questions for PO +## All Questions Answered ✅ -1. Do we have Yemot API KEY for each user or system-level? -2. Who pays for calls? Budget/limits per user? -3. Which entity gets bulk button first? -4. TTS only or audio files in MVP? -5. Scheduled campaigns needed in MVP? -6. Detail level for campaign reports? -7. Who can create templates - all users or admin only? -8. Legal requirements for calling hours, opt-out? -9. Integrate with existing YemotCall entity? -10. Preferred UI terminology? +1. ✅ API KEY strategy: Per-user in settings +2. ✅ Billing: User's own account +3. ✅ First integration: Student attendance pivot +4. ✅ MVP content: TTS only +5. ✅ Scheduling: Not in MVP +6. ✅ Report detail: MVP level (summary) +7. ✅ Permissions: Each user their own +8. ✅ Compliance: Handled by Yemot +9. ✅ YemotCall integration: Not related +10. ✅ Terminology: Provided in EN & HE ## Success Criteria -- Users can create templates and execute campaigns -- Campaigns with 100+ numbers execute timely -- 95%+ success rate for valid numbers -- Users need no training -- 50%+ adoption rate +- Users can configure API KEY and create templates +- Bulk action executes campaigns from attendance pivot +- Campaigns execute with TTS messages +- Users can manually refresh campaign status +- 95%+ success rate for valid phone numbers ## Next Steps -1. Review and approve HLD -2. Answer open questions -3. Create implementation tickets -4. Set up Yemot API test environment -5. Begin Phase 1 implementation +1. ✅ HLD approved with all decisions +2. Create Phase 1 implementation tickets +3. Add User.yemotApiKey migration +4. Set up Yemot test environment +5. Begin Phase 1 development From 6d3b42f56c4d4cab9fb26e07ec5d73ae2fc8cbcc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 15 Jan 2026 16:25:31 +0000 Subject: [PATCH 5/5] Refine HLD based on additional feedback - no custom routes, entity service integration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Hebrew campaign term: "משלוח שיחות" (Mishloach Sichot) only - API KEY in User.additionalData.yemotApiKey (no schema changes) - No custom routes - use standard BaseEntityModule patterns only - Entity services (StudentAttendanceService) override doAction to handle campaign execution - Entity service extracts phone numbers and calls PhoneCampaignService.executeCampaign() - Campaign execution flow: Entity /action endpoint → Entity service → PhoneCampaignService - Added detailed code example for entity service integration - Updated all references from User.yemotApiKey field to User.additionalData.yemotApiKey - Clarified no database migrations needed for User table - Updated terminology, implementation phases, and migration sections Co-authored-by: buzz-code <72219643+buzz-code@users.noreply.github.com> --- docs/HLD-Automated-Phone-Calls.md | 134 ++++++++++++++++++++++-------- docs/HLD-Summary.md | 38 +++++---- 2 files changed, 122 insertions(+), 50 deletions(-) diff --git a/docs/HLD-Automated-Phone-Calls.md b/docs/HLD-Automated-Phone-Calls.md index a4cf62be..047d56be 100644 --- a/docs/HLD-Automated-Phone-Calls.md +++ b/docs/HLD-Automated-Phone-Calls.md @@ -6,7 +6,7 @@ This HLD describes a new feature for sending automated phone calls to students, **Key Terminology:** - **English**: "Phone Template" for template, "Campaign" for execution -- **Hebrew**: "תבנית שיחה" (Tavnit Sicha) for template, "מסע פרסום" (Masa Parsum) or "משלוח שיחות" (Mishloach Sichot) for campaign +- **Hebrew**: "תבנית שיחה" (Tavnit Sicha) for template, "משלוח שיחות" (Mishloach Sichot) for campaign ## 2. Feature Overview @@ -162,7 +162,7 @@ New service to interact with Yemot REST API for campaigns/templates. - `getCampaignStatus(apiKey, campaignId)` - Get campaign status (user-triggered) - `downloadCampaignReport(apiKey, campaignId)` - Get campaign results -**Authentication:** Uses per-user API KEY from User.yemotApiKey field, passed in authorization header. +**Authentication:** Uses per-user API KEY from User.additionalData.yemotApiKey, passed in authorization header. #### 3.2.2 PhoneTemplateService @@ -183,13 +183,22 @@ Business logic for managing phone templates, integrated with BaseEntityModule. Business logic for executing and tracking campaigns. **Key Methods:** -- `executeCampaign(req, body)` - Handles bulk action, extracts phone numbers from selected records, creates campaign, and executes via YemotApiService -- `refreshCampaignStatus(campaignId)` - User-triggered status update from Yemot -- `getCampaignResults(campaignId)` - Fetch detailed results from Yemot - -**Custom Endpoint:** -- `POST /phone-campaign/execute` - Bulk action endpoint (added to BaseEntityController via doAction method) -- `POST /phone-campaign/:id/refresh-status` - Manual status refresh button +- `doAction(req, body)` - Overrides base method to handle custom actions: + - `action: 'execute'` - Called from entity-specific service (e.g., StudentService), creates campaign and executes via YemotApiService + - `action: 'refresh-status'` - User-triggered status update from Yemot +- `executeCampaign(userId, templateId, phoneNumbers)` - Core execution logic, called by entity services +- `refreshCampaignStatus(campaignId, userId)` - Fetch and update status from Yemot +- `getCampaignResults(campaignId, userId)` - Fetch detailed results from Yemot + +**Execution Flow:** +1. Entity service (e.g., StudentService or AttendanceService) receives bulk action request +2. Entity service extracts phone numbers from its own entity records +3. Entity service calls `PhoneCampaignService.executeCampaign(userId, templateId, phoneNumbers)` +4. PhoneCampaignService creates PhoneCampaign record and executes via YemotApiService + +**Standard Endpoints (via BaseEntityModule):** +- `POST /phone_campaign/action` with `{ action: 'execute', templateId, selectedIds }` - Routed to entity service +- `POST /phone_campaign/action` with `{ action: 'refresh-status', campaignId }` - Updates campaign status #### 3.2.4 Entity Modules - `/server/src/entity-modules/phone-template.config.ts` - Uses BaseEntityModule.register() @@ -197,6 +206,56 @@ Business logic for executing and tracking campaigns. Standard CRUD configuration following existing patterns, registered in entities.module.ts. +#### 3.2.5 Entity Service Integration (Example: StudentAttendanceService) +**Location:** Entity-specific service that handles bulk actions + +To trigger phone campaigns from an entity (e.g., student attendance pivot table): + +**Implementation Pattern:** +```typescript +// In StudentAttendanceService or AttendanceService +@Injectable() +export class StudentAttendanceService extends BaseEntityService { + constructor( + @InjectEntityRepository repo: Repository, + mailSendService: MailSendService, + private phoneCampaignService: PhoneCampaignService, // Inject campaign service + ) { + super(repo, mailSendService); + } + + async doAction(req: CrudRequest, body: any): Promise { + if (body.action === 'execute-phone-campaign') { + // Extract phone numbers from selected records + const selectedIds = body.selectedIds; + const phoneNumbers = await this.extractPhoneNumbers(selectedIds); + + // Execute campaign via PhoneCampaignService + return this.phoneCampaignService.executeCampaign( + req.auth.userId, + body.templateId, + phoneNumbers + ); + } + return super.doAction(req, body); + } + + private async extractPhoneNumbers(selectedIds: number[]): Promise { + // Query pivot table or entity records to get phone numbers + // Implementation specific to entity structure + const records = await this.repo.findByIds(selectedIds); + return records.map(r => r.phoneNumber).filter(Boolean); + } +} +``` + +**Flow:** +1. Frontend sends: `POST /api/student_attendance/action` with `{ action: 'execute-phone-campaign', templateId, selectedIds }` +2. StudentAttendanceService.doAction() receives request +3. Service extracts phone numbers from its own entity records +4. Service calls PhoneCampaignService.executeCampaign() +5. PhoneCampaignService creates campaign and executes via YemotApiService + ### 3.3 Frontend Components #### 3.3.1 Phone Template Entity @@ -233,8 +292,8 @@ Reusable bulk action button integrated with student attendance report pivot tabl 1. Opens dialog showing user's active phone templates 2. User selects template 3. Confirms action with phone count -4. Triggers POST to /phone-campaign/execute with selected IDs and templateId -5. Backend extracts phone numbers and executes campaign +4. Triggers POST to entity's /action endpoint (e.g., /api/student_attendance/action) with `{ action: 'execute-phone-campaign', templateId, selectedIds }` +5. Entity service (StudentAttendanceService) extracts phone numbers and calls PhoneCampaignService 6. Shows success/error notification #### 3.3.4 Integration Example @@ -277,9 +336,10 @@ const additionalBulkButtons = [ - `GET /api/phone_campaign/report` - Get campaign report data **Custom actions via POST /api/phone_campaign/action:** -- `{ action: 'execute', templateId, selectedIds: [] }` - Execute new campaign (bulk action) - `{ action: 'refresh-status', campaignId }` - Manually refresh campaign status from Yemot +**Note:** Campaign execution is triggered from the entity where bulk action is performed (e.g., student attendance pivot). The entity service (e.g., AttendanceService) handles the bulk action, extracts phone numbers, and calls PhoneCampaignService.executeCampaign() directly. + ## 4. Yemot API Integration ### 4.1 Required Yemot API Methods @@ -317,32 +377,35 @@ const additionalBulkButtons = [ ### 4.2 Authentication Strategy -**Approach: Per-User API KEY in User Settings** +**Approach: Per-User API KEY in User Settings (additionalData)** **Implementation:** -1. Add `yemotApiKey` field to User entity (varchar, nullable, encrypted at rest) +1. Store `yemotApiKey` in User.additionalData JSON field (encrypted at application level) 2. User configures their own Yemot API KEY in user settings/profile page -3. Backend retrieves user's API KEY when executing Yemot API calls +3. Backend retrieves user's API KEY from additionalData when executing Yemot API calls 4. Pass API KEY in `authorization` header to Yemot API 5. Each user pays for their own calls via their Yemot account **Benefits:** - ✅ User manages their own Yemot account and billing -- ✅ No shared system-level credentials +- ✅ No database schema changes needed (uses existing additionalData field) - ✅ User-level quota and budget management (handled by Yemot) - ✅ Secure per-user isolation - ✅ Users can set different Yemot accounts if needed **Security:** -- Store API KEY encrypted in database +- Encrypt API KEY at application level before storing in additionalData - Validate API KEY on first use (test API call) - Show masked API KEY in UI (e.g., `077***1234`) - Audit log for all Yemot API calls with userId -**User Entity Addition:** +**Storage Format:** ```typescript -@Column({ type: 'varchar', length: 255, nullable: true }) -yemotApiKey: string; // Encrypted Yemot API KEY +// User.additionalData structure +{ + yemotApiKey: 'encrypted_api_key_string', + // ... other user settings +} ``` ### 4.3 File Storage Strategy @@ -481,20 +544,21 @@ async executeCampaign(req: CrudRequest, body: { templateId: number }) { ### Phase 1: MVP (Minimum Viable Product) **Target:** First working version with essential features -- Add `yemotApiKey` field to User entity (varchar, nullable) +- Store yemotApiKey in User.additionalData (no schema changes) - User settings page to configure Yemot API KEY - PhoneTemplate entity with TTS text only - PhoneCampaign entity with summary-level tracking - YemotApiService with core Yemot API methods (CreateTemplate, RunCampaign, GetCampaignStatus) - PhoneTemplateService and PhoneCampaignService extending BaseEntityService - Entity configs using BaseEntityModule.register() +- StudentAttendanceService integration: override doAction to handle 'execute-phone-campaign' action - phone-template.jsx - Basic CRUD for templates -- phone-campaign.jsx - View campaign history with "Refresh Status" button +- phone-campaign.jsx - View campaign history with "Refresh Status" action button - PhoneTemplateBulkButton integrated with student attendance pivot table -- Execute campaign via doAction endpoint +- Execute campaign via entity service doAction (e.g., /api/student_attendance/action) - TTS messages only (no audio files) - Summary-level campaign tracking (total, success, failed counts) -- Manual status refresh (user-triggered button) +- Manual status refresh via /action endpoint (user-triggered button) ### Phase 2: Enhanced Features - Audio file upload support (messageFilePath) @@ -575,7 +639,8 @@ async executeCampaign(req: CrudRequest, body: { templateId: number }) { ### 10.1 Database Migrations - Create PhoneTemplate table - Create PhoneCampaign table -- Add indexes for performance +- Add indexes for performance (userId, status, createdAt) +- No User table changes needed (uses existing additionalData field) ### 10.2 Configuration - Environment variables for Yemot API credentials @@ -593,7 +658,7 @@ async executeCampaign(req: CrudRequest, body: { templateId: number }) { All major questions have been answered by the product owner: -1. **Authentication**: ✅ Per-user API KEY stored in user settings (User.yemotApiKey field) +1. **Authentication**: ✅ Per-user API KEY stored in user settings (User.additionalData.yemotApiKey) 2. **Billing**: ✅ Each user pays via their own Yemot account/API KEY 3. **Priority**: ✅ Student attendance report pivot table gets bulk button first 4. **Content**: ✅ MVP supports TTS only, audio files in Phase 2 @@ -602,14 +667,14 @@ All major questions have been answered by the product owner: 7. **Permissions**: ✅ Each user creates their own templates and campaigns (per-user isolation) 8. **Compliance**: ✅ Handled by Yemot system (calling hours, opt-out, etc.) 9. **Integration**: ✅ Not related to existing YemotCall entity (incoming calls), separate logic -10. **Terminology**: ✅ EN: "Phone Template" / "Campaign", HE: "תבנית שיחה" / "מסע פרסום" +10. **Terminology**: ✅ EN: "Phone Template" / "Campaign", HE: "תבנית שיחה" / "משלוח שיחות" ### Additional Implementation Details Confirmed: -11. **API Endpoints**: ✅ Use standard BaseEntityModule endpoints with doAction for custom operations -12. **Campaign Execution**: ✅ Backend extracts phone numbers from selected IDs, creates and executes campaign -13. **Status Updates**: ✅ User-triggered "Refresh Status" button (no background polling in MVP) -14. **Phone Number Source**: ✅ Extract from student attendance pivot table records +11. **API Endpoints**: ✅ Use standard BaseEntityModule endpoints only (no custom routes) +12. **Campaign Execution**: ✅ Entity service (e.g., StudentAttendanceService) extracts phone numbers and calls PhoneCampaignService.executeCampaign() +13. **Status Updates**: ✅ User-triggered "Refresh Status" button via /action endpoint (no background polling in MVP) +14. **Phone Number Source**: ✅ Extract from student attendance pivot table records by entity service ## 12. Dependencies and Risks @@ -664,15 +729,16 @@ This HLD provides a comprehensive design for the Automated Phone Calls feature w **Key Design Principles:** - Per-user templates and campaigns with strong isolation -- Standard BaseEntityModule patterns for consistency +- Standard BaseEntityModule patterns only (no custom routes) +- API KEY stored in User.additionalData (no schema changes) - TTS-only for MVP, audio files deferred -- Backend-driven phone number extraction +- Entity services extract phone numbers and call PhoneCampaignService - User-triggered status updates (simple, no background polling) - Each user manages their own Yemot account and billing **Next Steps:** 1. ✅ HLD approved with all decisions finalized 2. Create detailed implementation tickets for Phase 1 -3. Add User.yemotApiKey field migration +3. Configure user settings UI for additionalData.yemotApiKey 4. Set up Yemot API test environment with test API KEY 5. Begin Phase 1 implementation diff --git a/docs/HLD-Summary.md b/docs/HLD-Summary.md index cebb7180..e3e3a042 100644 --- a/docs/HLD-Summary.md +++ b/docs/HLD-Summary.md @@ -6,23 +6,24 @@ ## Feature Overview -Create reusable phone message templates and trigger automated phone campaigns via bulk actions using Yemot API. Each user manages their own templates, campaigns, and Yemot API KEY. +Create reusable phone message templates and trigger automated phone campaigns via bulk actions using Yemot API. Each user manages their own templates, campaigns, and Yemot API KEY stored in user settings. ## Key Components ### Entities 1. **PhoneTemplate** - Per-user reusable message template (TTS text in MVP) 2. **PhoneCampaign** - Per-user campaign execution tracking -3. **User.yemotApiKey** - New field for per-user Yemot API KEY +3. **User.additionalData.yemotApiKey** - API KEY stored in existing user settings field ### Services 1. **YemotApiService** - Yemot REST API integration (uses per-user API KEY) 2. **PhoneTemplateService** - Extends BaseEntityService, template business logic 3. **PhoneCampaignService** - Extends BaseEntityService, campaign execution & tracking +4. **Entity Services** - StudentAttendanceService (or similar) integrates via doAction override ### Frontend 1. **phone-template.jsx** - Template CRUD (per-user, TTS only) -2. **phone-campaign.jsx** - Campaign history with manual "Refresh Status" button +2. **phone-campaign.jsx** - Campaign history with manual "Refresh Status" action button 3. **PhoneTemplateBulkButton.jsx** - Bulk action for student attendance pivot table ## Yemot API Methods Required @@ -38,30 +39,31 @@ Create reusable phone message templates and trigger automated phone campaigns vi ## Key Design Decisions (Finalized) -1. **Authentication**: Per-user API KEY in User.yemotApiKey field +1. **Authentication**: Per-user API KEY in User.additionalData.yemotApiKey 2. **Billing**: Each user pays via their own Yemot account 3. **Template Creation Timing**: Immediate on save 4. **Message Content**: TTS text only (MVP), embedded in PhoneTemplate -5. **Phone Extraction**: Backend extracts from selected IDs (pivot table) -6. **Execution Flow**: Synchronous for MVP (simple, user waits for completion) +5. **Phone Extraction**: Entity service extracts from its own records +6. **Execution Flow**: Entity service → PhoneCampaignService.executeCampaign() 7. **Result Tracking**: Summary only (total, success, failed counts) -8. **Terminology**: EN: "Phone Template"/"Campaign", HE: "תבנית שיחה"/"מסע פרסום" +8. **Terminology**: EN: "Phone Template"/"Campaign", HE: "תבנית שיחה"/"משלוח שיחות" 9. **Multi-Tenancy**: Per-user templates and campaigns (CrudAuth filtering) -10. **Status Updates**: User-triggered "Refresh Status" button (no background polling) -11. **API Endpoints**: Standard BaseEntityModule with doAction for custom operations -12. **First Integration**: Student attendance report pivot table +10. **Status Updates**: User-triggered via /action endpoint (no background polling) +11. **API Endpoints**: Standard BaseEntityModule only (no custom routes) +12. **Entity Integration**: Each entity service overrides doAction for campaign execution ## Implementation Phases ### Phase 1: MVP -- User.yemotApiKey field & settings page +- User.additionalData.yemotApiKey & settings page - PhoneTemplate & PhoneCampaign entities - YemotApiService core methods - BaseEntityModule configs - TTS messages only +- Entity service integration (StudentAttendanceService overrides doAction) - Bulk button on student attendance pivot -- Execute campaign via doAction -- Manual status refresh +- Execute via entity /action endpoint (e.g., /api/student_attendance/action) +- Manual status refresh via /action - Summary-level tracking ### Phase 2: Enhanced @@ -80,7 +82,7 @@ Create reusable phone message templates and trigger automated phone campaigns vi ## All Questions Answered ✅ -1. ✅ API KEY strategy: Per-user in settings +1. ✅ API KEY strategy: Per-user in additionalData 2. ✅ Billing: User's own account 3. ✅ First integration: Student attendance pivot 4. ✅ MVP content: TTS only @@ -89,7 +91,11 @@ Create reusable phone message templates and trigger automated phone campaigns vi 7. ✅ Permissions: Each user their own 8. ✅ Compliance: Handled by Yemot 9. ✅ YemotCall integration: Not related -10. ✅ Terminology: Provided in EN & HE +10. ✅ Terminology: "משלוח שיחות" for Hebrew campaign + +**Additional Clarifications:** +11. ✅ No custom routes - use standard BaseEntityModule only +12. ✅ Entity services (like StudentAttendanceService) handle phone extraction and call PhoneCampaignService ## Success Criteria @@ -103,6 +109,6 @@ Create reusable phone message templates and trigger automated phone campaigns vi 1. ✅ HLD approved with all decisions 2. Create Phase 1 implementation tickets -3. Add User.yemotApiKey migration +3. Configure user settings UI for additionalData.yemotApiKey 4. Set up Yemot test environment 5. Begin Phase 1 development