diff --git a/README.md b/README.md index 221acad..761c893 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ FluentAI now requires `OPENAI_API_KEY` for real lesson and conversation runs. Te [![Current FluentAI lesson screen — mocked model responses](docs/media/current-lesson.png)](docs/media/workflow-demo.mp4) -**[Watch the 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md) +**[Watch the narrated 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md) -This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, voice, and camera behavior are not shown. +This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, in-app voice, and camera behavior are not shown. Explanatory English narration was added after recording. [Captions](docs/media/narration.vtt) · [Narration script](docs/media/narration.json). ## What works now diff --git a/docs/DEMO.md b/docs/DEMO.md index 715778d..63463b9 100644 --- a/docs/DEMO.md +++ b/docs/DEMO.md @@ -30,7 +30,7 @@ npx playwright install chromium node scripts/record_portfolio_demo.cjs ``` -The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. The demo uses on-screen chapter labels instead of narration. +The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. This recreates the silent base recording with on-screen chapter labels; the published MP4 additionally contains the post-production narration described below. Optional environment overrides: `PYTHON_EXECUTABLE` for the interpreter, `CHROMIUM_EXECUTABLE` for an already installed browser, and `DEMO_PORT` for an available localhost port (default 7862). An existing Playwright installation can be resolved through `NODE_PATH`. The checked recording reused an installed Chromium binary; the commands above install Playwright's own compatible browser for a fresh reproduction. Timing, session IDs, and timestamps may vary; `recording.json` records the actual duration and source commit. @@ -39,3 +39,9 @@ To inspect the workflow manually, run `python scripts/portfolio_demo.py` and ope ## Checks and provenance The screenshot and video are generated by the two scripts above. `recording.json` identifies the pre-recording source commit and automated assertions. The Git commit containing the artifacts also contains the exact harness and driver. Original design mockups and their notices remain in `mockups/`. The repository's MIT license covers Johan Vaz's original work; third-party dependencies retain their own terms. + +## Explanatory narration + +English voiceover and optional embedded captions explain the committed recording. [Timed cue text and provenance](media/narration.json) and [WebVTT captions](media/narration.vtt) accompany the MP4. The generic synthetic Kokoro `af_heart` voice was generated locally after capture; it is not FluentAI's Realtime voice feature. + +The original H.264 video stream and its timing are unchanged. For another recording, re-time the cues to the captured actions, generate narration within those windows, and mux audio/captions while copying the video stream. The capture command does not generate speech. The narration manifest preserves source/output hashes, and Git history retains the silent original. Validation on 13 September 2026 confirmed unchanged video packets, AAC narration, ten captions, and all speech ending inside its allotted window. diff --git a/docs/media/narration.json b/docs/media/narration.json new file mode 100644 index 0000000..5afa984 --- /dev/null +++ b/docs/media/narration.json @@ -0,0 +1,87 @@ +{ + "generated_at": "2026-09-13", + "engine": "kokoro-onnx 0.6.1 / Kokoro v1.0", + "voice": "af_heart", + "language": "en-US", + "purpose": "Synthetic explanatory voiceover added in post-production; not app-generated speech.", + "source_video_sha256": "0507e4e7f6e495552f4c7d7c065b642256236b7af1ef1668476f3d02231595ba", + "output_video_sha256": "4cf66e258ca8003fb10239d44e62486a27065febecdaa2e46816d9c887585d73", + "source_duration_seconds": 82.7, + "video_stream": "Copied unchanged; no cuts or speed changes.", + "audio_codec": "AAC", + "audio_sample_rate": 24000, + "model_source": "https://github.com/thewh1teagle/kokoro-onnx/releases/tag/model-files-v1.0", + "model_license": "Apache-2.0", + "cues": [ + { + "start": 0.5, + "end": 7.2, + "text": "Fluent A I links lessons with learner memory. This demo uses scripted model responses.", + "actual_end": 6.047, + "speed": 0.95 + }, + { + "start": 7.8, + "end": 14.2, + "text": "A temporary learner profile opens a past-tense lesson, with a visible selection reason.", + "actual_end": 13.176, + "speed": 0.95 + }, + { + "start": 14.8, + "end": 25.8, + "text": "The learner answers six questions, mixing vocabulary, sentences, and translation. One answer is deliberately wrong so we can follow the feedback.", + "actual_end": 24.464, + "speed": 0.95 + }, + { + "start": 26.5, + "end": 33.3, + "text": "Five answers are correct. The mistake gets feedback and becomes a scheduled review item.", + "actual_end": 32.175, + "speed": 0.95 + }, + { + "start": 34.0, + "end": 40.8, + "text": "Back home, the score, experience points, and review queue reflect the completed lesson.", + "actual_end": 39.227, + "speed": 0.95 + }, + { + "start": 41.8, + "end": 47.3, + "text": "Now text practice picks up the same past-tense topic.", + "actual_end": 45.021, + "speed": 0.95 + }, + { + "start": 48.0, + "end": 53.4, + "text": "A short Spanish reply receives feedback and a follow-up prompt.", + "actual_end": 51.947, + "speed": 0.95 + }, + { + "start": 54.0, + "end": 60.8, + "text": "A second reply continues the exchange. These are scripted responses, exercising the app's practice workflow.", + "actual_end": 60.613, + "speed": 1.083 + }, + { + "start": 62.0, + "end": 68.8, + "text": "The memory inspector exposes the saved profile, skills, topic scores, and mistakes.", + "actual_end": 67.675, + "speed": 0.95 + }, + { + "start": 70.0, + "end": 81.8, + "text": "Reloading keeps the lesson result and review queue. This demonstrates saved progress across the workflow; live voice and model quality are outside this recording.", + "actual_end": 80.176, + "speed": 0.95 + } + ] +} diff --git a/docs/media/narration.vtt b/docs/media/narration.vtt new file mode 100644 index 0000000..5bbde25 --- /dev/null +++ b/docs/media/narration.vtt @@ -0,0 +1,56 @@ +WEBVTT + +1 +00:00:00.500 --> 00:00:06.047 +Fluent A I links lessons with learner memory. +This demo uses scripted model responses. + +2 +00:00:07.800 --> 00:00:13.176 +A temporary learner profile opens a past-tense +lesson, with a visible selection reason. + +3 +00:00:14.800 --> 00:00:24.464 +The learner answers six questions, mixing +vocabulary, sentences, and translation. One +answer is deliberately wrong so we can follow +the feedback. + +4 +00:00:26.500 --> 00:00:32.175 +Five answers are correct. The mistake gets +feedback and becomes a scheduled review item. + +5 +00:00:34.000 --> 00:00:39.227 +Back home, the score, experience points, and +review queue reflect the completed lesson. + +6 +00:00:41.800 --> 00:00:45.021 +Now text practice picks up the same past-tense +topic. + +7 +00:00:48.000 --> 00:00:51.947 +A short Spanish reply receives feedback and a +follow-up prompt. + +8 +00:00:54.000 --> 00:01:00.613 +A second reply continues the exchange. These are +scripted responses, exercising the app's +practice workflow. + +9 +00:01:02.000 --> 00:01:07.675 +The memory inspector exposes the saved profile, +skills, topic scores, and mistakes. + +10 +00:01:10.000 --> 00:01:20.176 +Reloading keeps the lesson result and review +queue. This demonstrates saved progress across +the workflow; live voice and model quality are +outside this recording. diff --git a/docs/media/recording.json b/docs/media/recording.json index c6121c5..204ce7b 100644 --- a/docs/media/recording.json +++ b/docs/media/recording.json @@ -1,7 +1,7 @@ { "source_commit": "85241bc625a2435324fffeca0bda2b0847d3f143", "duration_seconds": 82.7, - "size_bytes": 1312712, + "size_bytes": 2226409, "viewport": "1440x1000", "model_boundary": "Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling", "external_browser_requests": "Blocked", @@ -16,5 +16,8 @@ "Realtime voice", "Camera", "Signed desktop distribution" - ] + ], + "original_capture_size_bytes": 1312712, + "sha256": "4cf66e258ca8003fb10239d44e62486a27065febecdaa2e46816d9c887585d73", + "post_production": "Synthetic explanatory narration and captions added; see narration.json. In-app realtime voice remains excluded." } diff --git a/docs/media/workflow-demo.mp4 b/docs/media/workflow-demo.mp4 index 99b309d..5111619 100644 Binary files a/docs/media/workflow-demo.mp4 and b/docs/media/workflow-demo.mp4 differ diff --git a/notes.md b/notes.md index b19d650..fbe8b59 100644 --- a/notes.md +++ b/notes.md @@ -28,3 +28,5 @@ The full construction log and original five design proposals remain available in ## Portfolio recording (2026-09-11) Added a current-app screenshot and 83-second workflow recording using a labelled mock-provider harness and temporary learner data. Real bridge/UI checks verify quiz XP, review scheduling, and persistence after reload. The recording is not evidence of live model, voice, camera, or signed-app quality. Python/JS regressions and Ruff passed; see [demo reproduction and provenance](docs/DEMO.md). + +The same recording gained explanatory synthetic narration and captions on 2026-09-13. Video packets and timing remain unchanged; AAC audio, ten cue windows, and source/output hashes were verified. This is a media edit, not acceptance of the app's Realtime voice feature.