Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ FluentAI now requires `OPENAI_API_KEY` for real lesson and conversation runs. Te

[![Current FluentAI lesson screen — mocked model responses](docs/media/current-lesson.png)](docs/media/workflow-demo.mp4)

**[Watch the 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md)
**[Watch the narrated 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md)

This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, voice, and camera behavior are not shown.
This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, in-app voice, and camera behavior are not shown. Explanatory English narration was added after recording. [Captions](docs/media/narration.vtt) · [Narration script](docs/media/narration.json).

## What works now

Expand Down
8 changes: 7 additions & 1 deletion docs/DEMO.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ npx playwright install chromium
node scripts/record_portfolio_demo.cjs
```

The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. The demo uses on-screen chapter labels instead of narration.
The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. This recreates the silent base recording with on-screen chapter labels; the published MP4 additionally contains the post-production narration described below.

Optional environment overrides: `PYTHON_EXECUTABLE` for the interpreter, `CHROMIUM_EXECUTABLE` for an already installed browser, and `DEMO_PORT` for an available localhost port (default 7862). An existing Playwright installation can be resolved through `NODE_PATH`. The checked recording reused an installed Chromium binary; the commands above install Playwright's own compatible browser for a fresh reproduction. Timing, session IDs, and timestamps may vary; `recording.json` records the actual duration and source commit.

Expand All @@ -39,3 +39,9 @@ To inspect the workflow manually, run `python scripts/portfolio_demo.py` and ope
## Checks and provenance

The screenshot and video are generated by the two scripts above. `recording.json` identifies the pre-recording source commit and automated assertions. The Git commit containing the artifacts also contains the exact harness and driver. Original design mockups and their notices remain in `mockups/`. The repository's MIT license covers Johan Vaz's original work; third-party dependencies retain their own terms.

## Explanatory narration

English voiceover and optional embedded captions explain the committed recording. [Timed cue text and provenance](media/narration.json) and [WebVTT captions](media/narration.vtt) accompany the MP4. The generic synthetic Kokoro `af_heart` voice was generated locally after capture; it is not FluentAI's Realtime voice feature.

The original H.264 video stream and its timing are unchanged. For another recording, re-time the cues to the captured actions, generate narration within those windows, and mux audio/captions while copying the video stream. The capture command does not generate speech. The narration manifest preserves source/output hashes, and Git history retains the silent original. Validation on 13 September 2026 confirmed unchanged video packets, AAC narration, ten captions, and all speech ending inside its allotted window.
87 changes: 87 additions & 0 deletions docs/media/narration.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
{
"generated_at": "2026-09-13",
"engine": "kokoro-onnx 0.6.1 / Kokoro v1.0",
"voice": "af_heart",
"language": "en-US",
"purpose": "Synthetic explanatory voiceover added in post-production; not app-generated speech.",
"source_video_sha256": "0507e4e7f6e495552f4c7d7c065b642256236b7af1ef1668476f3d02231595ba",
"output_video_sha256": "4cf66e258ca8003fb10239d44e62486a27065febecdaa2e46816d9c887585d73",
"source_duration_seconds": 82.7,
"video_stream": "Copied unchanged; no cuts or speed changes.",
"audio_codec": "AAC",
"audio_sample_rate": 24000,
"model_source": "https://github.com/thewh1teagle/kokoro-onnx/releases/tag/model-files-v1.0",
"model_license": "Apache-2.0",
"cues": [
{
"start": 0.5,
"end": 7.2,
"text": "Fluent A I links lessons with learner memory. This demo uses scripted model responses.",
"actual_end": 6.047,
"speed": 0.95
},
{
"start": 7.8,
"end": 14.2,
"text": "A temporary learner profile opens a past-tense lesson, with a visible selection reason.",
"actual_end": 13.176,
"speed": 0.95
},
{
"start": 14.8,
"end": 25.8,
"text": "The learner answers six questions, mixing vocabulary, sentences, and translation. One answer is deliberately wrong so we can follow the feedback.",
"actual_end": 24.464,
"speed": 0.95
},
{
"start": 26.5,
"end": 33.3,
"text": "Five answers are correct. The mistake gets feedback and becomes a scheduled review item.",
"actual_end": 32.175,
"speed": 0.95
},
{
"start": 34.0,
"end": 40.8,
"text": "Back home, the score, experience points, and review queue reflect the completed lesson.",
"actual_end": 39.227,
"speed": 0.95
},
{
"start": 41.8,
"end": 47.3,
"text": "Now text practice picks up the same past-tense topic.",
"actual_end": 45.021,
"speed": 0.95
},
{
"start": 48.0,
"end": 53.4,
"text": "A short Spanish reply receives feedback and a follow-up prompt.",
"actual_end": 51.947,
"speed": 0.95
},
{
"start": 54.0,
"end": 60.8,
"text": "A second reply continues the exchange. These are scripted responses, exercising the app's practice workflow.",
"actual_end": 60.613,
"speed": 1.083
},
{
"start": 62.0,
"end": 68.8,
"text": "The memory inspector exposes the saved profile, skills, topic scores, and mistakes.",
"actual_end": 67.675,
"speed": 0.95
},
{
"start": 70.0,
"end": 81.8,
"text": "Reloading keeps the lesson result and review queue. This demonstrates saved progress across the workflow; live voice and model quality are outside this recording.",
"actual_end": 80.176,
"speed": 0.95
}
]
}
56 changes: 56 additions & 0 deletions docs/media/narration.vtt
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
WEBVTT

1
00:00:00.500 --> 00:00:06.047
Fluent A I links lessons with learner memory.
This demo uses scripted model responses.

2
00:00:07.800 --> 00:00:13.176
A temporary learner profile opens a past-tense
lesson, with a visible selection reason.

3
00:00:14.800 --> 00:00:24.464
The learner answers six questions, mixing
vocabulary, sentences, and translation. One
answer is deliberately wrong so we can follow
the feedback.

4
00:00:26.500 --> 00:00:32.175
Five answers are correct. The mistake gets
feedback and becomes a scheduled review item.

5
00:00:34.000 --> 00:00:39.227
Back home, the score, experience points, and
review queue reflect the completed lesson.

6
00:00:41.800 --> 00:00:45.021
Now text practice picks up the same past-tense
topic.

7
00:00:48.000 --> 00:00:51.947
A short Spanish reply receives feedback and a
follow-up prompt.

8
00:00:54.000 --> 00:01:00.613
A second reply continues the exchange. These are
scripted responses, exercising the app's
practice workflow.

9
00:01:02.000 --> 00:01:07.675
The memory inspector exposes the saved profile,
skills, topic scores, and mistakes.

10
00:01:10.000 --> 00:01:20.176
Reloading keeps the lesson result and review
queue. This demonstrates saved progress across
the workflow; live voice and model quality are
outside this recording.
7 changes: 5 additions & 2 deletions docs/media/recording.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"source_commit": "85241bc625a2435324fffeca0bda2b0847d3f143",
"duration_seconds": 82.7,
"size_bytes": 1312712,
"size_bytes": 2226409,
"viewport": "1440x1000",
"model_boundary": "Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling",
"external_browser_requests": "Blocked",
Expand All @@ -16,5 +16,8 @@
"Realtime voice",
"Camera",
"Signed desktop distribution"
]
],
"original_capture_size_bytes": 1312712,
"sha256": "4cf66e258ca8003fb10239d44e62486a27065febecdaa2e46816d9c887585d73",
"post_production": "Synthetic explanatory narration and captions added; see narration.json. In-app realtime voice remains excluded."
}
Binary file modified docs/media/workflow-demo.mp4
Binary file not shown.
2 changes: 2 additions & 0 deletions notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,5 @@ The full construction log and original five design proposals remain available in
## Portfolio recording (2026-09-11)

Added a current-app screenshot and 83-second workflow recording using a labelled mock-provider harness and temporary learner data. Real bridge/UI checks verify quiz XP, review scheduling, and persistence after reload. The recording is not evidence of live model, voice, camera, or signed-app quality. Python/JS regressions and Ruff passed; see [demo reproduction and provenance](docs/DEMO.md).

The same recording gained explanatory synthetic narration and captions on 2026-09-13. Video packets and timing remain unchanged; AAC audio, ten cue windows, and source/output hashes were verified. This is a media edit, not acceptance of the app's Realtime voice feature.
Loading