Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Johan Vaz

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ FluentAI is an OpenAI-powered agentic language-learning demo. It feels like a co

FluentAI now requires `OPENAI_API_KEY` for real lesson and conversation runs. Tests use mocked OpenAI responses so CI never needs secrets.

[![Current FluentAI lesson screen — mocked model responses](docs/media/current-lesson.png)](docs/media/workflow-demo.mp4)

**[Watch the 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md)

This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, voice, and camera behavior are not shown.

## What works now

- **Lesson Mode**: personalized mini lesson, vocabulary, grammar, examples, adaptive quiz, grading, XP, weak-topic updates.
Expand Down Expand Up @@ -178,3 +184,7 @@ See [architecture and memory](docs/GAP_AND_DESIGN.md), [privacy controls](docs/O
- Add richer lesson packs for Hindi and French.
- Add stable browser-based voice regression tests with mocked Realtime events.
- Add camera-frame fixtures for vision-context tests.

## License

[MIT](LICENSE), copyright Johan Vaz. Dependencies and third-party materials retain their own licenses and notices.
41 changes: 41 additions & 0 deletions docs/DEMO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Recorded workflow demo

[Watch the MP4](media/workflow-demo.mp4) or inspect the [current lesson screenshot](media/current-lesson.png). Both are captured from the current shared browser/Electron renderer served through the real Python HTTP bridge, not from the historical mockups.

## What the recording proves

| Approximate time | Workflow |
| --- | --- |
| 0–6 s | Home loads an isolated learner profile with no prior practice |
| 6–26 s | Start a lesson from weak-topic memory; answer six quiz questions |
| 26–41 s | Submit one deliberate mistake; see feedback, XP, and scheduled review |
| 41–61 s | Practice two text conversation replies using the same learner memory |
| 61–83 s | Inspect memory, reload the app, and see persisted progress |

The yellow banner remains visible throughout: **mocked model responses, temporary learner profile, no API calls**. The Python harness replaces the provider with deterministic curriculum/tutor scaffolding. The renderer, HTTP bridge, local grading, transaction logic, review scheduling, and persistence remain real. It changes provider-success log wording so the recording does not claim a live OpenAI call.

This is workflow evidence, not a demonstration of OpenAI output quality, language-course quality, Realtime voice, camera accuracy, or desktop signing/distribution. The screenshot includes the same disclosure. No microphone/camera permission is requested. No personal profile, `.env`, or API credential is needed.

## Reproduce

Use Python 3.10+, Node.js, FFmpeg (`ffmpeg` and `ffprobe` on PATH), and a Playwright Chromium installation. From the repository root:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[dev]'
npm ci
npm install --no-save --package-lock=false playwright@1.62.1
npx playwright install chromium
node scripts/record_portfolio_demo.cjs
```

The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. The demo uses on-screen chapter labels instead of narration.

Optional environment overrides: `PYTHON_EXECUTABLE` for the interpreter, `CHROMIUM_EXECUTABLE` for an already installed browser, and `DEMO_PORT` for an available localhost port (default 7862). An existing Playwright installation can be resolved through `NODE_PATH`. The checked recording reused an installed Chromium binary; the commands above install Playwright's own compatible browser for a fresh reproduction. Timing, session IDs, and timestamps may vary; `recording.json` records the actual duration and source commit.

To inspect the workflow manually, run `python scripts/portfolio_demo.py` and open `http://127.0.0.1:7861`. Ctrl-C deletes the temporary state. This explicit development harness is not a production offline-tutor mode; ordinary app entrypoints continue to require OpenAI.

## Checks and provenance

The screenshot and video are generated by the two scripts above. `recording.json` identifies the pre-recording source commit and automated assertions. The Git commit containing the artifacts also contains the exact harness and driver. Original design mockups and their notices remain in `mockups/`. The repository's MIT license covers Johan Vaz's original work; third-party dependencies retain their own terms.
Binary file added docs/media/current-lesson.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 20 additions & 0 deletions docs/media/recording.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"source_commit": "85241bc625a2435324fffeca0bda2b0847d3f143",
"duration_seconds": 82.7,
"size_bytes": 1312712,
"viewport": "1440x1000",
"model_boundary": "Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling",
"external_browser_requests": "Blocked",
"checks": [
"Lesson awarded XP",
"Review queue populated",
"XP and review queue survive page reload",
"No browser errors or failed bridge responses"
],
"excluded": [
"Live model quality",
"Realtime voice",
"Camera",
"Signed desktop distribution"
]
}
Binary file added docs/media/workflow-demo.mp4
Binary file not shown.
4 changes: 4 additions & 0 deletions notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,7 @@ Current behavior is documented in [architecture](docs/GAP_AND_DESIGN.md), [curri
## Provenance

The full construction log and original five design proposals remain available in [the pre-cleanup Git snapshot](https://github.com/Jo2234/FluentAI/tree/d923a6a49243ed19cbad3c3470e165157f2c34b0). In that snapshot, `notes.md` records dated work and validation, and `docs/` contains the original north-star, schema, onboarding, curriculum, and packaging proposals. Their work-package assignments, future-tense descriptions, and old source line numbers are historical, not current implementation instructions. Git history retains authorship and the evolution of those decisions.

## Portfolio recording (2026-09-11)

Added a current-app screenshot and 83-second workflow recording using a labelled mock-provider harness and temporary learner data. Real bridge/UI checks verify quiz XP, review scheduling, and persistence after reload. The recording is not evidence of live model, voice, camera, or signed-app quality. Python/JS regressions and Ruff passed; see [demo reproduction and provenance](docs/DEMO.md).
82 changes: 82 additions & 0 deletions scripts/portfolio_demo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""Local recording harness. Real renderer/bridge; mocked provider; disposable state.

Not an application fallback. Never reads repository learner state or API keys.
"""
from __future__ import annotations

import argparse
import copy
import sys
import tempfile
from pathlib import Path
from unittest.mock import patch

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from fluent_ai import desktop_bridge, web
from fluent_ai.state import default_state, profile_state, save_state, utc_now


class DemoProvider:
available = True
model = "mocked-portfolio-fixture"
last_error = None

def status(self):
return "Mocked model responses — portfolio workflow recording."

def enhance_lesson(self, state, lesson):
result = copy.deepcopy(lesson)
# The production bridge requires this provider-success marker. The
# recording banner and logs explicitly identify the substituted service.
result["source"] = "openai"
return result

def conversation_tutor_reply(self, topic, state, transcript, phase, fallback):
return fallback


class DemoHandler(web.FluentAIHandler):
def _renderer_html(self):
banner = '''<div id="recordingDisclosure" style="position:fixed;top:0;left:0;
width:100%;z-index:99999;background:#fef3c7;color:#422006;padding:12px 24px;
font:600 15px system-ui;box-sizing:border-box;">
WORKFLOW DEMO · Mocked model responses · Temporary learner profile · No API calls
<span id="recordingChapter" style="float:right">Home → Lesson → Review → Conversation</span>
</div><style>body {padding-top:48px !important;}</style>'''
return super()._renderer_html().replace("<body>", "<body>" + banner)

def _run_bridge_command(self, command, body):
# Do not imply a live model call in the visible agent log.
result = super()._run_bridge_command(command, body)
result["logs"] = [
line.replace("[OpenAI Model Agent]", "[Mock Provider]")
.replace("with the OpenAI Responses API", "from the demo fixture")
for line in result.get("logs", [])
]
return result


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--port", type=int, default=7861)
args = parser.parse_args()
with tempfile.TemporaryDirectory(prefix="fluentai-recording-") as temp:
state = default_state("Spanish")
state["learner"].update(display_name="Demo Learner", onboarded_at=utc_now())
profile_state(state)["placement_completed_at"] = utc_now()
path = Path(temp) / "progress.json"
save_state(path, state)
DemoHandler.state_path = path
DemoHandler.language = "Spanish"
with patch.object(web, "OpenAIProvider", DemoProvider), patch.object(desktop_bridge, "OpenAIProvider", DemoProvider):
with web.ThreadingHTTPServer(("127.0.0.1", args.port), DemoHandler) as server:
print(f"Mocked portfolio recording: http://127.0.0.1:{args.port}", flush=True)
try:
server.serve_forever()
except KeyboardInterrupt:
pass


if __name__ == "__main__":
main()
121 changes: 121 additions & 0 deletions scripts/record_portfolio_demo.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
#!/usr/bin/env node
// Playwright drives the real renderer; only the separate Python harness mocks AI.
const assert = require('node:assert/strict');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { spawn, execFileSync } = require('node:child_process');
const { chromium } = require('playwright');

(async () => {
const root = path.resolve(__dirname, '..');
const output = path.join(root, 'docs', 'media');
const temp = fs.mkdtempSync(path.join(os.tmpdir(), 'fluentai-video-'));
const port = process.env.DEMO_PORT || '7862';
const url = `http://127.0.0.1:${port}`;
const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith('OPENAI_')));
const server = spawn(process.env.PYTHON_EXECUTABLE || 'python', ['scripts/portfolio_demo.py', '--port', port], { cwd: root, env, stdio: ['ignore', 'pipe', 'inherit'] });
let browser;
try {
await new Promise((resolve, reject) => {
const timeout = setTimeout(() => reject(new Error('Demo server did not start')), 15000);
server.once('exit', code => { clearTimeout(timeout); reject(new Error(`Demo server exited: ${code}`)); });
server.stdout.once('data', () => { clearTimeout(timeout); resolve(); });
});
browser = await chromium.launch({ headless: true, ...(process.env.CHROMIUM_EXECUTABLE ? { executablePath: process.env.CHROMIUM_EXECUTABLE } : {}) });
const context = await browser.newContext({ viewport: { width: 1440, height: 1000 }, recordVideo: { dir: temp, size: { width: 1440, height: 1000 } } });
// Fonts and media must not contact external services during the recording.
await context.route('**/*', route => route.request().url().startsWith(url) ? route.continue() : route.abort());
const page = await context.newPage();
const errors = [];
page.on('pageerror', error => errors.push(error.message));
page.on('response', async response => {
if (response.url().includes('/api/bridge/')) {
try { const data = await response.json(); if (data.ok === false) errors.push(`${response.url()}: ${data.error}`); } catch {}
}
});
const start = Date.now();
const at = async second => { const remaining = second * 1000 - (Date.now() - start); if (remaining > 0) await page.waitForTimeout(remaining); };
const chapter = text => page.locator('#recordingChapter').evaluate((node, value) => { node.textContent = value; }, text);
await page.goto(url);
await page.getByRole('button', { name: "Start today's lesson", exact: true }).waitFor();
await chapter('1 / 5 · Start from local learner memory');
await at(6);
await page.getByRole('button', { name: "Start today's lesson", exact: true }).click();
await page.locator('#quizForm').waitFor();
await chapter('2 / 5 · Adaptive lesson + a visible selection reason');
await at(11);
fs.mkdirSync(output, { recursive: true });
await page.screenshot({ path: path.join(output, 'current-lesson.png') });
await at(14);
await page.locator('input[name="q0"][value="yesterday"]').check();
await page.locator('input[name="q1"]').pressSequentially('manana', { delay: 90 });
await page.locator('input[name="q2"]').pressSequentially('Ayer fui al mercado.', { delay: 70 });
await page.locator('input[name="q3"][value="Comi con mi familia."]').check();
await page.locator('input[name="q4"]').pressSequentially('I went', { delay: 90 });
await page.locator('input[name="q5"]').pressSequentially('Ayer fui al mercado.', { delay: 70 });
await at(26);
await chapter('3 / 5 · One deliberate mistake → feedback + spaced review');
const submitted = page.waitForResponse(response => response.url().endsWith('/lesson_submit'));
await page.locator('.submit-quiz').click();
const result = await (await submitted).json();
assert.equal(result.ok, true);
assert(result.profile.xp > 0);
assert(result.profile.review_count > 0);
assert.equal(result.summary.score, '5/6');
await page.locator('.question[data-index="1"]').scrollIntoViewIfNeeded();
await at(34);
await page.locator('#headerHomeBtn').click();
await page.waitForTimeout(800);
await at(41);
await chapter('4 / 5 · Text practice uses the same learner memory');
await page.locator('#headerConversationBtn').click();
await page.locator('#startTextFallbackBtn').click();
await page.locator('#chatInput').waitFor({ state: 'visible' });
await page.locator('#chatInput').fill('Ayer fui al mercado.');
await at(47);
await page.locator('#sendBtn').click();
await page.waitForTimeout(1200);
await page.locator('#chatInput').fill('Comi con mi familia.');
await at(53);
await page.locator('#sendBtn').click();
await at(61);
await page.locator('#headerHomeBtn').click();
await page.waitForTimeout(800);
await chapter('5 / 5 · Inspect the saved evidence and review queue');
await page.locator('#memoryInspectorPanel summary').click();
await page.locator('#memoryInspectorContent').waitFor({ state: 'visible' });
await page.locator('#memoryInspectorPanel').scrollIntoViewIfNeeded();
await at(69);
await page.reload();
await page.locator('#homePane').waitFor({ state: 'visible' });
await chapter('Reloaded · Progress persists · Voice/model quality not shown');
const saved = await page.request.post(`${url}/api/bridge/status`, { data: {} });
const status = await saved.json();
assert(status.profile.xp >= result.profile.xp);
assert(status.profile.review_count > 0);
await at(79);
assert.deepEqual(errors, []);
const video = page.video();
await context.close();
const raw = await video.path();
execFileSync('ffmpeg', ['-y', '-i', raw, '-an', '-r', '20', '-c:v', 'libx264', '-preset', 'slow', '-crf', '27', '-pix_fmt', 'yuv420p', '-movflags', '+faststart', path.join(output, 'workflow-demo.mp4')], { stdio: 'ignore' });
const metadata = JSON.parse(execFileSync('ffprobe', ['-v', 'quiet', '-print_format', 'json', '-show_format', '-show_streams', path.join(output, 'workflow-demo.mp4')], { encoding: 'utf8' }));
assert(Number(metadata.format.duration) >= 60 && Number(metadata.format.duration) <= 90);
fs.writeFileSync(path.join(output, 'recording.json'), JSON.stringify({
source_commit: execFileSync('git', ['rev-parse', 'HEAD'], { cwd: root, encoding: 'utf8' }).trim(),
duration_seconds: Number(metadata.format.duration),
size_bytes: Number(metadata.format.size),
viewport: '1440x1000',
model_boundary: 'Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling',
external_browser_requests: 'Blocked',
checks: ['Lesson awarded XP', 'Review queue populated', 'XP and review queue survive page reload', 'No browser errors or failed bridge responses'],
excluded: ['Live model quality', 'Realtime voice', 'Camera', 'Signed desktop distribution']
}, null, 2) + '\n');
console.log(JSON.stringify({ ok: true, duration: metadata.format.duration, bytes: metadata.format.size }));
} finally {
if (browser) await browser.close();
server.kill('SIGINT');
fs.rmSync(temp, { recursive: true, force: true });
}
})().catch(error => { console.error(error); process.exitCode = 1; });
Loading