diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..a41accd --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Johan Vaz + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 2093613..221acad 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,12 @@ FluentAI is an OpenAI-powered agentic language-learning demo. It feels like a co FluentAI now requires `OPENAI_API_KEY` for real lesson and conversation runs. Tests use mocked OpenAI responses so CI never needs secrets. +[![Current FluentAI lesson screen — mocked model responses](docs/media/current-lesson.png)](docs/media/workflow-demo.mp4) + +**[Watch the 83-second workflow demo](docs/media/workflow-demo.mp4)** · [Download MP4](https://raw.githubusercontent.com/Jo2234/FluentAI/main/docs/media/workflow-demo.mp4) · [Reproduce the recording](docs/DEMO.md) + +This is the current browser app with **mocked model responses and temporary learner memory**, labelled throughout the recording. It shows a lesson, a deliberate quiz mistake, saved XP and spaced review, text conversation, and progress surviving reload. It demonstrates the interface and state flow; live model quality, voice, and camera behavior are not shown. + ## What works now - **Lesson Mode**: personalized mini lesson, vocabulary, grammar, examples, adaptive quiz, grading, XP, weak-topic updates. @@ -178,3 +184,7 @@ See [architecture and memory](docs/GAP_AND_DESIGN.md), [privacy controls](docs/O - Add richer lesson packs for Hindi and French. - Add stable browser-based voice regression tests with mocked Realtime events. - Add camera-frame fixtures for vision-context tests. + +## License + +[MIT](LICENSE), copyright Johan Vaz. Dependencies and third-party materials retain their own licenses and notices. diff --git a/docs/DEMO.md b/docs/DEMO.md new file mode 100644 index 0000000..715778d --- /dev/null +++ b/docs/DEMO.md @@ -0,0 +1,41 @@ +# Recorded workflow demo + +[Watch the MP4](media/workflow-demo.mp4) or inspect the [current lesson screenshot](media/current-lesson.png). Both are captured from the current shared browser/Electron renderer served through the real Python HTTP bridge, not from the historical mockups. + +## What the recording proves + +| Approximate time | Workflow | +| --- | --- | +| 0–6 s | Home loads an isolated learner profile with no prior practice | +| 6–26 s | Start a lesson from weak-topic memory; answer six quiz questions | +| 26–41 s | Submit one deliberate mistake; see feedback, XP, and scheduled review | +| 41–61 s | Practice two text conversation replies using the same learner memory | +| 61–83 s | Inspect memory, reload the app, and see persisted progress | + +The yellow banner remains visible throughout: **mocked model responses, temporary learner profile, no API calls**. The Python harness replaces the provider with deterministic curriculum/tutor scaffolding. The renderer, HTTP bridge, local grading, transaction logic, review scheduling, and persistence remain real. It changes provider-success log wording so the recording does not claim a live OpenAI call. + +This is workflow evidence, not a demonstration of OpenAI output quality, language-course quality, Realtime voice, camera accuracy, or desktop signing/distribution. The screenshot includes the same disclosure. No microphone/camera permission is requested. No personal profile, `.env`, or API credential is needed. + +## Reproduce + +Use Python 3.10+, Node.js, FFmpeg (`ffmpeg` and `ffprobe` on PATH), and a Playwright Chromium installation. From the repository root: + +```bash +python3 -m venv .venv +source .venv/bin/activate +python -m pip install -e '.[dev]' +npm ci +npm install --no-save --package-lock=false playwright@1.62.1 +npx playwright install chromium +node scripts/record_portfolio_demo.cjs +``` + +The recording script starts `scripts/portfolio_demo.py` on localhost, creates a disposable profile, drives visible controls, blocks external browser requests, and shuts down the server afterward. It checks for browser/bridge errors, quiz XP, scheduled review, and persistence after reload. It writes `docs/media/current-lesson.png`, `workflow-demo.mp4`, and `recording.json`; FFmpeg converts the captured browser video to a silent H.264 MP4. The demo uses on-screen chapter labels instead of narration. + +Optional environment overrides: `PYTHON_EXECUTABLE` for the interpreter, `CHROMIUM_EXECUTABLE` for an already installed browser, and `DEMO_PORT` for an available localhost port (default 7862). An existing Playwright installation can be resolved through `NODE_PATH`. The checked recording reused an installed Chromium binary; the commands above install Playwright's own compatible browser for a fresh reproduction. Timing, session IDs, and timestamps may vary; `recording.json` records the actual duration and source commit. + +To inspect the workflow manually, run `python scripts/portfolio_demo.py` and open `http://127.0.0.1:7861`. Ctrl-C deletes the temporary state. This explicit development harness is not a production offline-tutor mode; ordinary app entrypoints continue to require OpenAI. + +## Checks and provenance + +The screenshot and video are generated by the two scripts above. `recording.json` identifies the pre-recording source commit and automated assertions. The Git commit containing the artifacts also contains the exact harness and driver. Original design mockups and their notices remain in `mockups/`. The repository's MIT license covers Johan Vaz's original work; third-party dependencies retain their own terms. diff --git a/docs/media/current-lesson.png b/docs/media/current-lesson.png new file mode 100644 index 0000000..f00fe1a Binary files /dev/null and b/docs/media/current-lesson.png differ diff --git a/docs/media/recording.json b/docs/media/recording.json new file mode 100644 index 0000000..c6121c5 --- /dev/null +++ b/docs/media/recording.json @@ -0,0 +1,20 @@ +{ + "source_commit": "85241bc625a2435324fffeca0bda2b0847d3f143", + "duration_seconds": 82.7, + "size_bytes": 1312712, + "viewport": "1440x1000", + "model_boundary": "Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling", + "external_browser_requests": "Blocked", + "checks": [ + "Lesson awarded XP", + "Review queue populated", + "XP and review queue survive page reload", + "No browser errors or failed bridge responses" + ], + "excluded": [ + "Live model quality", + "Realtime voice", + "Camera", + "Signed desktop distribution" + ] +} diff --git a/docs/media/workflow-demo.mp4 b/docs/media/workflow-demo.mp4 new file mode 100644 index 0000000..99b309d Binary files /dev/null and b/docs/media/workflow-demo.mp4 differ diff --git a/notes.md b/notes.md index bf76c6c..b19d650 100644 --- a/notes.md +++ b/notes.md @@ -24,3 +24,7 @@ Current behavior is documented in [architecture](docs/GAP_AND_DESIGN.md), [curri ## Provenance The full construction log and original five design proposals remain available in [the pre-cleanup Git snapshot](https://github.com/Jo2234/FluentAI/tree/d923a6a49243ed19cbad3c3470e165157f2c34b0). In that snapshot, `notes.md` records dated work and validation, and `docs/` contains the original north-star, schema, onboarding, curriculum, and packaging proposals. Their work-package assignments, future-tense descriptions, and old source line numbers are historical, not current implementation instructions. Git history retains authorship and the evolution of those decisions. + +## Portfolio recording (2026-09-11) + +Added a current-app screenshot and 83-second workflow recording using a labelled mock-provider harness and temporary learner data. Real bridge/UI checks verify quiz XP, review scheduling, and persistence after reload. The recording is not evidence of live model, voice, camera, or signed-app quality. Python/JS regressions and Ruff passed; see [demo reproduction and provenance](docs/DEMO.md). diff --git a/scripts/portfolio_demo.py b/scripts/portfolio_demo.py new file mode 100644 index 0000000..9671ae9 --- /dev/null +++ b/scripts/portfolio_demo.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +"""Local recording harness. Real renderer/bridge; mocked provider; disposable state. + +Not an application fallback. Never reads repository learner state or API keys. +""" +from __future__ import annotations + +import argparse +import copy +import sys +import tempfile +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +from fluent_ai import desktop_bridge, web +from fluent_ai.state import default_state, profile_state, save_state, utc_now + + +class DemoProvider: + available = True + model = "mocked-portfolio-fixture" + last_error = None + + def status(self): + return "Mocked model responses — portfolio workflow recording." + + def enhance_lesson(self, state, lesson): + result = copy.deepcopy(lesson) + # The production bridge requires this provider-success marker. The + # recording banner and logs explicitly identify the substituted service. + result["source"] = "openai" + return result + + def conversation_tutor_reply(self, topic, state, transcript, phase, fallback): + return fallback + + +class DemoHandler(web.FluentAIHandler): + def _renderer_html(self): + banner = '''
+ WORKFLOW DEMO · Mocked model responses · Temporary learner profile · No API calls + Home → Lesson → Review → Conversation +
''' + return super()._renderer_html().replace("", "" + banner) + + def _run_bridge_command(self, command, body): + # Do not imply a live model call in the visible agent log. + result = super()._run_bridge_command(command, body) + result["logs"] = [ + line.replace("[OpenAI Model Agent]", "[Mock Provider]") + .replace("with the OpenAI Responses API", "from the demo fixture") + for line in result.get("logs", []) + ] + return result + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--port", type=int, default=7861) + args = parser.parse_args() + with tempfile.TemporaryDirectory(prefix="fluentai-recording-") as temp: + state = default_state("Spanish") + state["learner"].update(display_name="Demo Learner", onboarded_at=utc_now()) + profile_state(state)["placement_completed_at"] = utc_now() + path = Path(temp) / "progress.json" + save_state(path, state) + DemoHandler.state_path = path + DemoHandler.language = "Spanish" + with patch.object(web, "OpenAIProvider", DemoProvider), patch.object(desktop_bridge, "OpenAIProvider", DemoProvider): + with web.ThreadingHTTPServer(("127.0.0.1", args.port), DemoHandler) as server: + print(f"Mocked portfolio recording: http://127.0.0.1:{args.port}", flush=True) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + + +if __name__ == "__main__": + main() diff --git a/scripts/record_portfolio_demo.cjs b/scripts/record_portfolio_demo.cjs new file mode 100644 index 0000000..796a419 --- /dev/null +++ b/scripts/record_portfolio_demo.cjs @@ -0,0 +1,121 @@ +#!/usr/bin/env node +// Playwright drives the real renderer; only the separate Python harness mocks AI. +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const { spawn, execFileSync } = require('node:child_process'); +const { chromium } = require('playwright'); + +(async () => { + const root = path.resolve(__dirname, '..'); + const output = path.join(root, 'docs', 'media'); + const temp = fs.mkdtempSync(path.join(os.tmpdir(), 'fluentai-video-')); + const port = process.env.DEMO_PORT || '7862'; + const url = `http://127.0.0.1:${port}`; + const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith('OPENAI_'))); + const server = spawn(process.env.PYTHON_EXECUTABLE || 'python', ['scripts/portfolio_demo.py', '--port', port], { cwd: root, env, stdio: ['ignore', 'pipe', 'inherit'] }); + let browser; + try { + await new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new Error('Demo server did not start')), 15000); + server.once('exit', code => { clearTimeout(timeout); reject(new Error(`Demo server exited: ${code}`)); }); + server.stdout.once('data', () => { clearTimeout(timeout); resolve(); }); + }); + browser = await chromium.launch({ headless: true, ...(process.env.CHROMIUM_EXECUTABLE ? { executablePath: process.env.CHROMIUM_EXECUTABLE } : {}) }); + const context = await browser.newContext({ viewport: { width: 1440, height: 1000 }, recordVideo: { dir: temp, size: { width: 1440, height: 1000 } } }); + // Fonts and media must not contact external services during the recording. + await context.route('**/*', route => route.request().url().startsWith(url) ? route.continue() : route.abort()); + const page = await context.newPage(); + const errors = []; + page.on('pageerror', error => errors.push(error.message)); + page.on('response', async response => { + if (response.url().includes('/api/bridge/')) { + try { const data = await response.json(); if (data.ok === false) errors.push(`${response.url()}: ${data.error}`); } catch {} + } + }); + const start = Date.now(); + const at = async second => { const remaining = second * 1000 - (Date.now() - start); if (remaining > 0) await page.waitForTimeout(remaining); }; + const chapter = text => page.locator('#recordingChapter').evaluate((node, value) => { node.textContent = value; }, text); + await page.goto(url); + await page.getByRole('button', { name: "Start today's lesson", exact: true }).waitFor(); + await chapter('1 / 5 · Start from local learner memory'); + await at(6); + await page.getByRole('button', { name: "Start today's lesson", exact: true }).click(); + await page.locator('#quizForm').waitFor(); + await chapter('2 / 5 · Adaptive lesson + a visible selection reason'); + await at(11); + fs.mkdirSync(output, { recursive: true }); + await page.screenshot({ path: path.join(output, 'current-lesson.png') }); + await at(14); + await page.locator('input[name="q0"][value="yesterday"]').check(); + await page.locator('input[name="q1"]').pressSequentially('manana', { delay: 90 }); + await page.locator('input[name="q2"]').pressSequentially('Ayer fui al mercado.', { delay: 70 }); + await page.locator('input[name="q3"][value="Comi con mi familia."]').check(); + await page.locator('input[name="q4"]').pressSequentially('I went', { delay: 90 }); + await page.locator('input[name="q5"]').pressSequentially('Ayer fui al mercado.', { delay: 70 }); + await at(26); + await chapter('3 / 5 · One deliberate mistake → feedback + spaced review'); + const submitted = page.waitForResponse(response => response.url().endsWith('/lesson_submit')); + await page.locator('.submit-quiz').click(); + const result = await (await submitted).json(); + assert.equal(result.ok, true); + assert(result.profile.xp > 0); + assert(result.profile.review_count > 0); + assert.equal(result.summary.score, '5/6'); + await page.locator('.question[data-index="1"]').scrollIntoViewIfNeeded(); + await at(34); + await page.locator('#headerHomeBtn').click(); + await page.waitForTimeout(800); + await at(41); + await chapter('4 / 5 · Text practice uses the same learner memory'); + await page.locator('#headerConversationBtn').click(); + await page.locator('#startTextFallbackBtn').click(); + await page.locator('#chatInput').waitFor({ state: 'visible' }); + await page.locator('#chatInput').fill('Ayer fui al mercado.'); + await at(47); + await page.locator('#sendBtn').click(); + await page.waitForTimeout(1200); + await page.locator('#chatInput').fill('Comi con mi familia.'); + await at(53); + await page.locator('#sendBtn').click(); + await at(61); + await page.locator('#headerHomeBtn').click(); + await page.waitForTimeout(800); + await chapter('5 / 5 · Inspect the saved evidence and review queue'); + await page.locator('#memoryInspectorPanel summary').click(); + await page.locator('#memoryInspectorContent').waitFor({ state: 'visible' }); + await page.locator('#memoryInspectorPanel').scrollIntoViewIfNeeded(); + await at(69); + await page.reload(); + await page.locator('#homePane').waitFor({ state: 'visible' }); + await chapter('Reloaded · Progress persists · Voice/model quality not shown'); + const saved = await page.request.post(`${url}/api/bridge/status`, { data: {} }); + const status = await saved.json(); + assert(status.profile.xp >= result.profile.xp); + assert(status.profile.review_count > 0); + await at(79); + assert.deepEqual(errors, []); + const video = page.video(); + await context.close(); + const raw = await video.path(); + execFileSync('ffmpeg', ['-y', '-i', raw, '-an', '-r', '20', '-c:v', 'libx264', '-preset', 'slow', '-crf', '27', '-pix_fmt', 'yuv420p', '-movflags', '+faststart', path.join(output, 'workflow-demo.mp4')], { stdio: 'ignore' }); + const metadata = JSON.parse(execFileSync('ffprobe', ['-v', 'quiet', '-print_format', 'json', '-show_format', '-show_streams', path.join(output, 'workflow-demo.mp4')], { encoding: 'utf8' })); + assert(Number(metadata.format.duration) >= 60 && Number(metadata.format.duration) <= 90); + fs.writeFileSync(path.join(output, 'recording.json'), JSON.stringify({ + source_commit: execFileSync('git', ['rev-parse', 'HEAD'], { cwd: root, encoding: 'utf8' }).trim(), + duration_seconds: Number(metadata.format.duration), + size_bytes: Number(metadata.format.size), + viewport: '1440x1000', + model_boundary: 'Mocked Python provider; actual renderer, bridge, grading, state persistence, and review scheduling', + external_browser_requests: 'Blocked', + checks: ['Lesson awarded XP', 'Review queue populated', 'XP and review queue survive page reload', 'No browser errors or failed bridge responses'], + excluded: ['Live model quality', 'Realtime voice', 'Camera', 'Signed desktop distribution'] + }, null, 2) + '\n'); + console.log(JSON.stringify({ ok: true, duration: metadata.format.duration, bytes: metadata.format.size })); + } finally { + if (browser) await browser.close(); + server.kill('SIGINT'); + fs.rmSync(temp, { recursive: true, force: true }); + } +})().catch(error => { console.error(error); process.exitCode = 1; });