Robot tests#163
Closed
thestumonkey wants to merge 39 commits into
Closed
Conversation
thestumonkey
commented
Nov 28, 2025
thestumonkey
commented
Contributor
- can run tests with run_test.sh
- there's a testing_guidelines.md which is referenced by claude
- There's a user guide to explain how to setup
test framework streams faster than realtime and so we need this to wait
- Runs Robot Framework test suite on all branch pushes - Triggers on changes to test files and backend source code - Publishes test results and HTML reports as artifacts - Includes test summary in workflow output
- Create fresh .env file instead of appending to template - Eliminates duplicate key issues with ADMIN_EMAIL, OPENAI_API_KEY, etc. - Add TRANSCRIPTION_PROVIDER configuration - Improve readability with comments
- Create .env.test in tests/setup/ instead of tests/ - Matches what test_env.py expects (Path(__file__).parent / '.env.test') - This was causing test setup failures in CI
- Use docker-compose-test.yml explicitly in all commands - Add better health check loops with progress indicators - Show container status and logs on failures - Increase timeout attempts with clearer error messages - Fix cleanup to use correct compose file - Remove push trigger (PR-only workflow)
- Fix TEST_AUDIO_FILE path: remove duplicate 'tests/' prefix - Add clear instructions for downloading HTML reports from artifacts - Improve summary output formatting
- Update .gitignore to allow tests/test_assets/*.wav files - Add DIY_Experts_Glass_Blowing test audio files (1min and 4min) - Install websockets package for audio_stream_library.py - Add OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_MODEL to test step - Required for backend imports when loading test libraries
- Add workers-test to service startup - Remove src mount from workers-test in CI compose file - Fixes chat session creation and job processing in tests
- Show last 50 lines of backend and worker logs - Runs always (not just on failure) to debug test issues - Helps diagnose chat session creation and job processing failures
- Required for LLM client initialization in chat service - Fixes chat session creation 500 errors - Uses default https://api.openai.com/v1 if not provided
|
| Metric | Count |
|---|---|
| ✅ Passed | 62 |
| ❌ Failed | 27 |
| 📊 Total | 89 |
📊 View Reports
GitHub Pages (Live Reports):
Download Artifacts:
- robot-test-reports-html - HTML reports
- robot-test-results-xml - XML output
|
| Metric | Count |
|---|---|
| ✅ Passed | 62 |
| ❌ Failed | 27 |
| 📊 Total | 89 |
📊 View Reports
GitHub Pages (Live Reports):
Download Artifacts:
- robot-test-reports-html - HTML reports
- robot-test-results-xml - XML output
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.