Add GitHub Action for Automated Unit Testing on Pull Requests

[brylie]

Objective

Set up a GitHub Action to automatically run our pytest suite on every pull request, ensuring that all tests pass before changes can be merged into the main branch.

Description

We need to implement a Continuous Integration (CI) workflow using GitHub Actions that will run our pytest suite automatically whenever a pull request is opened or updated. This will help maintain code quality, catch potential issues early, and ensure that our main branch remains stable.

Tasks

1. Create a new GitHub Actions workflow file:

   • Create a .github/workflows/ directory in the repository if it doesn't exist
   • Add a new YAML file named pytest.yml in this directory

2. Configure the GitHub Action workflow:

   • Set up triggers for pull requests to the main branch
   • Define the job to run on the latest Ubuntu runner
   • Set up the Python environment (use the same version as our project)
   • Install project dependencies
   • Run pytest

3. Implement caching for pip dependencies to speed up workflow runs

4. Configure pytest settings:

   • Ensure the action can locate and run all tests
   • Set appropriate pytest arguments (e.g., verbosity, fail fast)

5. Handle test failures:

   • Configure the action to fail if any tests fail
   • Set up informative output for failed tests

6. (Optional) Set up test coverage reporting:

   • Add a step to generate a coverage report
   • Configure a way to display or comment the coverage results on the PR

7. Update the repository settings:

   • Require status checks to pass before merging
   • Add the new GitHub Action as a required status check for the main branch

8. Update documentation:

   • Add information about the CI process to the project README
   • Include a badge for the test status in the README

Implementation Details

Here's a basic structure for the GitHub Action YAML file (pytest.yml):

name: Python Tests

on:
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.x'  # Specify the version your project uses
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run pytest
      run: pytest tests/

    # Optional: Add coverage reporting
    - name: Run pytest with coverage
      run: pytest --cov=./ --cov-report=xml
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v2
      with:
        fail_ci_if_error: true

Acceptance Criteria

• The GitHub Action successfully runs on all pull requests to the main branch
• All tests in the pytest suite are executed
• The action fails if any tests fail, preventing merging of broken code
• Test results are clearly visible in the pull request interface
• Dependencies are cached to speed up subsequent runs
• (Optional) Test coverage is reported and visible in the pull request
• The process is documented in the project README, including a status badge
• Merging to the main branch is blocked if tests fail

Additional Considerations

• Consider running tests on multiple Python versions if needed
• Think about adding linting checks (e.g., flake8) in the same or a separate workflow
• Ensure that sensitive information (like API keys) is properly handled in tests and CI

Resources

• GitHub Actions Documentation
• pytest Documentation
• Codecov GitHub Action
• GitHub Marketplace for Actions

Future Enhancements

• Implement parallel testing if the test suite grows large
• Add performance benchmarks to catch significant performance regressions
• Integrate with a code quality tool for additional automated checks