Add failing tests for Issue #450: IndexError on empty test_results#463
Open
Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Open
Add failing tests for Issue #450: IndexError on empty test_results#463Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Serhan-Asad wants to merge 3 commits intopromptdriven:mainfrom
Conversation
…t_results This commit adds comprehensive unit and E2E tests that detect the bug reported in Issue promptdriven#450 where PDD crashes with IndexError when pytest fails to collect or execute tests. Root Cause: Line 213 in pdd/fix_error_loop.py uses output_data.get("test_results", [{}])[0] which assumes the default value protects against empty lists. However, Python's .get() only returns the default when the key is MISSING, not when it exists with an EMPTY list. Tests Added: - 6 unit tests in tests/test_fix_error_loop.py - Empty test_results list (primary bug) - Missing test_results key - Type validation for None, string, invalid dict - Integration test for collection failures - 10 E2E tests in tests/test_e2e_issue_450_pytest_collection_failure.py - Complete user workflow scenarios - Real-world pytest failure cases (ImportError, SyntaxError, etc.) - Edge cases with malformed data All tests currently FAIL on the buggy code (confirming they detect the bug). They will PASS once the fix is implemented. Related: promptdriven#450 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds failing tests that detect the bug reported in #450 where PDD crashes with
IndexError: list index out of rangewhen pytest fails to collect or execute tests.Test Files
tests/test_fix_error_loop.py(6 new test cases)tests/test_e2e_issue_450_pytest_collection_failure.py(10 new test cases)What This PR Contains
Root Cause
Location:
pdd/fix_error_loop.py:213The bug is a logic error in the defensive programming pattern:
The default value
[{}]only applies when the key is MISSING, not when it exists with an EMPTY list. When pytest collection/execution fails and returns{"test_results": []}, accessing[0]on the empty list causesIndexError.Test Coverage
Unit Tests (6 cases)
test_resultslist - Primary bug (IndexError)test_resultskey - Verifies default value workstest_resultsis None - Type validation (TypeError)test_resultsis string - Type validation (AttributeError)E2E Tests (10 cases)
Next Steps
pdd/fix_error_loop.py:213test_resultslist is non-empty before accessing[0]test_resultsis a listRecommended Fix Approach
The issue proposes a robust solution with:
Fixes #450
Generated by PDD agentic bug workflow (Steps 1-10 complete)