Code Coverage#643
Conversation
…tring so that nothing is removed from the coverage results if it is not Phase 3-4.
…ed line coverage checking if not for unitTests.
# Conflicts: # src/main/java/edu/byu/cs/autograder/test/TestGrader.java # src/main/java/edu/byu/cs/autograder/test/TestHelper.java
…e files showing if they have good coverage or not
…nd remove phase 6 partial credit
19mdavenport
left a comment
19mdavenport
left a comment
There was a problem hiding this comment.
I don't know if my feedback is wanted, but I noticed a couple of things that may be helpful.
In addition to the other two comments on lines, I noticed in several places you added the coverage config stuff to the "penalty" section in the config. I understand you may not want to add a whole new config endpoint, request, and other things, but do the code coverage settings qualify as penalties?
| if (coveragePercent > extraCreditPercent) { | ||
| return 1.05F; | ||
| } |
There was a problem hiding this comment.
Might you want the extra credit to be configurable from the frontend as well, rather than hardcoding it?
There was a problem hiding this comment.
extraCreditPercent is configurable! If it can't grab it from the dao though it defaults to 90%
There was a problem hiding this comment.
Oh wait did you mean the score?
I'll take your feedback! I figured they could qualify as penalties because if they don't reach a certain coverage their score is penalized :) |
There was a problem hiding this comment.
Besides my question about the Extra Credit portion, my version of the autograder did not work as expected.
I submitted my Phase 3 code to the autograder, and with line got 80.2%, and with branch I got 77%. This gave me extra credit version of the grade (26.25/25). Then when I went into the config and changed the percentages to be 80% for full points and 90% for extra credit, my 80.2% line coverage instead gave me 22.38/25, or about 90% of the points instead of full points.
I'll take a look at the code if there aren't any students coming in about what is causing this issue, but thought I would point out the problem and see if anyone else is experiencing issues.
EDIT: I realized that I had accidentally changed both to 90%, so that's why I only got 90% of the points that time around.
|
|
||
| return testResults.getNumTestsPassed() / totalTests; | ||
| if (coveragePercent >= extraCreditPercent) { | ||
| return 1.05F; |
There was a problem hiding this comment.
Did we want to use the Extra Credit Rubric set up by DavSmasher instead of returning 1.05, or is it just handled in this one spot?
|
You know what, ignore my prior message altogether. The first problem was because I didn't update the configuration in my database, so it defaulted to the extra credit and target percentage to be 0%, which I do indeed pass and get the extra credit. Then the second problem was I mistyped to update the percentages. TLDR; the code works, my comment was just a user error |
3 participants
Overview
Resolves #398. NOTE: this PR should not be merged until all other instruction has been updated and we are ready to start grading students on this requirement.
This PR changes how unit tests are graded, rather than counting numbers of tests, it evaluates the coverage on an area of source code that is relevant to the phase. The formula for grading is (student coverage / expected coverage). The idea is that unit tests are now graded on quality not quantity.
Details
Testing
Dependencies
The softwareconstruction repository with the instruction should be changed at the same time this goes into effect.
Future Work
Dr. Wilkerson has stated that we still may want to scan the code for bogus test assertions.
Additional Notes
Big shout out to @ZakkeryDaRebel for taking responsibility over this for a short while and @19mdavenport for writing the code that actually collects the coverage.