-
Notifications
You must be signed in to change notification settings - Fork 2
docs: code coverage rules documented in ADR #3 #244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
1cc3bff
docs: test coverage rules documented in ADR #3
Geenag 355cc23
docs: update README.md in order to be consistent with README.md from …
Geenag a1d7fb1
docs: retours PR
Geenag 8eeba6d
docs: adr test strategy
Geenag 5c02a34
docs: retours PR 2
Geenag File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| # Definition of standards for testing | ||
|
|
||
| ## Context and Problem Statement | ||
|
|
||
| As any application, we want IroCO2 to be reliable throughout time. To that end, tests should be written all along code writing, but those tests should not depend on the person who write code. Thus, a definition of standards appears to be mandatory for two reasons : | ||
| - this will facilitate test writing, as well as test reviewing | ||
| - this will ensure the application is resilient, and prevent any regression to be silently implemented | ||
|
|
||
| The frame we drew with standards to be respected is settled upon two lines of conduct : code coverage as an indicator, and test strategy as a mindset. | ||
|
|
||
| --- | ||
|
|
||
| ## 1. Code coverage | ||
|
|
||
| The definition of standards around coverage was mainly based on experience and knowledge of the Ippon Technology team in place at the moment of said definition. Nevertheless, online resources can easily be found if you wish to deepen your comprehension of these rules definition (see for example the article ["What is code coverage"](https://www.atlassian.com/continuous-delivery/software-testing/code-coverage)). | ||
|
|
||
| ### Objectives for code coverage by perimeter | ||
|
|
||
| Since each side of IroCO2 application has its specificities, minimal coverage rate was established project by project. | ||
|
|
||
| Please keep in mind, in particular for Back-end and Front-end projects, that coverage philosophy of IroCO2 team go beyond quantity of tests, thinking also about **quality** and **pertinence** of each test. | ||
|
|
||
| | Perimeter | Objective | | ||
| | -------------- | --------- | | ||
| | Infrastructure | 50%* | | ||
| | Back-end | 90% | | ||
| | Front-end | 80%** | | ||
|
|
||
| > \* : *indicative value which reflect the ambition to test main components with Terraform Test* | ||
| > | ||
| > \*\* : *defined without clear idea of necessary time to catch up on that objective, so it can be considered as a minimal* | ||
|
|
||
| --- | ||
|
|
||
| ## 2. Test strategy | ||
|
|
||
| The main idea behind our test strategy is the Test Pyramid, whose base is made of Unit tests, End-to-end tests at the top, with Integration tests between (see this [pragmatic article](https://martinfowler.com/articles/practical-test-pyramid.html) of Martin Fowler). | ||
|
|
||
| Still, the concrete strategy must match the reality of the project IroCO2, and moreover it seems necessary to define what we mean with unit tests, integration tests or end-to-end tests, for each perimeter. | ||
|
|
||
| --- | ||
|
|
||
| ### 2.a. Infrastructure | ||
|
|
||
| #### UNIT testing | ||
|
|
||
| - Definition | ||
| > Testing infrastructure elements | ||
| - Strategy | ||
| > Testing of main elements using a combination of Terraform Test, variable validation, pre/post conditions as well as check blocks. | ||
|
|
||
| #### INTEGRATION testing | ||
|
|
||
| - Definition | ||
| > Testing infrastructure elements | ||
| - Strategy | ||
| > Testing of main elements using a combination of Terraform Test, variable validation, pre/post conditions as well as check blocks. | ||
|
|
||
| #### END-TO-END testing | ||
| - Definition | ||
| > Testing the whole application. | ||
| - Strategy | ||
| > A few tests of use cases, to be executed on an AWS sandbox account (e.g. Ippon's one) to run a `terraform apply` on the whole stack. Automation can be implemented for Continuous Delivery on non production environment, based on the fork of the github repo. | ||
|
|
||
| --- | ||
|
|
||
| ### 2.b. Back-end | ||
|
|
||
| #### UNIT testing | ||
| - Definition | ||
| > Testing the classes individually, mocking anything which could be outside its scope. | ||
| - Strategy | ||
| > Our ambition is to test every possible output of every method of every classe. **The coverage with unit tests should never decrease due to a new development**. When possible, it should rise with new unit tests for existing classes with missing test cases. | ||
| > Also, an ambition of IroCO2 team is to produce back-end code with TDD. | ||
|
|
||
| #### INTEGRATION testing | ||
| - Definition | ||
| > Testing the components in a more complex way. Some (but not all) external behavior are mocked. | ||
| - Strategy | ||
| > For a given use case, several scenarii should be tested, including errors handling. | ||
|
|
||
| #### END-TO-END testing | ||
| - Definition | ||
| > Testing back-end from endpoints to databases and cloud services without mocking anything. | ||
| - Strategy | ||
| > Each endpoint should be tested through two test cases at least : one testing expected behavior, and one testing error handling | ||
|
|
||
| --- | ||
|
|
||
| ### 2.c. Front-end | ||
|
|
||
| #### UNIT testing | ||
| - Definition | ||
| > Testing the components individually, mocking what does not depend on its scope. | ||
| - Strategy | ||
| > Our ambition is to test every UI component of the application. **The coverage with unit tests should never decrease due to a new development**. When possible, it should rise with new unit tests for existing components with missing test cases. | ||
| > Also, an ambition of IroCO2 team is to produce front-end code with TDD. | ||
|
|
||
| #### INTEGRATION testing | ||
| - Definition | ||
| > Testing the components in a more complex way, navigating between components for example | ||
| - Strategy | ||
| > At the moment of the first definition of test strategy, this floor of the Front-end test pyramid is not a priority for IroCO2 team. Anyone is free to init the subject at any moment, beginning with replacing content of this field with the strategy for integration tests. | ||
|
|
||
| #### END-TO-END testing | ||
| - Definition | ||
| > Testing the whole application. | ||
| - Strategy | ||
| > A few tests of use cases, to be executed on an AWS sandbox account (e.g. Ippon's one) to run a `terraform apply` on the whole stack. Automation can be implemented for Continuous Delivery on non production environment, based on the fork of the github repo. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.