test: playwright cucumber setup

.github/workflows/main.yml

@@ -34,21 +34,27 @@ jobs:
       - name: Install playwright browsers
         run: yarn playwright install chromium
 
-      - name: Run Playwright Chrome Tests
-        run: yarn test:playwright-local:ci:chromium
+      - name: Run Cucumber Giftaid Sanity Tests
+        run: yarn test:ci:chromium
         continue-on-error: false
         env:
           NODE_ENV: development
           REACT_APP_ENDPOINT_URL: ${{ secrets.REACT_APP_ENDPOINT_URL }}
           FORCE_COLOR: 1
 
-      # Test run video was always captured, so this action uses "always()" condition
-      - name: Upload videos
+      - name: List Playwright artifacts
+        if: failure()
+        run: find playwright-local/test-results -type f || true
+
+      # Upload Playwright video and trace artifacts for failed scenarios only
+      - name: playwright-chromium-artifacts
+        if: failure()
         uses: actions/upload-artifact@v4
-        if: always()
         with:
-          name: playwright-report
-          path: playwright-report/
+          name: playwright-failure-artifacts-chromium
+          path: |
+            playwright-local/test-results/failed-videos/**/*.webm
+            playwright-local/test-results/traces/**/*.zip
           retention-days: 3
 
   playwright-mobile:
@@ -85,21 +91,27 @@ jobs:
       - name: Install playwright browsers (WebKit)
         run: yarn playwright install webkit
 
-      - name: Run Mobile WebKit Tests
-        run: yarn test:playwright-local:ci:mobile
+      - name: Run Mobile Cucumber Tests
+        run: yarn test:ci:mobile
         continue-on-error: false
         env:
           NODE_ENV: development
           REACT_APP_ENDPOINT_URL: ${{ secrets.REACT_APP_ENDPOINT_URL }}
           FORCE_COLOR: 1
 
-      # Test run video was always captured, so this action uses "always()" condition
-      - name: Upload videos
+      - name: List Playwright artifacts
+        if: failure()
+        run: find playwright-local/test-results -type f || true
+
+      # Upload Playwright video and trace artifacts for failed scenarios only
+      - name: playwright-mobile-artifacts
+        if: failure()
         uses: actions/upload-artifact@v4
-        if: always()
         with:
-          name: playwright-report
-          path: playwright-report/
+          name: playwright-failure-artifacts-mobile
+          path: |
+            playwright-local/test-results/failed-videos/**/*.webm
+            playwright-local/test-results/traces/**/*.zip
           retention-days: 3
 
   lint:

README.md

@@ -67,36 +67,106 @@ The domains for giftaid are as follows
 
 ## Testing
 
-### Playwright-Local Tests
+### Playwright-Local Cucumber Tests
 
-To run PR Playwright Tests locally (after running `yarn playwright install` if you haven't previously), you need to first export `REACT_APP_ENDPOINT_URL=https://giftaid-sandbox.sls.comicrelief.com/` in your terminal for the form to get submitted and then run the script `test:playwright-local:local` found in package.json; this script starts the http://localhost:3000 server in the background, config for this is found in `playwright-local/playwright-local.config.js` file and runs the tests in headless mode. 
+Local end-to-end tests use:
+- Cucumber
+- Playwright
+- Chromium and Mobile Safari
+- A locally running React app on `http://localhost:3000`
 
-To view a test in a _headed_ mode locally, use the according command; `test:playwright-local--h`.
+### Start the local server
 
-To run a single test, add `only` annotation
+Before running the tests locally, start the React app:
 
-eg: test.only('Valid giftaid submission', async ({ page }) => {
-    });
-
-### Staging Playwright Tests
+```bash
+yarn start
+```
+This starts the application on: http://localhost:3000
+
+### Running Local Tests
+
+Run all local tests:
+
+```bash
+yarn test:local
+```
+
+Run Chromium tests:
+
+```bash
+yarn test:local:chromium
+```
+
+Run Mobile Safari tests:
+
+```bash
+yarn test:local:mobile
+```
+
+### Run Tests in Headed Mode
+
+Run Chromium in headed mode:
+
+```bash
+yarn test:local:chromium:headed
+```
+
+Run Mobile Safari in headed mode:
+
+```bash
+yarn test:local:mobile:headed
+```
+
+### Run Specific Tests by Tag
+
+Example:
+
+```bash
+yarn test:local --tags "@valid-giftaid-submission"
+```
+
+Run address validation tests:
+
+```bash
+yarn test:local --tags "@address-validation"
+```
+
+### Local Test Structure
+
+```text
+playwright-local/
+├── tests/features
+├── tests/step-definitions
+├── tests/support
+└── tests/utils
+```
+
+- Feature files contain readable Gherkin scenarios using the `.feature` extension
+- Step definitions contain the Playwright automation implementation
+- Hooks create a fresh browser context and page for every scenario
+- Shared selectors are stored in `utils/locators.js`
+- Shared reusable helper methods are stored in `utils/commands.js`
+
+### Staging BrowserStack Tests
+
+Staging tests run against BrowserStack and require the following environment variables:
 
-In order to run Playwright end-to-end tests locally you need to change directory to playwright folder `cd playwright` and export the following environment variables to your terminal:`BASE_URL, BROWSERSTACK_ACCESS_KEY, BROWSERSTACK_USERNAME`
-Browserstack credentials can be found in https://github.com/comicrelief/serverless-giftaid/blob/master/concourse/private.yml
 ```bash
 export BASE_URL='https://giftaid-staging.comicrelief.com/'
 export BROWSERSTACK_USERNAME='<INSERT_USERNAME>'
 export BROWSERSTACK_ACCESS_KEY='<INSERT_ACCESS_KEY>'
 ```
-### Running tests 
 
-To run spa-feature-test or nightly-test-sanity, check the commands in playwright/package.json eg: `yarn test:sanity` 
+BrowserStack credentials can be found in:
+
+```text
+serverless-giftaid/concourse/private.yml
+```
 
-To run a single test, add `only` annotation
+### Running Staging Tests
 
-eg: test.only('submit form with valid inputs', async ({ page }) => {
-    });
-
-Check serverless-giftaid in order to get the right values and then you can run the test executing:
+Run staging tests:
 
 ```bash
 yarn test:sanity

package.json

@@ -37,15 +37,13 @@
     "start": "node scripts/start.js",
     "build": "node scripts/build.js",
     "lint": "eslint --color src",
-    "test:playwright-local:local": "playwright test --config=./playwright-local/playwright-local.config.js",
-    "test:playwright-local:local:chromium": "playwright test --config=./playwright-local/playwright-local.config.js --project=chromium",
-    "test:playwright-local:local:chromium--h": "playwright test --config=./playwright-local/playwright-local.config.js --project=chromium --headed",
-    "test:playwright-local:local:mobile": "playwright test --config=./playwright-local/playwright-local.config.js --project=mobile-safari",
-    "test:playwright-local:local:mobile--h": "playwright test --config=./playwright-local/playwright-local.config.js --project=mobile-safari --headed",
-    "test:playwright-local:ci": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:playwright-local",
-    "test:playwright-local:ci:chromium": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:playwright-local:local:chromium",
-    "test:playwright-local:ci:mobile": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:playwright-local:local:mobile",
-    "test:playwright-local:local:chrome:mobile": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:playwright-local:local",
+    "test:local": "cd playwright-local && BASE_URL=http://localhost:3000 cucumber-js --config config/cucumber.js",
+    "test:local:chromium": "cd playwright-local && BROWSER=chromium BASE_URL=http://localhost:3000 cucumber-js --config config/cucumber.js",
+    "test:local:chromium:headed": "cd playwright-local && BROWSER=chromium HEADED=true BASE_URL=http://localhost:3000 cucumber-js --config config/cucumber.js",
+    "test:local:mobile": "cd playwright-local && BROWSER=mobile-safari BASE_URL=http://localhost:3000 cucumber-js --config config/cucumber.js",
+    "test:local:mobile:headed": "cd playwright-local && BROWSER=mobile-safari HEADED=true BASE_URL=http://localhost:3000 cucumber-js --config config/cucumber.js",
+    "test:ci:chromium": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:local:chromium",
+    "test:ci:mobile": "export NODE_ENV=development; start-server-and-test start http://localhost:3000 test:local:mobile",
     "snyk-protect": "snyk-protect",
     "prepublish": "yarn snyk-protect"
   },
@@ -59,6 +57,7 @@
     "@babel/runtime": "^7.11.2",
     "@comicrelief/data-models": "^1.15.4",
     "@comicrelief/test-utils": "^1.5.13",
+    "@cucumber/cucumber": "8.9.1",
     "@playwright/test": "1.38.1",
     "babel-eslint": "^10.1.0",
     "babel-loader": "^8.1.0",

playwright-local/config/cucumber.js

@@ -0,0 +1,16 @@
+/** @type {Partial<import('@cucumber/cucumber').IConfiguration>} */
+module.exports = {
+  default: {
+    paths: ['tests/features/**/*.feature'],
+    require: [
+      'tests/support/**/*.js',
+      'tests/step-definitions/**/*.js',
+    ],
+    // Use 'pretty' locally for readable step-by-step output.
+    // On CI keep the output minimal to avoid noisy logs.
+    format: ['progress', 'summary'],
+    retry: 2, // Retry failed scenarios twice
+    parallel: 2, // Run scenarios in parallel workers
+    publishQuiet: true, // Hide the default Cucumber report publishing link (we check on Github actions)
+  },
+};

playwright-local/tests/features/submit/addressValidation.feature

@@ -0,0 +1,43 @@
+@address-validation
+Feature: Address validation
+
+  Background:
+    Given I am on the local Giftaid page
+    And I select the local Giftaid option
+    And I enter the local supporter details
+
+  Scenario: Empty postcode should show an error message
+    When I clear the local postcode field
+    And I submit the local Giftaid form
+    Then I should see the local postcode error message "Please enter your postcode"
+
+  Scenario Outline: Invalid postcodes should show error messages
+    When I enter the local postcode "<postcode>"
+    Then I should see the local postcode error message "<message>"
+
+    Examples:
+      | postcode     | message                                                                                               |
+      | 12SE17TP     | Please enter a valid UK postcode, using a space. For non-UK addresses, please use manual entry below. |
+      | comic relief | Please enter a valid UK postcode, using a space. For non-UK addresses, please use manual entry below. |
+      | cro 7tp      | Please enter a valid UK postcode, using a space. For non-UK addresses, please use manual entry below. |
+
+  Scenario: Entering a postcode without selecting an address should show an error message
+    When I enter the local postcode "E1 8QS"
+    And I search for the local postcode
+    Then I should see the local address dropdown
+    When I submit the local Giftaid form
+    Then I should see the local address select error message "Please select your address"
+
+  Scenario: Clicking the manual address link should show the address fields
+    When I enter the local postcode "E1 8QS"
+    Then I should see the local manual address link
+    When I click the local manual address link
+    Then I should see the local manual address fields
+
+  Scenario: Invalid address fields should show error messages
+    When I enter the local postcode "E1 8QS"
+    And I click the local manual address link
+    And I enter the local invalid address line 1
+    Then I should see the local address line 1 error message
+    When I enter the local invalid town
+    Then I should see the local town error message