Open
Conversation
Introduces a new utilities/test-runner.js script to dynamically build and execute Playwright test commands based on environment variables and tags. Updates package.json scripts to use the new runner and provide environment/tag-specific test commands. Extends env.ts to support 'prd' and 'int' as TARGET_ENV values. Updates CircleCI config to support test tags and improves Slack notifications for main and develop branches.
The commit-workflow in .circleci/config.yml now only runs the code-quality-check job, removing the test job and its dependency on eslint-check.
Introduces Playwright-based end-to-end test infrastructure, including page objects for patient, clinician, and account flows, endpoint schema validation utilities, and Xray JSON reporter integration. Updates CircleCI config to build TypeScript utilities and upload test results to Xray in JSON format. Adds supporting utilities, test fixtures, and documentation for Xray integration.
Refactored xray-json-reporter to improve code style, add a helper for status mapping, and use consistent array type annotations. Updated upload-to-xray.js to use node: imports and improved formatting. Added build/ and dist/ to .gitignore. Minor comment and formatting improvements in Playwright config.
Enhanced claude documentation to include new npm scripts for targeted test runs, expanded environment and tag filtering, and additional details on test tagging, reporting, and project structure. Updated environment and configuration file descriptions, added instructions for new test directories, and clarified integration with Xray and CircleCI.
Refactored clinic-helpers to remove duplicate findAndAccessAnyPatient, improved global stepCounter naming for network helpers, and added ESLint disables for process.exit and global require. Updated documentation and prompt files for clarity and formatting. Minor bugfixes and code style improvements in patient ProfilePage, patient-helpers, and test-runner utility. (Linting)
Eliminated steps for gathering and sending test evidence in the CircleCI pipeline. This streamlines the job by removing calls to browserstackEvidenceDownload.js and sendTestEvidenceToJira.js.
Updated CircleCI config to ensure Slack notifications are sent only from the first parallel node to avoid duplicate messages. Also adjusted Playwright test shard indexing and removed the JIRA test evidence step.
Add configurable Xray project key and modernize test-to-Xray upload flow. Key changes: - CircleCI: add xrayProjectKey pipeline parameter and propagate XRAY_PROJECT_KEY; simplify test run command and remove duplicated CI upload steps to rely on reporter. - Playwright config: remove legacy JUnit XML reporter and rely on JSON + custom Xray JSON reporter. - Xray reporter: major enhancements in utilities/xray-json-reporter.js — evidence classification (inline vs deferred vs skip), file-size threshold, video handling (only for failed tests), deferred-evidence plumbing (GraphQL upload stub), improved authentication/error messages, project key usage, and logging/stats. Reporter only attempts upload when credentials/execution key are configured. - Environment schema: default XRAY_PROJECT_KEY and XRAY_EVIDENCE_SIZE_THRESHOLD_KB added; optional JIRA fields added. - Utilities: add Xray types and JSON schema files; remove legacy xray-reporter and upload-to-xray scripts; include GraphQL evidence client hook. - Tests & fixtures: adopt validated tag usage in tests, fix step counter API (globalThis.stepCounter), rename/getNestedValue param, small bug fixes (loop increment, continue lint suppression), and other stability improvements in clinic/patient/network helpers. - Docs: update XRAY_INTEGRATION.md and CLAUDE.md to document new behavior, env vars, and usage examples. Also includes various build/test wiring updates and minor fixes to make the new reporter and CI flow work end-to-end.
Change default testEnvironment from 'qa1' to 'qa2'. Remove job-level static environment mappings and add a run step that appends exports to $BASH_ENV so project environment variables are preserved unless pipeline parameters are explicitly provided. Applied the update to both Playwright test jobs to ensure pipeline parameters override project env vars only when set.
Introduces a new utilities/test-runner.js script to dynamically build and execute Playwright test commands based on environment variables and tags. Updates package.json scripts to use the new runner and provide environment/tag-specific test commands. Extends env.ts to support 'prd' and 'int' as TARGET_ENV values. Updates CircleCI config to support test tags and improves Slack notifications for main and develop branches.
The commit-workflow in .circleci/config.yml now only runs the code-quality-check job, removing the test job and its dependency on eslint-check.
Introduces Playwright-based end-to-end test infrastructure, including page objects for patient, clinician, and account flows, endpoint schema validation utilities, and Xray JSON reporter integration. Updates CircleCI config to build TypeScript utilities and upload test results to Xray in JSON format. Adds supporting utilities, test fixtures, and documentation for Xray integration.
Refactored xray-json-reporter to improve code style, add a helper for status mapping, and use consistent array type annotations. Updated upload-to-xray.js to use node: imports and improved formatting. Added build/ and dist/ to .gitignore. Minor comment and formatting improvements in Playwright config.
Enhanced claude documentation to include new npm scripts for targeted test runs, expanded environment and tag filtering, and additional details on test tagging, reporting, and project structure. Updated environment and configuration file descriptions, added instructions for new test directories, and clarified integration with Xray and CircleCI.
Refactored clinic-helpers to remove duplicate findAndAccessAnyPatient, improved global stepCounter naming for network helpers, and added ESLint disables for process.exit and global require. Updated documentation and prompt files for clarity and formatting. Minor bugfixes and code style improvements in patient ProfilePage, patient-helpers, and test-runner utility. (Linting)
Eliminated steps for gathering and sending test evidence in the CircleCI pipeline. This streamlines the job by removing calls to browserstackEvidenceDownload.js and sendTestEvidenceToJira.js.
Updated CircleCI config to ensure Slack notifications are sent only from the first parallel node to avoid duplicate messages. Also adjusted Playwright test shard indexing and removed the JIRA test evidence step.
Add configurable Xray project key and modernize test-to-Xray upload flow. Key changes: - CircleCI: add xrayProjectKey pipeline parameter and propagate XRAY_PROJECT_KEY; simplify test run command and remove duplicated CI upload steps to rely on reporter. - Playwright config: remove legacy JUnit XML reporter and rely on JSON + custom Xray JSON reporter. - Xray reporter: major enhancements in utilities/xray-json-reporter.js — evidence classification (inline vs deferred vs skip), file-size threshold, video handling (only for failed tests), deferred-evidence plumbing (GraphQL upload stub), improved authentication/error messages, project key usage, and logging/stats. Reporter only attempts upload when credentials/execution key are configured. - Environment schema: default XRAY_PROJECT_KEY and XRAY_EVIDENCE_SIZE_THRESHOLD_KB added; optional JIRA fields added. - Utilities: add Xray types and JSON schema files; remove legacy xray-reporter and upload-to-xray scripts; include GraphQL evidence client hook. - Tests & fixtures: adopt validated tag usage in tests, fix step counter API (globalThis.stepCounter), rename/getNestedValue param, small bug fixes (loop increment, continue lint suppression), and other stability improvements in clinic/patient/network helpers. - Docs: update XRAY_INTEGRATION.md and CLAUDE.md to document new behavior, env vars, and usage examples. Also includes various build/test wiring updates and minor fixes to make the new reporter and CI flow work end-to-end.
Change default testEnvironment from 'qa1' to 'qa2'. Remove job-level static environment mappings and add a run step that appends exports to $BASH_ENV so project environment variables are preserved unless pipeline parameters are explicitly provided. Applied the update to both Playwright test jobs to ensure pipeline parameters override project env vars only when set.
ginnyyadav
force-pushed
the
Hook-up-CircleCI-and-JIRA
branch
from
f4ee158 to
b6bd30b
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
dist/tests/claimed/API-Profile/comprehensive-profile-access-test.spec.js
Fixed
Show fixed
Hide fixed
Delete the committed build/ and dist/ directories (compiled page-objects, endpoint schemas, tests, and utilities). These are generated artifacts that should be produced by the project's build step rather than stored in the repository to reduce repo size and merge noise; regenerate using the normal build command (e.g., `npm run build`).
Bump CircleCI orbs to newer releases: node -> 6.3.0, browser-tools -> 1.5.3, slack -> 4.15.0. Keeps CI configuration up-to-date with upstream orb updates.
Add buildGrepFromTags() to convert a TEST_TAGS env var into a Playwright grep RegExp and wire it into the config. Supports single tags, OR (comma-separated), AND (space-separated), optional @ prefix, and is case-insensitive. Returns undefined when no tags provided so default behavior is preserved.
Removed redundant blank lines and adjusted indentation in claimed-profile test. In login.spec.ts removed a duplicated TEST_TAGS import and consolidated/removed duplicate test.step wrappers and repeated assertions to streamline the login tests. Changes are purely cleanup/refactor to improve readability and reduce redundant test steps; no intended functional changes.
more CI debugging
Remove CircleCI debug run used for test discovery. Refactor buildGrepFromTags in playwright.config.ts by removing debug logging, simplifying control flow, and returning RegExp values directly. Re-enable grep via buildGrepFromTags(). Normalize project testMatch patterns (remove leading '**/' prefixes) and fix bs-chrome-personal testMatch from 'patient' to 'personal'.
Bump Playwright Docker image to v1.57.0 in CircleCI jobs and add optional TEST_TAGS support to the test run command (normalizes to lowercase, ensures leading '@', and passes as --grep). Preserve parallel sharding. Also convert Playwright project testMatch values from glob strings to regular expressions for more precise matching. Files changed: .circleci/config.yml, playwright.config.ts.
Add a short debug step in CircleCI to list discovered Playwright tests before applying --grep (helps diagnose test discovery/grep issues). Quote the GREP_ARG to preserve tags with special chars/spaces. In playwright.config.ts remove the buildGrepFromTags helper and the config-level grep (move filtering to the CLI), and replace project testMatch regexes with explicit glob patterns because apparently we need to go back to that.
Replace the GREP_ARG pattern with an explicit SHARD_ARG and invoke npm test inside the if/else so --shard is always passed and --grep is only added when TEST_TAGS is present. Applied the change to both Playwright test job blocks to ensure correct CircleCI sharding and avoid quoting/argument concatenation issues.
Remove duplicated debug steps that listed discovered Playwright tests from .circleci/config.yml. The 'List discovered tests (debug)' run steps (running `npx playwright test --list 2>&1 | head -20 || true`) were redundant and noisy; removing them cleans up the CI config without affecting Playwright install or test execution.
Replace hardcoded headless: false with headless: !process.env.CI in playwright.config.ts for the personal, claimed, and clinician projects. This makes tests run headless in CI while keeping the headed behavior locally for debugging.
Change headless flag to !!process.env.CI so Playwright runs headless when CI is set. This replaces the previous !process.env.CI logic that caused tests to run headed on CI; updated for personal/claimed/clinician storageState projects in playwright.config.ts for consistent CI behavior.
Lower the number of test retries in CI from 2 to 1 in playwright.config.ts. This reduces repeated test reruns in CI while keeping local retries at 0.
Replace attachment.name with path.basename(attachment.path) when constructing test evidence so the filename is reliably derived from the attachment path. This should allow xray to recognize the correct attachment form.
Replace legacy Nightwatch + BrowserStack README with a comprehensive Playwright-based testing guide. Adds Getting Started, environment setup, npm/playwright install steps, .env variables, test-running commands and tag filters, Jira-triggered test runs, integrated Xray reporter details, development guide (POM, fixtures, auth, import aliases), and CI/CD notes. Remove docs/XRAY_INTEGRATION.md since its Xray integration content has been consolidated into the updated README.
Rename capture variables in claimed-profile-edit-fullname.spec.ts to use a `retry` prefix and update the step name to indicate a retry; adjust log output to reference the renamed variables. Remove a duplicate `stepCounterObj` declaration in network-helpers.ts. These edits improve clarity of the retry validation flow and eliminate a redundant variable declaration.
If an existing testExecutionKey is provided (e.g., for sharded CI runs), omit the info block to avoid overwriting the existing execution's summary/description and counts. When no key is present, preserve the previous behavior of populating summary, description, startDate, and finishDate. Added a clarifying comment for the intent.
…ol-org/WebUITests into Hook-up-CircleCI-and-JIRA
Have automated tests upload to QAE project unless otherwise specified in env
…ol-org/WebUITests into Hook-up-CircleCI-and-JIRA
Enhance clinician page objects and network tooling to support adding/editing patients and stronger API capture/validation. Key changes: - page-objects/clinician/ClinicianDashboardPage: add MRN and Email inputs, editPatientDetails button, clickPatientCell/clickEditPatientDetails helpers, loosen patient cell matching. - page-objects/clinician/ClinicianNavigation: update Profile/ProfileEdit verify locators to target Edit Patient Details heading/button. - page-objects/patient/ProfilePage: normalize field locators (Full Name / Birthdate / Date of Birth), add saveButton and fillDateOfBirth, simplify saveProfile to click the button. - tests: add new clinician API-profile tests (Clinician-AddAndDeletePatient, Clinician-EditCustodialProfile) and personal AccountSettings test; rename many test files and spec titles to standardize naming; remove obsolete duplicate tests. - tests/fixtures/network-helpers.ts: add waitForCaptureMatching, reloadPage, validateResponseFields, and improve getNestedValue to handle array indices for robust response validation and timing-sensitive GET captures. - playwright.config.ts: point xray reporter to built JS output. These changes enable full add/edit/delete patient flows from clinician UI and improve deterministic API response capture/validation for tests. TLDR: Renamed Tests for xray test reporting reasons, fixed local executions so the report to xray correctly targets the right test/execution project. Test coverage corrected to match specifications of seagull merge ticket.
| const updatedName = `Custodial Patient Updated ${Math.floor( | ||
| Math.random() * 10000, | ||
| )}-${workspace}`; | ||
| const updateBirthYear = 1990 + Math.floor(Math.random() * 30); |
Check failure
Code scanning / CodeQL
Insecure randomness High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 24 days ago
In general, to fix insecure randomness in Node/TypeScript you should replace Math.random() with a cryptographically secure source such as crypto.randomInt or crypto.randomBytes, and derive your needed ranges/strings from that secure source. This ensures that any values that might be interpreted as identifiers, secrets, or security-relevant inputs are not predictably generated.
For this specific file, we need to stop using Math.random() for:
updatedNamesuffix (Math.floor(Math.random() * 10000))updateBirthYear(1990 + Math.floor(Math.random() * 30)) – the line flagged by CodeQLupdateMRN(digits fromMath.random() * 10)updateEmailsuffix (Math.floor(Math.random() * 10000))
The best way, without changing observable functionality beyond making randomness stronger, is:
- Import Node’s
cryptomodule at the top of this spec file. - Add a small helper to generate cryptographically secure integers within a range using
crypto.randomInt(min, maxExclusive). - Replace all
Math.random()usages with calls to that helper:- For
updatedName, usesecureRandomInt(0, 10000). - For
updateBirthYear, use1990 + secureRandomInt(0, 30). - For
updateMRN, generate digits viasecureRandomInt(0, 10)insideArray.from. - For
updateEmail, usesecureRandomInt(0, 10000).
- For
All changes will be made within tests/clinician/API-Profile/Clinician-EditCustodialProfile.spec.ts, and only in the displayed region: adding the crypto import, defining the helper function, and replacing the four Math.random() calls.
Suggested changeset
1
tests/clinician/API-Profile/Clinician-EditCustodialProfile.spec.ts
| @@ -4,21 +4,27 @@ | ||
| import { createNetworkHelper } from '../../fixtures/network-helpers'; | ||
| import { TEST_TAGS, createValidatedTags } from '../../fixtures/test-tags'; | ||
| import { ProfilePage } from '../../../page-objects/patient/ProfilePage'; | ||
| import { randomInt } from 'node:crypto'; | ||
|
|
||
| function secureRandomInt(min: number, maxExclusive: number): number { | ||
| return randomInt(min, maxExclusive); | ||
| } | ||
|
|
||
| ALL_WORKSPACE_KEYS.forEach((workspace: WorkspaceKey) => { | ||
| test.describe('Custodial patients are allowed access and modification of profile details', () => { | ||
| // Define the patient search term | ||
| const CUSTODIAL_PATIENT_SEARCH = 'Custodial Patient'; | ||
|
|
||
| const updatedName = `Custodial Patient Updated ${Math.floor( | ||
| Math.random() * 10000, | ||
| const updatedName = `Custodial Patient Updated ${secureRandomInt( | ||
| 0, | ||
| 10000, | ||
| )}-${workspace}`; | ||
| const updateBirthYear = 1990 + Math.floor(Math.random() * 30); | ||
| const updateBirthYear = 1990 + secureRandomInt(0, 30); | ||
| const updateBirthDate = `05/20/${updateBirthYear}`; | ||
| const updateMRN = Array.from({ length: 15 }, () => | ||
| Math.floor(Math.random() * 10).toString(), | ||
| secureRandomInt(0, 10).toString(), | ||
| ).join(''); | ||
| const updateEmail = `webuiautomation+updatedprofile${Math.floor(Math.random() * 10000)}@tidepool.org`; // must be lowercase to pass email validation | ||
| const updateEmail = `webuiautomation+updatedprofile${secureRandomInt(0, 10000)}@tidepool.org`; // must be lowercase to pass email validation | ||
|
|
||
| // API Test cases require this to capture network activity | ||
| let api: ReturnType<typeof createNetworkHelper>; |
Copilot is powered by AI and may make mistakes. Always verify output.
|
|
||
| test.describe('Personal Accounts allow access and modification of profile details', () => { | ||
| const updatedName = `Personal Patient Updated ${Math.floor(Math.random() * 10000)}`; | ||
| const updateBirthYear = 1990 + Math.floor(Math.random() * 30); |
Check failure
Code scanning / CodeQL
Insecure randomness High test
Copilot Autofix
AI 17 days ago
Copilot could not generate an autofix suggestion
Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.
Change the Playwright reporter entry to reference build/utilities/xray-json-reporter.ts instead of the .js file. This updates the reporter path in playwright.config.ts so the TypeScript reporter source is used.
Update Playwright reporter path from './build/utilities/xray-json-reporter.ts' to './utilities/xray-json-reporter.ts' so the test runner uses the source reporter location (no longer relying on a build output path). This ensures the custom JSON reporter is resolved correctly during test runs.
Use the CI environment flag to control Playwright's headless mode instead of hardcoding false. Updated three projects in playwright.config.ts (personal, claimed, clinician) to set headless: !!process.env.CI so tests run headed locally but run headless in CI environments.
Strip noisy debug console.logs across evidence handling and upload flow, and remove unused reporter fields (config, rootSuite). When an existing test execution key is provided, omit the info block to avoid overwriting execution metadata (useful for sharded CI runs). Also minor cleanups: use i += 1 in batch loop and ignore the config parameter in onBegin (no longer storing it).
Replace the early `continue` when `actualValue` is undefined with an `else` block that scopes the comparison logic. This refactor preserves existing validation behavior while keeping control flow explicit and improving readability by avoiding an early loop exit.
Make pipeline parameters always take precedence over project-level environment variables in .circleci/config.yml. Replace the shell fallback assignments (e.g. ${TARGET_ENV:-<< pipeline.parameters.testEnvironment >>}) with direct exports from pipeline.parameters and update the explanatory comment. Changes apply to both job blocks setting TARGET_ENV, TEST_EXECUTION_KEY, and TEST_TAGS to ensure pipeline-provided values are used unconditionally.
Add resilient typing to ClinicianDashboardPage.searchForPatient: retry up to 3 times, verify inputValue, clear between attempts, and throw a descriptive error on failure. Refactor clinician patient filter test to run across ALL_WORKSPACE_KEYS, use clinic-specific unique MRN/email generation, add tagging/validation, create patients only if missing, and restructure steps for clearer setup and verification.
Created an MD file to help aid test migration from other playwright tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After merging, you should be able to define a project and test execution key in you local env for exporting test results to JIRA/Xray. I have updated the JIRA automation to call the dev branch and the "trigger webui automated cases" rule can be executed on test executions.
notes :
If i rerun the tests and point to the same execution what happens?
title changes (date updates). test statuses are updated, results are loaded into same tests, no duplicates.
What if i rerun and change the name of a test?
new test is created. do not change name of tests if we don't want that to happen
what if test fails?
Actual, test fails, step fails, actual result describes test failure. two screenshots attahced. global evidence with screenshot of ui, and another of the error in context.
This is pretty MVP, but can be built upon now that we have a working connection to JIRA/XRAY.