Skip to content

chore: Add test comments across multiple components and files - #110

Open
umarraza086 wants to merge 2 commits into
stagingfrom
final-intent-testing
Open

chore: Add test comments across multiple components and files#110
umarraza086 wants to merge 2 commits into
stagingfrom
final-intent-testing

Conversation

@umarraza086

Copy link
Copy Markdown
Collaborator

Pull Request: [Feature/Fix Description]

📝 Description

Closes: [JIRA-XXXX or GitHub Issue #XXX]

🎯 Type of Change

  • New feature
  • Bug fix
  • Documentation update
  • Performance improvement
  • Refactoring
  • Dependency update

🔍 Changes Made

  • Change 1
  • Change 2
  • Change 3

🧪 E2E Testing Checklist

Required before merging to staging:

  • Feature tested locally: npm run dev
  • No console errors (DevTools → Console)
  • Signup test: created test user testcoach+staging+[random]@example.com
  • Baseline assessment: answered questions, received persona (A/B/C/D)
  • Module viewing: clicked Module 1, video played
  • Content gate: video 90% watched before quiz unlocks
  • Quiz: answered questions, score calculated correctly
  • Module completion: passed at 80%+ score
  • Dashboard: shows completed modules and next module
  • Endline: accessible only after all modules passed
  • Certificate: generated and PDF downloads correctly
  • Supabase: new user appears in auth.users table
  • Supabase: new profiles/assessments created in correct tables
  • Mobile responsive: tested at 375px viewport
  • No sensitive data in code: no API keys, passwords, emails hardcoded
  • Code follows project conventions and standards

📸 Screenshots (if UI change)

🔄 Deployment Checklist

  • Tested on staging environment (if auto-deployed)
  • No breaking changes to database schema (or migrations included)
  • Environment variables documented (if new ones added)
  • Performance: page load time acceptable

🚨 Breaking Changes

  • This PR introduces breaking changes (describe below)
  • Requires data migration
  • Requires configuration change

If checked, describe the impact:

📋 Checklist

  • Code follows project standards (DEVELOPMENT_STANDARDS.md)
  • No hardcoded secrets or sensitive data
  • Tests added/updated (if applicable)
  • Documentation updated (if needed)
  • Commit messages are clear and descriptive
  • No merge conflicts with staging branch

🔗 Related Issues

  • Related to: [Issue/PR]
  • Depends on: [Issue/PR]
  • Blocked by: [Issue/PR]

👀 Reviewers

⚠️ Notes for Reviewers


DO NOT MERGE until:

  1. ✅ Code review approved
  2. ✅ All E2E tests pass (checkbox above)
  3. ✅ CI/CD (GitHub Actions) passes

Remember: This goes to staging first, then production after staging QA passes.

@github-actions

github-actions Bot commented May 19, 2026

Copy link
Copy Markdown

🤖 GitHub Actions — Intent Validation Report

🔍 Intent Validation

PR #110 · chore: Add test comments across multiple components and files · Features login, signup, onboarding · 2026-05-19 12:14 UTC · logs

@chunk = deep validation · no @chunk = all scenarios · human QA still required

IDD Validation Report

Feature: User Login
File: login/login.feature
Mode: chunk
Date: 2026-05-19
Verdict: ❌ FAIL


Scope Boundary

File Status
src/pages/Login.tsx Loaded
src/contexts/AuthContext.tsx Loaded

Unresolvable paths (out of scope):

  • @/integrations/supabase/client → outside scope
  • @/components/ui/* → outside scope
  • sonner → outside scope
  • react-router-dom → outside scope

Scenario Results

# Scenario Status Severity
1 Successful login with valid credentials ✅ COVERED
2 Login page shows all required fields and controls ✅ COVERED
3 Loading state is shown while login is processing ✅ COVERED
4 Navigate to Sign Up page from Login page ✅ COVERED
5 Successful password reset request ✅ COVERED
6 Return to login form from Forgot Password mode ✅ COVERED
7 Submit login form with both fields empty ✅ COVERED
8 Submit login form with Email field missing ✅ COVERED
9 Submit login form with Password field missing ✅ COVERED
10 Login with correct email but wrong password ⚠️ PARTIAL Medium
11 Login with an email that does not exist ⚠️ PARTIAL Medium
12 Fields containing only whitespace are treated as empty ✅ COVERED
13 Server returns a network error during login ❌ MISSING Medium
14 Submit Forgot Password form without entering an email ✅ COVERED
15 Forgot Password API returns an error for unknown email ❌ MISSING Medium

Findings

Scenario 10 — Login with correct email but wrong password ⚠️ PARTIAL

Evidence (covered):

  • file: src/pages/Login.tsx | function: handleSubmit | line: 41if (error) { toast.error(error.message) } — an error toast is shown on failure
  • file: src/pages/Login.tsx | function: handleSubmit | line: 43navigate("/dashboard") is called only inside the else branch — redirection is correctly blocked on failure

Gap:
The specific error message "Invalid login credentials" is not produced by any in-scope code. handleSubmit passes error.message directly from the Supabase client (@/integrations/supabase/client is outside the declared scope boundary). The exact string shown to the user cannot be verified from within scope.

Actual behavior: Login failure correctly prevents redirection and displays a toast error. Whether the toast reads "Invalid login credentials" depends entirely on the Supabase client's response — unresolvable within scope.


Scenario 11 — Login with an email that does not exist ⚠️ PARTIAL

Evidence (covered):

  • file: src/pages/Login.tsx | function: handleSubmit | line: 38–44 — same execution path as Scenario 10

Gap: Identical to Scenario 10. The specific message "Invalid login credentials" originates from the out-of-scope Supabase client and cannot be verified from within the declared scope boundary.

Actual behavior: Redirection prevention is covered. The error message text is passed through verbatim from the Supabase client response — content is outside scope.


Scenario 13 — Server returns a network error during login ❌ MISSING

Evidence: None

Dry-run trace:

  • Entry: Login.tsx handleSubmit()signIn(email, password) (AuthContext.tsx signIn()) → supabase.auth.signInWithPassword({ email, password }) (out of scope)
  • On network failure the Supabase client returns { error } with its own error.message
  • Login.tsx line 41: if (error) { toast.error(error.message) } — the raw Supabase message is shown
  • No in-scope code inspects the error type, checks for offline or network conditions, or maps any error value to the string "Unable to connect. Please check your internet connection."
  • Last resolvable operation: toast.error(error.message) — the specific required string is never produced by any in-scope function

Actual behavior: A toast error is displayed with whatever string Supabase's client returns for a network failure. No custom error classification or message mapping exists in scope — the specific message "Unable to connect. Please check your internet connection." is never generated by in-scope code.


Scenario 15 — Forgot Password API returns an error for unknown email ❌ MISSING

Evidence: None for specific message

Dry-run trace:

  • Entry: Login.tsx handleForgotPassword()email.trim() non-empty, validation passes
  • supabase.auth.resetPasswordForEmail(email, { redirectTo: ... }) called — Supabase client is outside scope
  • Login.tsx: if (error) { toast.error(error.message) } — raw Supabase message passed through
  • No in-scope code maps any error response to the string "Email address not found"
  • Secondary concern: resetPasswordForEmail commonly returns no error for unknown addresses as a deliberate security measure — if that behavior applies, the error branch never executes and setResetSent(true) runs instead, producing the inbox-confirmation success view — the opposite of the expected outcome. This behavior is out of scope to confirm, but the expected error path may never trigger regardless

Actual behavior: If Supabase does return an error, raw error.message is shown — the string "Email address not found" is never produced by in-scope code. If Supabase silently succeeds (security design), the success message is shown instead, contradicting the Gherkin expectation entirely.


Summary

Metric Count
Scenarios validated 15
✅ COVERED 11
⚠️ PARTIAL 2
❌ MISSING 2
❌ VIOLATION 0

Verdict: ❌ FAIL — 2 MISSING scenarios (13, 15). The login form's core validation flows — empty fields, whitespace trimming, loading state, navigation, and forgot-password UI transitions — are well-implemented. The two gaps are: (1) no custom network error detection or message mapping for Scenario 13, and (2) no in-scope code that produces the specific error message "Email address not found" for Scenario 15, with a secondary risk that Supabase may silently succeed for unknown emails, making the entire error path unreachable.

IDD Validation Report: Feature Sign Up

Summary

Verdict: FAIL
Scenarios Validated: 15
Status Breakdown: ✅ 5 | ⚠️ 0 | ❌ 10


Detailed Findings

# Scenario Status Severity Evidence Issue
1 Successful registration with all fields filled ✅ COVERED src/pages/Signup.tsx:handleSubmit (line 23–38), AuthContext.signUp (line 95–161) All required fields validated, account creation called, success toast triggered, navigation to /onboarding executed.
2 Successful registration without optional Full Name ⚠️ PARTIAL High src/pages/Signup.tsx:handleSubmit (line 23), AuthContext.signUp (line 95–102) Full Name is marked required on input (line 71), but validation at line 25 requires fullName.trim() to be truthy. Scenario expects success with empty Full Name; actual behavior: validation fails with "Please fill in all required fields" error. Full Name cannot be submitted empty.
3 All required form fields are visible on the page ✅ COVERED src/pages/Signup.tsx (lines 69–95) Full Name label (line 69), Email label (line 76), Phone Number label (line 83), Password label (line 90), and "Create Account" button (line 103) all rendered. All marked with * for required except Full Name visual marking is absent from label.
4 Loading state is shown while the request is processing ✅ COVERED src/pages/Signup.tsx (lines 21, 36, 103–105) Button text changes to "Creating account..." when loading === true (line 104), button disabled state set via disabled={loading} (line 103), setLoading(false) called after signUp completes (line 37).
5 Navigate to Sign In page from Sign Up page ✅ COVERED src/pages/Signup.tsx (lines 107–110) "Sign in" link rendered with to="/login" navigation. Link text visible and functional.
6 Submit form with all fields empty ❌ MISSING High src/pages/Signup.tsx:handleSubmit (line 25) Validation checks `!fullName.trim()
7 Submit form with Email field missing ❌ MISSING High src/pages/Signup.tsx:handleSubmit (line 25) Validation requires email.trim() to be truthy. Email is empty, condition TRUE, error shown. But Full Name is ALSO required in the validation (line 25). If Full Name is not filled, this scenario cannot be tested in isolation—it will fail on Full Name first. Gherkin assumes Full Name is optional.
8 Submit form with Phone Number field missing ❌ MISSING High src/pages/Signup.tsx:handleSubmit (line 25) Validation requires phone.trim() to be truthy. But Full Name is required in validation (line 25). If Full Name is not filled, validation fails before reaching Phone check. Gherkin implicitly assumes Full Name is optional.
9 Submit form with Password field missing ❌ MISSING High src/pages/Signup.tsx:handleSubmit (line 25) Validation requires password.trim() to be truthy. But Full Name is required in validation (line 25). If Full Name is not filled, validation fails before reaching Password check. Gherkin implicitly assumes Full Name is optional.
10 Password is shorter than 8 characters ✅ COVERED src/pages/Signup.tsx:handleSubmit (line 29–31) Password length check: password.length < 8 triggers error "Password must be at least 8 characters" (line 30), account not created. Matches expectation.
11 Password is exactly 7 characters — boundary below minimum ✅ COVERED src/pages/Signup.tsx:handleSubmit (line 29–31) Password length = 7: 7 < 8 is TRUE, error shown, account not created. Matches expectation.
12 Register with an email that already exists ❌ VIOLATION Critical src/pages/Signup.tsx (lines 36–38), AuthContext.signUp (lines 98–102, 141–154) Scenario expects error "User already registered" and remain on Sign Up page. Actual behavior: Supabase auth.signUp() at line 98 is called without checking if email exists first. If email exists, Supabase returns error with message "User already registered" or similar (handled at line 39). Error is displayed via toast (line 40). No violation. Navigation only happens on error === null (line 41). Account not created, user stays on page. Matches expectation. ✅ (Corrected: this IS covered.)
13 Server returns an unexpected error during registration ✅ COVERED src/pages/Signup.tsx (lines 36–40), AuthContext.signUp (line 98–161) If signUp() returns an error object, if (error) at line 39 is TRUE, error message displayed via toast (line 40), no navigation (line 41 not reached). User stays on Sign Up page. Matches expectation.
14 Fields containing only whitespace are treated as empty ✅ COVERED src/pages/Signup.tsx:handleSubmit (line 25) Validation uses .trim() on all fields. Whitespace-only input after .trim() = empty string = falsy. Validation fails, error shown. Account not created. Matches expectation.
15 Full Name field marked as required in validation ❌ VIOLATION High src/pages/Signup.tsx (line 25, line 69) Feature Gherkin Scenario 2 states: "Successful registration without optional Full Name" — implying Full Name is optional. Code reality: Full Name validation at line 25 is mandatory (!fullName.trim()). The HTML input at line 71 has required attribute. This violates the feature contract. Scenarios 2, 6–9 expect Full Name to be optional but code enforces it as required. All scenarios omitting Full Name will fail validation, contradicting Gherkin intent.

Root Cause Analysis

Critical Issue: Full Name Field Requirement Mismatch

  • Gherkin Intent: Full Name is optional (Scenario 2 explicitly tests registration without it)
  • Code Behavior: Full Name is mandatory in validation (line 25) and HTML (required attribute at line 71)
  • Impact: Scenarios 2, 6, 7, 8, 9 fail because they either omit Full Name or implicitly assume it's optional while testing other fields
  • Verdict Trigger: ❌ VIOLATION in Scenario 15 + ⚠️ PARTIAL in Scenario 2 + ❌ MISSING in Scenarios 6–9 = FAIL

Secondary Issues

  • Scenarios 6–9: Cannot be tested in isolation because Full Name validation failure masks the specific field-missing validation being tested. Gherkin does not provide Full Name values in these scenarios.

Verdict Justification

FAIL — ≥1 ❌ MISSING or ❌ VIOLATION found.

Detected violations:

  • Scenario 15: Full Name required in code but optional in Gherkin contract
  • Scenario 2: Full Name passed as optional in Gherkin; code enforces mandatory validation
  • Scenarios 6–9: Missing Full Name causes validation to fail before testing the intended field-missing check

The feature cannot pass until Full Name requirement is either:

  1. Made optional in code to match Gherkin Scenario 2, OR
  2. Made mandatory in Gherkin (all scenarios must provide Full Name)

IDD Validation Report

Feature: onboarding/onboarding.feature
Mode: chunk
Scenarios Validated: 9

Scenario Findings

Scenario Status Evidence Severity Impact
Complete onboarding with all fields filled file: src/pages/Onboarding.tsx | function: handleSubmit | line: 77 Safe
Complete onboarding without optional Teachers coached field file: src/pages/Onboarding.tsx | function: handleSubmit | line: 88-90 Safe
All required form fields are visible on the page file: src/pages/Onboarding.tsx | line: 243 — School Name marked as Optional, not Required School field labeled "(Optional)" but scenario expects it marked as required High
Region dropdown shows all available options file: src/pages/Onboarding.tsx | line: 67-71, 228-231 Safe
Loading state is shown while the request is processing file: src/pages/Onboarding.tsx | line: 79, 318 Safe
Auto-redirect to dashboard if user already has school assigned file: src/pages/Onboarding.tsx | function: useEffect | line: 57-62 Safe
Submit form without selecting Region file: src/pages/Onboarding.tsx | function: handleSubmit | line: 80-83 Safe
Submit form without entering School Name file: src/pages/Onboarding.tsx | line: 88-90 — school field is optional, no validation required School field is optional; no validation error expected or implemented Safe
School Name field containing only whitespace is treated as empty file: src/pages/Onboarding.tsx | line: 88 — school value used directly without trimming or whitespace validation Whitespace-only input stored as-is without validation, form proceeds without error Medium

Verdict

FAIL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant