๐Ÿ” Black Trigram (ํ‘๊ด˜) - End-to-End Test Plan

May 5, 2026 ยท View on GitHub

๐Ÿ” ISMS Alignment: This E2E test plan implements Hack23 Secure Development Policy Section 4.3.3 - End-to-End Testing Requirements.

๐Ÿ“‹ Executive Summary

This End-to-End (E2E) Test Plan provides comprehensive testing coverage for the Black Trigram Korean martial arts combat simulator, ensuring all critical user journeys and game mechanics function correctly across different browsers, screen sizes, and performance scenarios.

ISMS Compliance Requirements

Per Hack23 AB's Secure Development Policy, this project maintains:

๐ŸŽฏ Requirement๐Ÿ“Š Implementationโœ… Status๐Ÿ“‹ ISMS Reference
Critical Path CoverageComplete game flow testingโœ… ImplementedSection 4.3.3.1
Browser TestingChrome, Firefox, Edgeโœ… ValidatedSection 4.3.3.2
Automated ExecutionEvery PR via Cypressโœ… ActiveSection 4.3.3.3
Public ReportingMochawesome reportsโœ… PublishedSection 4.3.3.4
Performance AssertionsLoad time & FPS monitoringโœ… TrackedSection 4.3.3.5

Evidence Links:

See Also:


๐ŸŽฏ Testing Objectives

  • ๐Ÿฅ‹ Validate Critical Game Workflows: Ensure all Korean martial arts gameplay paths work correctly
  • ๐ŸŽฎ Verify Combat Mechanics: Test trigram system, vital point targeting, and combat calculations
  • ๐Ÿ”„ Confirm State Management: Validate game state persistence and transitions
  • ๐ŸŒ Cross-Browser Compatibility: Test across Chrome, Firefox, and Edge browsers
  • ๐Ÿ“ฑ Responsive Design Validation: Ensure proper functionality on desktop, tablet, and mobile viewports
  • โšก Performance Verification: Confirm 60fps rendering and acceptable load times
  • ๐ŸŽจ PixiJS Integration: Validate WebGL rendering and canvas interactions
  • ๐ŸŽต Audio System: Test Korean traditional music and combat sound effects

๐Ÿงฉ Test Categories

1. ๐Ÿ›๏ธ Core Game Journey Workflows

Test Coverage:

  • Complete navigation cycle: Intro โ†’ Training โ†’ Combat โ†’ Intro
  • All 8 trigram stances in training mode
  • Combat mechanics with movement and techniques
  • State management and transitions
  • Performance under load

Critical User Journeys:

JourneyPriorityTest FileStatus
Intro โ†’ Training โ†’ Introโœ… Highgame-journey.cy.tsImplemented
Intro โ†’ Combat โ†’ Introโœ… Highgame-journey.cy.tsImplemented
Training - All Stancesโœ… Hightraining-flow.cy.tsImplemented
Combat - Movementโœ… Highcombat-flow.cy.tsImplemented
Combat - Trigram Techniquesโœ… Highcombat-system-integration.cy.tsImplemented
Combat - Vital Pointsโœ… Highpixi-korean-martial-arts.cy.tsImplemented

2. โš”๏ธ Combat System Integration Workflows

Combat Mechanics Tested:

  • Movement system (WASD + Arrow keys)
  • Trigram stance selection (1-8 keys)
  • Technique execution (Space bar)
  • Vital point targeting (Mouse clicks)
  • Damage calculations (VitalPointSystem)
  • Health/Ki resource management
  • Combat feedback (visual + audio)
  • AI opponent behavior

3. ๐ŸŽจ PixiJS Rendering and Performance

Performance Assertions:

MetricTargetTest Method
Initial Load Time< 3 secondsMeasure time from visit() to canvas ready
Screen Transition< 500msTime between ESC and new screen render
Input Response< 100msKey press to visual feedback
Combat Action< 200msTechnique execution to effect render
Frame Rate60fps ยฑ5%Monitor FPS during intense combat
Memory Usage< 512MBCheck heap size after 5 minutes

4. ๐Ÿ“ฑ Responsive Design and Browser Compatibility

Browser Testing Matrix:

BrowserVersionPlatformWebGL SupportThree.js CompatibilityPriorityStatus
ChromeLatest StableWindows/Mac/Linuxโœ… Excellent (SwiftShader)โœ… Excellentโœ… HighTested
FirefoxLatest StableWindows/Mac/Linuxโœ… Excellentโœ… Excellentโœ… HighTested
EdgeLatest StableWindowsโœ… Excellent (Chromium)โœ… Excellentโœ… HighTested
SafariLatest StableMacโœ… Goodโœ… Goodโš ๏ธ MediumPlanned

Chrome Configuration for Three.js (See CHROME_CYPRESS_THREEJS_CONFIG.md for details):

  • SwiftShader software rendering for WebGL in headless mode
  • Optimized Chrome flags for Three.js performance
  • 60fps frame rate capping for consistency
  • 4GB Node.js heap for memory-intensive Three.js scenes

Viewport Testing Matrix:

Device TypeWidth x HeightOrientationPriority
Desktop HD1280 x 720Landscapeโœ… High
Desktop FHD1920 x 1080Landscapeโœ… High
Tablet768 x 1024Portraitโš ๏ธ Medium
Mobile375 x 667Portrait๐Ÿ“ฑ Low

5. ๐ŸŽต Audio System Integration

Audio Features Tested:

  • Traditional Korean music in Training Mode
  • Cyberpunk combat music in Combat Mode
  • Impact sounds for technique execution
  • Bone crack audio for critical hits
  • Audio synchronization with visuals

๐Ÿงช Test Implementation with Cypress

Test Structure

cypress/
โ”œโ”€โ”€ e2e/
โ”‚   โ”œโ”€โ”€ game-journey.cy.ts           # Complete game flow tests
โ”‚   โ”œโ”€โ”€ core-features.cy.ts          # Essential gameplay features
โ”‚   โ”œโ”€โ”€ combat-system-integration.cy.ts  # Combat mechanics
โ”‚   โ”œโ”€โ”€ training-system-integration.cy.ts # Training mode
โ”‚   โ”œโ”€โ”€ combat-flow.cy.ts            # Combat-specific workflows
โ”‚   โ”œโ”€โ”€ training-flow.cy.ts          # Training-specific workflows
โ”‚   โ”œโ”€โ”€ combat-screen-layout.cy.ts   # Combat UI validation
โ”‚   โ”œโ”€โ”€ combat-mode.cy.ts            # Combat mode entry/exit
โ”‚   โ”œโ”€โ”€ training-mode.cy.ts          # Training mode entry/exit
โ”‚   โ”œโ”€โ”€ pixi-korean-martial-arts.cy.ts # PixiJS integration (legacy)
โ”‚   โ”œโ”€โ”€ performance-threejs.cy.ts    # Three.js FPS and performance
โ”‚   โ”œโ”€โ”€ webgl-verification.cy.ts     # WebGL context and rendering verification โœจ NEW
โ”‚   โ”œโ”€โ”€ app.cy.ts                    # Application smoke tests
โ”‚   โ””โ”€โ”€ game-flow.cy.ts              # Navigation flows
โ”œโ”€โ”€ fixtures/
โ”‚   โ””โ”€โ”€ test-data.json               # Test data fixtures
โ”œโ”€โ”€ support/
โ”‚   โ”œโ”€โ”€ e2e.ts                       # E2E support file
โ”‚   โ”œโ”€โ”€ component.ts                 # Component support
โ”‚   โ”œโ”€โ”€ commands.ts                  # Custom Cypress commands
โ”‚   โ”œโ”€โ”€ performance.ts               # Performance monitoring utilities
โ”‚   โ”œโ”€โ”€ test-isolation.ts            # Test isolation utilities
โ”‚   โ””โ”€โ”€ resource-monitoring.ts       # Resource leak detection

WebGL Verification Tests โœจ NEW

Test File: webgl-verification.cy.ts

Comprehensive WebGL and Three.js verification tests to ensure proper rendering in headless CI:

Coverage:

  1. WebGL Context Verification

    • Verify WebGL/WebGL2 context creation
    • Check renderer, version, and capabilities
    • Validate texture size and viewport limits
    • Test WebGL extensions support
  2. Frame Rate Performance

    • Measure average FPS (target: 30-60fps)
    • Monitor frame timing consistency
    • Detect frame drops or stuttering
    • Validate software rendering performance
  3. Rendering Quality

    • Verify active canvas rendering (pixel changes)
    • Check for blank/black canvas issues
    • Validate visual content is present
    • Test canvas state changes
  4. Memory and Resource Management

    • Detect WebGL context leaks
    • Monitor GPU memory usage
    • Track memory growth during gameplay
    • Verify resource cleanup
  5. Mode-Specific Testing

    • Combat mode WebGL performance
    • Training mode WebGL performance
    • Screen transition stability
    • Context preservation across scenes

Running WebGL Tests:

# Full WebGL verification
npx cypress run --spec "cypress/e2e/webgl-verification.cy.ts"

# With memory optimization
NODE_OPTIONS="--max-old-space-size=4096" \
  npx cypress run --spec "cypress/e2e/webgl-verification.cy.ts"

For detailed configuration and troubleshooting, see CHROME_CYPRESS_THREEJS_CONFIG.md.

Custom Cypress Commands

Available Commands:

  • cy.visitWithWebGLMock() - Visit page with WebGL mocking for headless
  • cy.waitForCanvasReady() - Wait for PixiJS canvas initialization
  • cy.gameActions() - Execute game inputs with proper timing
  • cy.annotate() - Add test annotations for reporting
  • cy.enterTrainingMode() - Navigate to training mode
  • cy.enterCombatMode() - Navigate to combat mode
  • cy.returnToIntro() - Return to intro screen
  • cy.practiceStance() - Practice specific trigram stance

๐Ÿ“Š Test Data Management

Game State Test Data

Test fixtures are defined in cypress/fixtures/test-data.json:

Player Archetypes:

  • ๋ฌด์‚ฌ (Musa) - Traditional Warrior
  • ์•”์‚ด์ž (Amsalja) - Shadow Assassin
  • ํ•ด์ปค (Hacker) - Cyber Warrior
  • ์ •๋ณด์š”์› (Jeongbo Yowon) - Intelligence Operative
  • ์กฐ์งํญ๋ ฅ๋ฐฐ (Jojik Pokryeokbae) - Organized Crime

Trigram Stances:

  • ๊ฑด (Geon) - Heaven
  • ํƒœ (Tae) - Lake
  • ๋ฆฌ (Li) - Fire
  • ์ง„ (Jin) - Thunder
  • ์† (Son) - Wind
  • ๊ฐ (Gam) - Water
  • ๊ฐ„ (Gan) - Mountain
  • ๊ณค (Gon) - Earth

๐Ÿ“Š Test Reporting with Mochawesome

Report Generation Process

1. Test Execution:

npm run test:e2e        # Local execution with Chrome
npm run test:e2e:ci     # CI execution (headless)

2. Report Merging:

npm run test:e2ereportmerge    # Merge JSON reports
npm run test:e2ereporthtmlall  # Generate HTML report

3. Report Location (Local):

  • Individual JSON: docs/cypress/mochawesome/*.json
  • Merged JSON: docs/cypress/mochawesome-all.json
  • HTML Report: docs/cypress/mochawesome/index.html
  • JUnit XML: docs/cypress/junit/results-*.xml

4. CI Artifacts: CI uploads artifacts from the following locations (configured in cypress.config.ts to write to docs/cypress/ but uploaded from standard Cypress paths):

  • Videos: cypress/videos/ (failures only) โ†’ uploaded to docs/cypress/videos/
  • Screenshots: cypress/screenshots/ (failures only) โ†’ uploaded to docs/cypress/screenshots/
  • Results: cypress/results/ โ†’ uploaded to docs/cypress/results/
  • Reports: Available as downloadable artifacts from GitHub Actions workflow runs

Mochawesome Report Contents

  • โœ… Test pass/fail summary with percentages
  • ๐Ÿ“Š Interactive charts showing test distribution
  • ๐Ÿ–ผ๏ธ Embedded screenshots for failed tests
  • ๐ŸŽฌ Video playback for failed test runs
  • โฑ๏ธ Execution time for each test suite
  • ๐Ÿ” Detailed error messages and stack traces

๐Ÿ”„ CI/CD Integration

GitHub Actions Workflow

E2E Test Job:

e2e-tests:
  needs: [prepare, build-validation]
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-node@v6
      with:
        node-version-file: '.nvmrc'
    - name: Install dependencies
      run: npm ci
    - name: Start app and run Cypress tests
      run: |
        xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" npm run test:e2e
    - name: Upload Cypress results
      if: always()
      uses: actions/upload-artifact@v5
      with:
        name: cypress-results
        path: |
          cypress/videos
          cypress/screenshots
          cypress/results

Trigger Conditions:

  • โœ… Every push to main branch
  • โœ… Every pull request to main branch
  • โš™๏ธ Manual workflow dispatch (optional)

Environment:

  • OS: Ubuntu Latest (Linux)
  • Node: from .nvmrc
  • Display: Xvfb (virtual framebuffer for headless)
  • Resolution: 1280x720x24 bit color depth
  • Browser: Chrome (headless)

Dependencies:

  • Korean font support: fonts-noto-cjk, fonts-noto-cjk-extra
  • System libraries: libgtk-3-0, libgbm-dev, libnss3, etc.
  • google-chrome-stable

๐Ÿ“‹ Test Coverage and Success Criteria

Critical User Journeys Coverage

JourneyCoverageTestsStatus
Intro โ†’ Training โ†’ Intro100%game-journey.cy.tsโœ…
Intro โ†’ Combat โ†’ Intro100%game-journey.cy.tsโœ…
Training Mode - All Stances100%training-flow.cy.tsโœ…
Combat - Movement100%combat-flow.cy.tsโœ…
Combat - Trigram Stances100%combat-system-integration.cy.tsโœ…
Combat - Vital Points100%pixi-korean-martial-arts.cy.tsโœ…
Responsive Design100%core-features.cy.tsโœ…
Cross-Browser100%All testsโœ…

Test Execution KPIs

MetricTargetCurrent (Enhanced)PreviousStatus
Test Coverage>90% of user journeys~95%~95%โœ… Excellent
Pass Rate>95%100% (10/10)~98%โœ… Excellent
Execution Time10-12 minutes~10-12 min (optimized)~20 minโœ… Target Met
Browser Compatibility100% Chrome/Firefox/Edge100%100%โœ… Complete
Performance Compliance<3s load, <500ms interactionMeasuredMeasuredโœ… Met
Flaky Test Rate0%0% (enhanced monitoring)<1%โœ… Zero Flakes
Retry Countโ‰ค11 (runMode), 0 (openMode)2 (runMode), 1 (openMode)โœ… Improved
Resource Leaks0 detectedMonitored (10-30MB growth)Not monitoredโœ… Tracked
Test IsolationCompleteโœ… ImplementedPartialโœ… Complete

Enhanced Reliability Features (Nov 2025):

  • โœ… Test Isolation: Complete state reset between tests (localStorage, sessionStorage, game state)
  • โœ… Resource Monitoring: Tracks audio elements, canvas elements, memory usage, event listeners
  • โœ… Global Cleanup Hooks: Automatic cleanup in beforeEach/afterEach for all tests
  • โœ… Flakiness Detection: CI runs tests 3x on PRs to detect inconsistent behavior
  • โœ… Reliability Reporting: Automated HTML report generation with flaky test detection
  • โœ… Reduced Retries: From 2โ†’1 (runMode), 1โ†’0 (openMode) to catch flakes faster

Note: Execution time optimized from ~20 minutes to 10-12 minutes through configuration tuning, wait time reduction, and performance monitoring. Test reliability improvements (Nov 2025) added comprehensive isolation and monitoring to achieve 0% flaky rate. See E2E_OPTIMIZATION_RESULTS.md for details.

Feature Coverage Matrix

FeatureUnit TestsE2E TestsStatus
Trigram Systemโœ… TrigramCalculator.test.tsโœ… combat-system-integration.cy.tsComplete
Vital Pointsโœ… VitalPointSystem.test.tsโœ… pixi-korean-martial-arts.cy.tsComplete
Combat Mechanicsโœ… GameIntegration.test.tsโœ… combat-flow.cy.tsComplete
Player Utilsโœ… playerUtils.test.tsโœ… game-journey.cy.tsComplete
UI Componentsโœ… CombatStatsPanel.test.tsxโœ… combat-screen-layout.cy.tsComplete
State Managementโœ… Integration testsโœ… game-journey.cy.tsComplete

๐ŸŽฎ Game Screen Test Scenarios

IntroScreen (์ธํŠธ๋กœ ํ™”๋ฉด)

Test Coverage:

  • โœ… Korean logo rendering (ํ‘๊ด˜)
  • โœ… Menu navigation (Training/Combat buttons)
  • โœ… Bilingual text display (Korean/English)
  • โœ… Background visual effects
  • โœ… Audio system initialization
  • โœ… Keyboard navigation (1, 2 keys)

Test Files: app.cy.ts, core-features.cy.ts, game-journey.cy.ts

TrainingScreen (ํ›ˆ๋ จ ๋ชจ๋“œ)

Test Coverage:

  • โœ… All 8 trigram stance practice
  • โœ… Korean technique names display
  • โœ… Visual technique feedback
  • โœ… Training progression tracking
  • โœ… Return to intro (ESC)

Test Files: training-mode.cy.ts, training-flow.cy.ts, training-system-integration.cy.ts

CombatScreen (์ „ํˆฌ ํ™”๋ฉด)

Test Coverage:

  • โœ… Combat HUD rendering
  • โœ… Health bar display (player + opponent)
  • โœ… Ki resource management
  • โœ… Movement controls (WASD, arrows)
  • โœ… Trigram stance selection (1-8)
  • โœ… Technique execution (Space)
  • โœ… Vital point targeting (Mouse)
  • โœ… Combat log updates
  • โœ… Damage calculations
  • โœ… Visual effects (blood, impacts)
  • โœ… Audio feedback (bone impacts)

Test Files: combat-mode.cy.ts, combat-flow.cy.ts, combat-system-integration.cy.ts, combat-screen-layout.cy.ts, pixi-korean-martial-arts.cy.ts

๐Ÿ”„ Test Maintenance Strategy

Test Review Schedule

  • ๐Ÿ“… Weekly: Review failing tests and identify flaky tests
  • ๐Ÿ”„ Bi-weekly: Update test scenarios based on new features
  • ๐Ÿ“Š Monthly: Comprehensive test coverage analysis
  • ๐ŸŽฏ Quarterly: Test framework and strategy evaluation

Flaky Test Handling

  • ๐Ÿ” Retries: Configured in cypress.config.ts
    • runMode: 2 - CI environment retries
    • openMode: 1 - Local development retries
  • โฑ๏ธ Timeouts: Optimized for game loading
    • defaultCommandTimeout: 6000ms
    • pageLoadTimeout: 15000ms

Test Optimization

  • ๐Ÿš€ Memory Management: experimentalMemoryManagement: true
  • ๐Ÿ’พ Test Retention: numTestsKeptInMemory: 5
  • ๐ŸŽฌ Video Optimization: videoUploadOnPasses: false (failures only)
  • ๐Ÿ“ธ Screenshots: Only on failure
  • ๐Ÿงน Cleanup: trashAssetsBeforeRuns: true

๐Ÿ” Security Testing Integration

Security Test Coverage

  • ๐Ÿ”’ XSS Prevention: Input validation tests
  • ๐Ÿ›ก๏ธ CSP Compliance: Content Security Policy validation
  • ๐Ÿ” Dependency Scanning: Automated via npm audit
  • ๐Ÿ“Š OWASP ZAP: Periodic security scans
  • ๐ŸŽฏ CodeQL: Security vulnerability scanning

Security Test Files:

  • ZAP scan: .github/workflows/zap-scan.yml
  • CodeQL: .github/workflows/codeql.yml
  • Scorecard: .github/workflows/scorecards.yml

๐Ÿ“– Test Execution Guide

Local Development

# Install Cypress
npm run cypress:install
npm run cypress:verify

# Run all E2E tests
npm run test:e2e

# Open Cypress Test Runner (interactive)
npx cypress open

# Run specific test file
npx cypress run --spec "cypress/e2e/game-journey.cy.ts"

# Run tests in specific browser
npx cypress run --browser firefox

CI Environment

# Tests run automatically on:
# - Push to main branch
# - Pull requests to main branch

# View test results:
# https://github.com/Hack23/blacktrigram/actions

Troubleshooting

Issue: Tests fail in headless mode

xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" npm run test:e2e

Issue: Korean fonts not rendering

sudo apt-get install fonts-noto-cjk fonts-noto-cjk-extra

๐ŸŽฏ Compliance Mapping

ISO 27001 Alignment

ControlRequirementImplementation
A.8.9Configuration ManagementVersion-controlled test plans
A.14.2Testing in DevelopmentAutomated E2E testing
A.14.3Test Data ProtectionSanitized test data only

NIST CSF Alignment

FunctionCategoryImplementation
PR.IP-1Baseline ConfigurationCypress config management
PR.IP-2SDLC IntegrationE2E tests in CI/CD
DE.CM-4Malicious Code DetectionDependency scanning

CIS Controls Alignment

ControlDescriptionImplementation
2.1Software InventoryPackage.json tracking
16.9Automated TestingE2E test automation
16.10Test Environment SegregationIsolated test runs

๐Ÿ“š Additional Resources

Documentation

โšก Performance Optimization

Execution Time Improvement

The E2E test suite has been optimized to reduce execution time from ~20 minutes to the target of 10-12 minutes while maintaining full test coverage and reliability.

Key Optimizations:

  1. Configuration Tuning (2-3 min savings)

    • Video compression optimized for faster encoding
    • Memory management improved
    • Timeout settings reduced appropriately
  2. Code Optimization (3-4 min savings)

    • Fixed wait times replaced with assertions
    • Command delays reduced
    • Test transitions optimized
  3. Performance Monitoring (observability)

    • Per-test timing metrics
    • Slowest test identification
    • CI duration reporting

Test Suite Options:

  • Smoke Tests (npm run test:e2e:smoke): 5-6 minutes - Critical path validation
  • Full Suite (npm run test:e2e): 10-12 minutes - Comprehensive testing
  • Reliability Check (npm run test:reliability): <1 minute - Analyze flaky tests

For detailed optimization analysis and metrics, see E2E_OPTIMIZATION_RESULTS.md.


๐Ÿ›ก๏ธ Test Reliability & Stability (Nov 2025)

Test Isolation Infrastructure

Global Cleanup Hooks (cypress/support/e2e.ts):

beforeEach(() => {
  // Clear browser storage
  cy.clearLocalStorage();
  cy.clearCookies();
  
  // Reset viewport
  cy.viewport(1280, 720);
  
  // Clear game state
  // Start resource monitoring
  cy.startResourceMonitoring();
});

afterEach(() => {
  // Detect resource leaks
  cy.detectResourceLeaks();
  
  // Log test metrics
  // Force cleanup resources
});

Test Isolation Utilities (cypress/support/test-isolation.ts):

  • cy.isolateTest() - Reset to clean state before test
  • cy.cleanupTest() - Clean up after test completion
  • cy.captureState() - Capture current state for restoration
  • cy.restoreState() - Restore previously captured state
  • State management: localStorage, sessionStorage, game state, timers, event listeners

Resource Monitoring (cypress/support/resource-monitoring.ts):

  • cy.startResourceMonitoring() - Begin tracking resources
  • cy.detectResourceLeaks() - Check for leaks after test
  • cy.logResourceReport() - Log detailed resource usage
  • cy.forceResourceCleanup() - Force cleanup all resources
  • Tracks: audio elements, canvas elements, memory usage, event listeners

Flakiness Detection (CI)

Pull Request Testing (.github/workflows/test-and-report.yml):

  • Runs smoke tests 3 times on every PR
  • Detects inconsistent behavior (0 < failures < 3 = flaky)
  • Reports consistent failures (all 3 runs fail)
  • Uploads results as artifacts for analysis

Reliability Reporting (scripts/generate-reliability-report.cjs):

npm run test:reliability

Generates:

  • HTML report with metrics and flaky test details
  • Console summary with pass rates and failures
  • Identifies flaky tests with pass rate percentages
  • Lists consistently failing tests
  • Calculates average test duration

Configuration Changes

Cypress Retries (cypress.config.ts):

  • runMode: Reduced from 2 to 1 (better flaky detection)
  • openMode: Reduced from 1 to 0 (immediate feedback)
  • Encourages fixing tests rather than masking flakiness

Timeout Settings (optimized for reliability):

  • defaultCommandTimeout: 5000ms (fast failure detection)
  • pageLoadTimeout: 12000ms (sufficient for app load)
  • requestTimeout: 6000ms (reasonable for API calls)
  • responseTimeout: 6000ms (matches request timeout)

Best Practices for Test Authors

Using Test Isolation:

describe('My Feature Tests', () => {
  beforeEach(() => {
    cy.isolateTest(); // Optional: explicit isolation
    cy.visit('/');
  });
  
  afterEach(() => {
    cy.cleanupTest(); // Optional: explicit cleanup
  });
  
  it('should work reliably', () => {
    // Test implementation
    // Global hooks handle most cleanup automatically
  });
});

Resource Monitoring:

it('should not leak resources', () => {
  cy.startResourceMonitoring(); // Optional: already in global hook
  
  // Perform test actions
  cy.get('[data-testid="button"]').click();
  
  cy.detectResourceLeaks(); // Optional: already in global hook
  // Logs warnings if leaks detected
});

Checking for Flakiness Locally:

# Run tests multiple times
for i in {1..5}; do npm run test:e2e:smoke; done

# Generate reliability report
npm run test:reliability

Metrics & Achievements

Current Status (as of Nov 2025):

  • โœ… 0% Flaky Test Rate (target achieved)
  • โœ… 100% Pass Rate (10/10 smoke tests)
  • โœ… Resource Monitoring Active (detects 10-30MB memory growth)
  • โœ… Complete Test Isolation (state reset between tests)
  • โœ… Reduced Retries (1 or 0, down from 2)
  • โœ… CI Flakiness Detection (3x runs on PRs)

Before Improvements:

  • Flaky Test Rate: <1% (acceptable but not ideal)
  • Retries: 2 (could mask flaky tests)
  • Resource Leaks: Not monitored
  • Test Isolation: Partial

After Improvements:

  • Flaky Test Rate: 0% (zero tolerance)
  • Retries: 1 (runMode), 0 (openMode)
  • Resource Leaks: Monitored and logged
  • Test Isolation: Complete with global hooks

๐Ÿ“‹ Document Metadata:
โœ… Approved by: Development Team
๐Ÿ“ค Distribution: Development Team, QA Team, Security Team
๐Ÿ”„ Review Cycle: Monthly
โฐ Last Updated: 2025-11-17
๐Ÿ“ Version: 1.2.0 (Test Reliability Enhanced)
๐Ÿ‘ค Maintained by: Test Specialist Agent


๐Ÿฅ‹ ํ‘๊ด˜์˜ ํ’ˆ์งˆ์„ ์ง€ํ‚ค๋ผ - Protect the Quality of Black Trigram