Visual and Accessibility Testing Strategy — When, Where and How Much to Test

Visual and accessibility testing are powerful but can become a maintenance burden if applied indiscriminately. A visual snapshot of every page at every viewport on every browser creates thousands of baselines to maintain. An axe-core scan of every page catches hundreds of violations that cannot all be fixed in one sprint. A practical strategy applies these tests where they provide the highest value with the lowest maintenance cost.

Visual and Accessibility Testing Strategy #

The strategy answers three questions: what to test, how often to test it, and how to integrate into the development workflow.

// ── WHAT to test visually (highest ROI targets) ──

const VISUAL_TARGETS = {
  'Always test visually': [
    'Design system components (buttons, forms, cards, alerts)',
    'Landing pages and marketing pages (brand consistency)',
    'Checkout and payment pages (trust and conversion)',
    'Login and registration forms (first impression)',
    'Responsive breakpoints for critical pages (mobile, tablet, desktop)',
  ],
  'Selectively test visually': [
    'Admin dashboards (functional correctness matters more than pixel perfection)',
    'Settings pages (low visual complexity, rarely redesigned)',
    'Internal tools (smaller user base, lower visual bar)',
  ],
  'Skip visual testing': [
    'Pages with highly dynamic content (real-time feeds, dashboards with live data)',
    'Pages under active redesign (baselines change every sprint)',
    'Third-party embedded content (iframes you do not control)',
  ],
};


// ── WHAT to test for accessibility (prioritised) ──

const A11Y_TARGETS = {
  'Priority 1 — Every page (automated + manual)': [
    'cy.checkA11y() on every page — catches missing labels, contrast, headings',
    'Keyboard: Tab through all interactive elements on key pages',
    'Focus management: every modal, drawer, and popup',
  ],
  'Priority 2 — Key workflows (manual testing)': [
    'Screen reader: complete checkout flow narrated by VoiceOver/NVDA',
    'Screen reader: form error announcement and recovery',
    'Zoom: page usable at 200% zoom without horizontal scrolling',
  ],
  'Priority 3 — Comprehensive audit (quarterly)': [
    'Full WCAG 2.1 Level AA audit by accessibility specialist',
    'User testing with people who use assistive technologies',
    'Automated + manual combined report with remediation plan',
  ],
};


// ── Integration into CI pipeline ──

const CI_INTEGRATION = {
  'Every PR (automated, < 1 minute)': [
    'cy.checkA11y() — automated axe-core scan on key pages',
    'Visual snapshots — component-level snapshots for design system',
    'Block PR merge on: critical/serious a11y violations',
  ],
  'Nightly (automated, 10-15 minutes)': [
    'Full visual regression — all pages at 3 viewports',
    'Full a11y scan — all pages with all axe rules enabled',
    'Report: email or Slack summary of new violations',
  ],
  'Pre-release (manual, 2-4 hours)': [
    'Screen reader walkthrough of critical user journeys',
    'Keyboard-only navigation of entire application',
    'Responsive spot-check on real devices',
    'Visual review of pages with recent design changes',
  ],
};


// ── Metrics to track ──

const METRICS = [
    {
        metric: 'A11y violation count',
        target: 'Decrease or maintain each sprint; never increase',
        how: 'Track cy.checkA11y() violation count over time',
    },
    {
        metric: 'Critical/serious violations',
        target: 'Zero — these block users with disabilities',
        how: 'CI blocks PRs with critical/serious; minor tracked in backlog',
    },
    {
        metric: 'Visual baseline update rate',
        target: '< 5 baseline updates per sprint (indicates stable UI)',
        how: 'Count baseline image changes in Git history per sprint',
    },
    {
        metric: 'False positive rate',
        target: '< 10% of visual failures are false positives',
        how: 'Track how many visual failures are dismissed vs fixed',
    },
];


// ── Best practices summary ──

const BEST_PRACTICES = [
    'Add cy.checkA11y() to every existing functional test — two lines for free coverage',
    'Snapshot components, not full pages, for lower maintenance visual testing',
    'Generate visual baselines in CI (Docker), not on developer machines',
    'Block PRs on critical/serious a11y violations; track minor in backlog',
    'Use fixtures to stabilise dynamic content before visual snapshots',
    'Run full visual regression nightly, not on every PR (unless design system)',
    'Supplement automated a11y with quarterly manual audits',
    'Track violation count as a team metric in retrospectives',
    'Document suppressed axe rules with ticket numbers and planned fix dates',
    'Test keyboard navigation for every modal, dropdown, and interactive widget',
];

console.log('Visual & A11y Testing Strategy:');
console.log('\nCI Integration:');
Object.entries(CI_INTEGRATION).forEach(([tier, items]) => {
  console.log(`\n  ${tier}:`);
  items.forEach(i => console.log(`    - ${i}`));
});

Common Mistakes #

Mistake 1 — Treating accessibility testing as a one-time audit #

❌ Wrong: Hiring an accessibility consultant once, fixing the reported issues, and never testing again — new violations accumulate with every code change.

✅ Correct: Automated a11y checks in CI on every PR (prevent new violations) + quarterly manual audits (catch what automation misses). Accessibility is continuous, not a one-time event.

Mistake 2 — Visual testing every page at every viewport on every commit #

❌ Wrong: 50 pages x 3 viewports x every commit = 150 snapshot comparisons per push — massive baseline maintenance.

✅ Correct: Component-level snapshots for the design system (every PR), full-page snapshots for critical pages (nightly), and responsive checks for landing pages (pre-release). Match testing frequency to change frequency and business impact.