A framework that runs only on a developer’s laptop is a tool. A framework wired into a CI/CD pipeline — triggered on every commit, running against every pull request, blocking merges on failure — is a quality gate. CI/CD integration is what transforms your Selenium tests from a manual verification step into an automated safety net that catches regressions before they reach main. This lesson covers the practical steps to run your framework in GitHub Actions and Jenkins with Docker, artefact storage, and quality gate enforcement.
CI/CD Pipeline Design for Selenium Tests
A well-designed CI pipeline for Selenium tests has five stages: checkout code, set up the environment, start the Grid, run tests, and publish results.
# ── GitHub Actions workflow for Selenium tests ──
GITHUB_ACTIONS_YAML = """
# .github/workflows/selenium-tests.yml
name: Selenium Tests
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
services:
# Selenium Grid as a service container
selenium-hub:
image: selenium/standalone-chrome:4.18.1
ports:
- 4444:4444
options: --shm-size=2g
steps:
# Stage 1: Checkout code
- uses: actions/checkout@v4
# Stage 2: Set up Python environment
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: pip install -r requirements.txt
# Stage 3: Wait for Grid to be ready
- name: Wait for Selenium Grid
run: |
for i in $(seq 1 30); do
curl -s http://localhost:4444/status | grep -q '"ready":true' && break
echo "Waiting for Grid... ($i/30)"
sleep 2
done
# Stage 4: Run tests
- name: Run Selenium tests
env:
BASE_URL: https://www.saucedemo.com
GRID_URL: http://localhost:4444
HEADLESS: true
run: |
pytest tests/ \\
-n 4 \\
--reruns 2 \\
--html=reports/report.html \\
--self-contained-html \\
--junitxml=reports/results.xml \\
-v
# Stage 5: Upload artefacts (always, even on failure)
- name: Upload test reports
uses: actions/upload-artifact@v4
if: always()
with:
name: test-reports
path: reports/
retention-days: 14
"""
# ── Quality gate enforcement ──
QUALITY_GATES = [
{
"gate": "All tests pass",
"enforcement": "Pipeline fails if any test fails after retries",
"config": "pytest exits with non-zero code on failure; CI treats as failure",
},
{
"gate": "Minimum pass rate threshold",
"enforcement": "Pipeline fails if pass rate < 95%",
"config": "Parse JUnit XML results; calculate pass rate; fail if below threshold",
},
{
"gate": "No new flaky tests",
"enforcement": "Alert if retry count exceeds the flake budget",
"config": "Post-test step checks flake log; fails if budget exceeded",
},
{
"gate": "PR blocking",
"enforcement": "Pull requests cannot merge until Selenium tests pass",
"config": "GitHub branch protection rule: require 'Selenium Tests' check to pass",
},
]
# ── CI/CD best practices ──
CI_BEST_PRACTICES = [
"Use Docker for reproducible environments (Grid + test runner)",
"Run tests in parallel (pytest -n 4) to keep pipeline under 15 min",
"Always upload reports/screenshots as artefacts — even on success",
"Set retention policy for artefacts (14-30 days) to manage storage",
"Use JUnit XML output for CI dashboard integration",
"Tag test runs with commit SHA for traceability",
"Separate smoke tests (every commit) from full suite (nightly)",
"Set a pipeline timeout (30 min max) to catch hanging tests",
]
print("GitHub Actions Pipeline Stages:")
print("=" * 55)
print(" 1. Checkout code")
print(" 2. Set up Python + dependencies")
print(" 3. Wait for Selenium Grid readiness")
print(" 4. Run pytest with parallel execution + retries")
print(" 5. Upload reports and screenshots as artefacts")
print("\n\nQuality Gates:")
for gate in QUALITY_GATES:
print(f"\n {gate['gate']}")
print(f" Enforcement: {gate['enforcement']}")
print("\n\nCI/CD Best Practices:")
for bp in CI_BEST_PRACTICES:
print(f" * {bp}")
if: always() condition on the artefact upload step is critical. Without it, artefacts are only uploaded when the pipeline succeeds — but the reports and screenshots you need most are from failed runs. if: always() ensures that test reports, screenshots, and logs are uploaded regardless of the test outcome. This is the single most important CI configuration for Selenium test debugging.TIMEOUT=15 in the pipeline versus TIMEOUT=5 locally. Hardcoded timeouts are the second most common cause of "works locally, fails in CI" (after missing waits).Common Mistakes
Mistake 1 — Not waiting for Grid readiness before running tests
❌ Wrong: Starting tests immediately after the Grid container starts — the Grid may not be ready to accept sessions yet, causing "connection refused" errors.
✅ Correct: Polling the Grid's /status endpoint until it returns "ready": true before running any tests. The retry loop with a 30-attempt, 2-second delay handles startup times up to 60 seconds.
Mistake 2 — Not uploading artefacts on failure
❌ Wrong: Reports and screenshots are generated but not uploaded — they are lost when the CI container is destroyed after the pipeline finishes.
✅ Correct: Using if: always() on the artefact upload step so that reports are preserved regardless of test outcome. Set retention (14-30 days) to manage storage costs.