Test Coverage, Mocking and CI Integration

A test suite without coverage metrics is incomplete — you do not know which parts of your code are never exercised. pytest-cov measures which lines, branches, and functions are executed during tests and generates reports showing gaps. Beyond coverage, a professional test setup mocks all external services (S3, email, Redis) so tests are fast and deterministic, organises tests with markers for selective execution, and runs automatically in CI on every commit. This final lesson ties together testing best practices for a production FastAPI application.

pytest-cov Configuration #

pip install pytest-cov

# Run tests with coverage report
pytest --cov=app --cov-report=term-missing --cov-report=html

# Coverage output:
# Name                          Stmts   Miss  Cover
# -----------------------------------------------
# app/main.py                       42      2    95%
# app/routers/posts.py              87      5    94%
# app/services/post_service.py      63     12    81%
# -----------------------------------------------
# TOTAL                            482     22    95%

# Fail if coverage drops below threshold
pytest --cov=app --cov-fail-under=90

# .coveragerc — coverage configuration
[run]
source = app
omit   =
    app/migrations/*
    app/models/__init__.py
    */conftest.py

[report]
exclude_lines =
    pragma: no cover
    if TYPE_CHECKING:
    raise NotImplementedError
    if __name__ == .__main__.:

[html]
directory = htmlcov

Mocking External Services #

# tests/conftest.py — mock all external services
import pytest
from unittest.mock import patch, AsyncMock, MagicMock

@pytest.fixture(autouse=True)
def mock_s3():
    """
    autouse=True: applied to EVERY test automatically.
    Prevents any test from accidentally uploading to real S3.
    """
    with patch("app.storage.s3.put_object") as mock:
        mock.return_value = {"ETag": '"fake-etag"'}
        yield mock

@pytest.fixture(autouse=True)
def mock_email():
    """Suppress all email sending in tests."""
    with patch("app.email.config.fast_mail.send_message",
               new_callable=AsyncMock) as mock:
        yield mock

@pytest.fixture(autouse=True)
def mock_redis():
    """Mock Redis for ARQ task enqueueing."""
    with patch("app.main.create_arq_pool") as mock_pool:
        pool = AsyncMock()
        pool.enqueue_job = AsyncMock(return_value=None)
        mock_pool.return_value = pool
        yield pool

# ── Using mocks in specific tests ─────────────────────────────────────────────
def test_register_sends_welcome_email(client, mock_email):
    response = client.post("/auth/register", json={
        "email": "new@example.com", "name": "New", "password": "Password1"
    })
    assert response.status_code == 201

    # Verify the mock was called (email was "sent")
    mock_email.assert_called_once()
    call_args = mock_email.call_args[0][0]   # MessageSchema argument
    assert "new@example.com" in str(call_args.recipients)

def test_avatar_upload_stores_to_s3(client, mock_s3):
    import io
    # Create a minimal valid JPEG bytes for testing
    fake_image = b"\xff\xd8\xff\xe0" + b"\x00" * 100   # JPEG magic bytes + padding
    response = client.post(
        "/users/me/avatar",
        files={"avatar": ("test.jpg", io.BytesIO(fake_image), "image/jpeg")},
    )
    mock_s3.assert_called_once()
    key = mock_s3.call_args.kwargs.get("Key", "")
    assert "avatars/" in key

pytest Marks and CI Configuration #

# tests/markers.py — custom markers (register in pytest.ini)
# [pytest]
# markers =
#     unit: Fast unit tests (no database or network)
#     integration: Integration tests (use test database)
#     slow: Slow tests (>1 second)
#     auth: Tests for authentication endpoints

# Apply markers to tests:
@pytest.mark.unit
def test_password_hashing(): ...

@pytest.mark.integration
@pytest.mark.auth
def test_login_flow(client, user): ...

# .github/workflows/ci.yml
name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_USER: test_user
          POSTGRES_PASSWORD: test_pass
          POSTGRES_DB: test_db
        ports: ["5432:5432"]
        options: --health-cmd pg_isready --health-interval 10s --health-retries 5

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: {python-version: "3.12"}

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run unit tests (fast)
        run: pytest -m unit -v

      - name: Run all tests with coverage
        env:
          TEST_DATABASE_URL: postgresql://test_user:test_pass@localhost/test_db
          SECRET_KEY: test-secret-key-for-ci-only
        run: pytest --cov=app --cov-report=xml --cov-fail-under=85 -v

      - name: Upload coverage report
        uses: codecov/codecov-action@v4
        with:
          file: coverage.xml

Common Mistakes #

Mistake 1 — Not using autouse=True for external service mocks #

❌ Wrong — test accidentally calls real S3:

def test_upload_avatar(client):
    # No S3 mock — makes a real AWS API call!
    client.post("/users/me/avatar", files={...})

✅ Correct — autouse=True mock in conftest.py prevents any accidental real calls.

Mistake 2 — Setting coverage threshold too high too early #

❌ Wrong — 95% threshold on a new project blocks every commit that adds untested code:

pytest --cov-fail-under=95   # blocks all commits — demoralising

✅ Correct — start at 70%, raise by 5% as coverage improves over time.

Mistake 3 — Asserting mock was called without checking arguments #

❌ Wrong — mock called but not with correct email:

mock_email.assert_called()   # passes even if wrong email sent!

✅ Correct — check the actual arguments:

call_args = mock_email.call_args[0][0]
assert "correct@example.com" in str(call_args.recipients)   # ✓

Quick Reference — Full Testing Stack #