This document describes the testing infrastructure for Claude How To.

Overview

The project uses GitHub Actions to automatically run tests on every push and pull request. Tests cover:

Running Tests Locally

Prerequisites

# Install uv (fast Python package manager)
pip install uv

# Or on macOS with Homebrew
brew install uv

Setup Environment

# Clone the repository
git clone YOUR_REPO
cd claude-howto

# Create virtual environment
uv venv

# Activate it
source .venv/bin/activate  # macOS/Linux
# or
.venv\Scripts\activate     # Windows

# Install development dependencies
uv pip install -r requirements-dev.txt

Run Tests

# Run all unit tests
pytest scripts/tests/ -v

# Run tests with coverage
pytest scripts/tests/ -v --cov=scripts --cov-report=html

# Run specific test file
pytest scripts/tests/test_build_epub.py -v

# Run specific test function
pytest scripts/tests/test_build_epub.py::test_function_name -v

# Run tests in watch mode (requires pytest-watch)
ptw scripts/tests/

Run Linting

# Check code formatting
ruff format --check scripts/

# Auto-fix formatting issues
ruff format scripts/

# Run linter
ruff check scripts/

# Auto-fix linter issues
ruff check --fix scripts/

Run Security Scan

# Run Bandit security scan
bandit -c pyproject.toml -r scripts/ --exclude scripts/tests/

# Generate JSON report
bandit -c pyproject.toml -r scripts/ --exclude scripts/tests/ -f json -o bandit-report.json

Run Type Checking

# Check types with mypy
mypy scripts/ --ignore-missing-imports --no-implicit-optional

GitHub Actions Workflow

Triggered On

Jobs

1. Unit Tests (pytest)

Outcome: If any test fails, the workflow fails (critical)

2. Code Quality (Ruff)

Outcome: Non-blocking (warning only)

3. Security Scan (Bandit)

Outcome: Non-blocking (warning only)

4. Type Checking (mypy)

Outcome: Non-blocking (warning only)

5. Build EPUB

Outcome: If build fails, the workflow fails (critical)

6. Summary

Writing Tests

Test Structure

Tests should be placed in scripts/tests/ with names like test_*.py:

# scripts/tests/test_example.py
import pytest
from scripts.example_module import some_function

def test_basic_functionality():
    """Test that some_function works correctly."""
    result = some_function("input")
    assert result == "expected_output"

def test_error_handling():
    """Test that some_function handles errors gracefully."""
    with pytest.raises(ValueError):
        some_function("invalid_input")

@pytest.mark.asyncio
async def test_async_function():
    """Test async functions."""
    result = await async_function()
    assert result is not None

Test Best Practices

Fixtures

Common fixtures are defined in scripts/tests/conftest.py:

# Use fixtures in your tests
def test_something(tmp_path):
    """tmp_path fixture provides temporary directory."""
    test_file = tmp_path / "test.txt"
    test_file.write_text("content")
    assert test_file.read_text() == "content"

Coverage Reports

Local Coverage

# Generate coverage report
pytest scripts/tests/ --cov=scripts --cov-report=html

# Open the coverage report in your browser
open htmlcov/index.html

Coverage Goals

Pre-commit Hooks

The project uses pre-commit hooks to run checks automatically before commits:

# Install pre-commit hooks
pre-commit install

# Run hooks manually
pre-commit run --all-files

# Skip hooks for a commit (not recommended)
git commit --no-verify

Configured hooks in .pre-commit-config.yaml:

Troubleshooting

Tests Pass Locally but Fail in CI

Common causes:

  1. Python version difference: CI uses 3.10, 3.11, 3.12
  2. Missing dependencies: Update requirements-dev.txt
  3. Platform differences: Path separators, environment variables
  4. Flaky tests: Tests that depend on timing or order

Solution:

# Test with the same Python versions
uv python install 3.10 3.11 3.12

# Test with clean environment
rm -rf .venv
uv venv
uv pip install -r requirements-dev.txt
pytest scripts/tests/

Bandit Reports False Positives

Some security warnings may be false positives. Configure in pyproject.toml:

[tool.bandit]
exclude_dirs = ["scripts/tests"]
skips = ["B101"]  # Skip assert_used warning

Type Checking Too Strict

Relax type checking for specific files:

# Add at the top of file
# type: ignore

# Or for specific lines
some_dynamic_code()  # type: ignore

Continuous Integration Best Practices

  1. Keep tests fast: Each test should complete in <1 second
  2. Don't test external APIs: Mock external services
  3. Test in isolation: Each test should be independent
  4. Use clear assertions: assert x == 5 not assert x
  5. Handle async tests: Use @pytest.mark.asyncio
  6. Generate reports: Coverage, security, type checking

Resources