QA Framework

Project Overview

This is a production-grade QA automation framework demonstrating full-stack testing skills across three testing disciplines. The framework uses Playwright with TypeScript for end-to-end UI testing following the Page Object Model pattern, pytest with Python for REST API testing with schema validation and SQL persistence, and k6 for threshold-based performance testing. Two publicly accessible demo applications serve as the systems under test: saucedemo.com (e-commerce UI) and reqres.in (REST API). The framework is fully containerised with Docker and runs in a parallel CI/CD pipeline via GitHub Actions.

---

Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│                        QA FRAMEWORK                             │
├─────────────┬──────────────────┬─────────────────────────────-──┤
│  UI Tests   │   API Tests      │   Performance Tests            │
│  Playwright │   pytest +       │   k6                           │
│  TypeScript │   requests       │                                │
│  POM Pattern│   jsonschema     │   Load / Spike                 │
├─────────────┴──────────────────┴───────────────────────────────-┤
│                    Systems Under Test                           │
│         saucedemo.com          reqres.in                        │
├────────────────────────────────────────────────────────────────-┤
│                    Data Layer                                   │
│            SQLite (API response persistence)                    │
├─────────────────────────────────────────────────────────────────┤
│                    Reporting                                    │
│         Allure Reports   │   GitHub Actions Artifacts           │
└─────────────────────────────────────────────────────────────────┘

---

Prerequisites

Tool	Version	Purpose
Python	3.11+	API test runner (pytest)
Node.js	20+	UI test runner (Playwright)
npm	10+	Package management
k6	latest	Performance testing
Docker	24+	Containerised test execution
Allure CLI	2.x	Test report generation (optional)

---

Quick Start

# Clone the repository
git clone https://github.com/your-username/qa-framework.git
cd qa-framework

# Copy environment config
cp .env.example .env

# Install Python dependencies
pip install -r requirements.txt

# Install Node dependencies and Playwright browsers
npm install
npx playwright install --with-deps chromium firefox

---

Running Tests

API Tests (pytest)

# Run all API tests
pytest tests/api/ -v

# Run a specific marker group
pytest tests/api/ -m smoke -v
pytest tests/api/ -m crud -v
pytest tests/api/ -m auth -v
pytest tests/api/ -m schema -v

# Run with Allure reporting
pytest tests/api/ --alluredir=allure-results -v

UI Tests (Playwright)

# Run all UI tests (both browsers)
npm run test:ui

# Run against a single browser
npm run test:ui:chromium
npm run test:ui:firefox

# Run in headed mode (watch the browser)
npm run test:ui:headed

# Run a specific spec file
npx playwright test tests/ui/login.spec.ts --project=chromium

Performance Tests (k6)

# Load test — 50 VUs for 2 minutes
k6 run tests/performance/load_test.js

# Spike test — rapid ramp from 1 to 100 VUs
k6 run tests/performance/spike_test.js

# Export results to JSON
k6 run tests/performance/load_test.js --out json=load-results.json

Docker

# Build and run all tests in a container
docker compose up --build

# Run only the test service
docker compose run tests

# Run API tests only inside Docker
docker compose run tests pytest tests/api/ -v

Allure Reporting

# Serve the Allure report (after running pytest with --alluredir)
npm run report:serve

# Open a previously generated report
npm run report:open

# Generate static HTML report
allure generate allure-results --clean -o allure-report
allure open allure-report

The Allure Docker service is also available via docker compose up report, which serves the report at http://localhost:8080.

---

CI/CD

The GitHub Actions pipeline (.github/workflows/ci.yml) runs three parallel jobs on every push and pull request to main:

• api-tests — installs Python 3.11, runs the full pytest suite, and uploads Allure results as artifacts.
• ui-tests — uses a matrix strategy to run Playwright tests against both chromium and firefox simultaneously. Screenshots and videos are uploaded on failure.
• performance-tests — installs k6 from the official Debian repository, runs both the load and spike tests, and uploads JSON result files as artifacts. Both performance jobs use continue-on-error: true so a threshold breach does not block the pipeline but remains visible in the artifact output.

Artifacts are retained for 7 days.

---

Folder Structure

qa-framework/
├── .github/workflows/      # GitHub Actions CI pipeline
├── config/                 # Centralised configuration (URLs, timeouts, DB path)
├── db/                     # SQLite helper for API response persistence
├── fixtures/               # pytest fixtures and typed test data dataclasses
├── pages/                  # Playwright Page Object Model classes
├── tests/
│   ├── api/                # pytest API test suites (users, auth, schema)
│   ├── ui/                 # Playwright UI test specs (login, products, cart, checkout, logout)
│   └── performance/        # k6 load and spike test scripts
├── playwright.config.ts    # Playwright configuration (browsers, reporter, base URL)
├── pytest.ini              # pytest configuration (test paths, markers, allure dir)
├── conftest.py             # Root pytest conftest — registers fixtures and markers
├── requirements.txt        # Pinned Python dependencies
├── package.json            # Node dependencies and npm scripts
├── tsconfig.json           # TypeScript compiler configuration
├── Dockerfile              # Multi-tool image (Python + Node + k6 + Playwright)
├── docker-compose.yml      # Compose services: tests + Allure report server
├── .dockerignore           # Files excluded from Docker build context
└── .env.example            # Template for environment variables

---

Tech Stack

Layer	Technology	Version
UI Testing	Playwright	^1.47.0
UI Language	TypeScript	^5.5.4
API Testing	pytest	8.3.2
HTTP Client	requests	2.32.3
Schema Validation	jsonschema	4.23.0
Reporting	allure-pytest	2.13.5
Performance	k6	latest stable
Database	SQLite (stdlib)	Python 3.11
Container	Docker + Compose	24+
CI/CD	GitHub Actions	—

---

Key QA Patterns Demonstrated

Pattern	Description
Page Object Model (POM)	Each UI page is encapsulated in its own TypeScript class under pages/, keeping selectors and interactions decoupled from test logic.
Data-driven testing	Typed dataclasses in fixtures/test_data.py centralise all test inputs, making it trivial to add new user profiles or payload variants without touching test files.
Schema validation	test_schema_validation.py uses jsonschema.validate() against module-level schema constants to enforce API response contracts independently of functional assertions.
SQL assertion layer	db/sqlite_helper.py persists API responses and user records to SQLite during test runs, allowing tests to assert on stored state rather than relying solely on in-memory response objects.
Threshold-based performance testing	k6 scripts declare explicit thresholds (p95 latency < 500 ms, error rate < 1%) so a build fails automatically when the system under test degrades.
Containerised test execution	The Dockerfile bundles Python, Node.js, k6, and Playwright browsers into a single reproducible image, eliminating environment drift between local and CI runs.
Cross-browser testing	The Playwright matrix in playwright.config.ts and the GitHub Actions matrix.browser strategy ensure every UI spec is validated on both Chromium and Firefox.
Negative testing	Each test suite includes deliberate failure scenarios: invalid login credentials, locked-out users, missing checkout fields, unauthenticated API registration, and non-existent user IDs.

---

Test Coverage

File / Module	Scenarios Covered
tests/ui/login.spec.ts	Valid login, invalid credentials error, locked-out user message, error recovery on re-login
tests/ui/products.spec.ts	Product list display, A-Z sort, Z-A sort, price low-to-high, price high-to-low
tests/ui/cart.spec.ts	Add single item, add multiple items (badge count), item appears in cart, remove item
tests/ui/checkout.spec.ts	Full checkout success, missing first name error, missing last name error, order summary on step two
tests/ui/logout.spec.ts	Successful logout, redirect to login page after logout
tests/api/test_users_list.py	List users 200, pagination, data is list, per_page matches, single user, 404 for unknown, DB persistence
tests/api/test_users_crud.py	Create 201, response has id, name matches, DB persistence, PUT 200, name updated, PATCH 200, DELETE 204
tests/api/test_auth.py	Register 200, token in register response, missing password 400, error key present, login 200, login token, invalid login 400
tests/api/test_schema_validation.py	List users schema, single user schema, create user schema, register schema, login schema, support object schema
tests/performance/load_test.js	50 VUs sustained for 2 min — p95 latency and error rate thresholds on list and single-user endpoints
tests/performance/spike_test.js	Ramp from 1 to 100 VUs — GET and POST threshold validation under sudden traffic spike