Testing
Testing AWS S3 without hitting real buckets is straightforward with moto, a Python library that intercepts boto3 calls and simulates S3 in memory. This guide covers unit testing bucket operations, presigned URL generation, versioning, and lifecycle policies with runnable examples. Key Takeaways moto replaces real AWS calls in tests. Decorate
Testing
moto is great for unit tests but stops at service boundaries. When you need SQS triggering Lambda, SNS fan-out to multiple SQS queues, or Lambda updating DynamoDB — you need LocalStack. This guide sets up a full LocalStack integration test environment with Docker Compose, pytest fixtures, and reusable infrastructure helpers.
Testing
EventBridge is AWS's serverless event bus. Testing it means verifying that events match the right rules, rules route to the right targets, schemas validate correctly, and cross-bus or cross-account flows work as expected. moto supports EventBridge for unit tests; LocalStack handles the full cross-service integration.
Testing
SNS is a pub/sub service where one publisher sends to a topic and multiple subscribers receive. Testing SNS means verifying: messages reach the right subscribers, message filters work correctly, delivery failures go to the DLQ, and fan-out to SQS queues behaves as expected. moto handles all of this
Testing
Redis Streams differ from simple pub/sub — they persist messages and track consumer group acknowledgments via the Pending Entries List (PEL). Testing consumer groups means verifying XACK behavior, idle consumer claim logic, and re-delivery after consumer failure. Testcontainers provides a real Redis instance; Python's redis-py gives
Testing
Testing SQS consumers and producers without hitting real AWS is the only way to run fast, isolated tests. moto gives you an in-process mock for unit tests. LocalStack gives you a full local AWS stack for integration tests. This guide covers both, with patterns for FIFO queues, dead-letter
Python
Celery powers background task processing in Python applications. It's powerful but notoriously tricky to test because of its asynchronous, distributed nature. The good news: Celery ships with testing infrastructure that makes most scenarios straightforward. The tricky parts — retries, chains, countdown — require some extra care. Core Testing Modes CELERY_
Database
Alembic is the standard database migration tool for SQLAlchemy applications, but its revision system is easy to break silently. This post covers a complete pytest-based testing strategy: upgrade and downgrade testing with real databases, Testcontainers for isolated environments, SQLAlchemy fixture patterns, and idempotency verification to catch the errors that
Data Engineering
Spark Structured Streaming tests fall into three layers: transformation unit tests using static DataFrames, micro-batch simulation using MemoryStream for source-side logic, and full integration tests with Testcontainers-Kafka. Watermark and late-data behavior requires careful trigger and clock control that MemoryStream provides without real streaming infrastructure. Key Takeaways
Data Engineering
Great Expectations structures data quality as code: Expectations describe what data should look like, Validators apply them to DataFrames or tables, and Checkpoints bundle validation into CI-runnable steps. Data Docs turn validation results into human-readable HTML reports your team can share. Key Takeaways Write expectations from profiles, refine
Testing
Syrupy is the most popular snapshot testing plugin for pytest. It stores snapshots in __snapshots__/ directories alongside your test files, uses YAML-like .ambr format for human-readable diffs, and integrates with pytest's assertion rewriting for clear failure messages. This guide covers installation, writing snapshot tests, updating snapshots,
Testing
WeasyPrint converts HTML/CSS to PDF in Python, making it popular for invoice generation, report rendering, and document exports in Django/Flask apps. Testing WeasyPrint output involves extracting text from the generated PDF bytes, comparing page structure, and doing visual regression checks against PNG renderings. This guide covers the full