Testing
Mailtrap provides a fake SMTP inbox that captures outgoing emails so they never reach real users during testing. Beyond just trapping emails, its API lets you fetch email content, assert on subject and body, check spam scores, and validate HTML rendering. This guide covers the full Mailtrap testing workflow for
Data Engineering
Spark Structured Streaming tests fall into three layers: transformation unit tests using static DataFrames, micro-batch simulation using MemoryStream for source-side logic, and full integration tests with Testcontainers-Kafka. Watermark and late-data behavior requires careful trigger and clock control that MemoryStream provides without real streaming infrastructure. Key Takeaways Test transformations with static
Data Engineering
dbt's testing model goes well beyond the four built-in tests. Generic tests written as macros are reusable across your entire project, singular tests handle complex multi-table business rules, and the dbt-expectations package brings 50+ Great Expectations-style assertions into SQL. In CI, dbt test --select state:modified+ ensures only
Data Engineering
Data pipeline QA requires a layered approach: unit tests for transformation logic, contract tests for schema agreements between systems, integration tests for end-to-end correctness, and SLA-based operational tests for latency and freshness. Idempotency and chaos testing are the two most commonly skipped layers — and the two most likely to catch
Data Engineering
Testing Apache Flink requires specialized tools at each layer: MiniClusterWithClientResource for topology-level tests, KeyedOneInputStreamOperatorTestHarness for stateful operators, and EmbeddedKafkaCluster for end-to-end integration. Event-time semantics and exactly-once guarantees demand explicit test harness control over watermarks and checkpoints. Key Takeaways Unit test operators in isolation. Use KeyedOneInputStreamOperatorTestHarness to feed elements and watermarks
Data Engineering
Great Expectations structures data quality as code: Expectations describe what data should look like, Validators apply them to DataFrames or tables, and Checkpoints bundle validation into CI-runnable steps. Data Docs turn validation results into human-readable HTML reports your team can share. Key Takeaways Write expectations from profiles, refine manually. The
Observability
Sentry integration is easy to break silently—a wrong DSN, a muted beforeSend hook, or a missing SDK init call can cause errors to vanish without anyone noticing. This guide covers how to verify Sentry actually captures what it should: SDK mocks in unit tests, integration tests for captureException, testing
Observability
Prometheus alerting rules can fail silently for years: a typo in a PromQL expression, a label mismatch in Alertmanager routing, or a threshold that never actually fires. promtool provides a native unit testing framework for alert rules—YAML-defined tests that validate expressions against synthetic time-series data without running a live
Observability
OpenTelemetry instrumentation can silently break—spans stop being created, attributes go missing, or trace context fails to propagate across service boundaries. The OTel SDK ships with an InMemorySpanExporter specifically for testing: it captures spans in memory so you can assert their structure without running a collector. Key Takeaways InMemorySpanExporter is
Observability
Logs are observable outputs of your system. Like return values and HTTP responses, log output can be tested—and should be. This guide covers structured log assertions in integration tests, capturing logs in Python and Java, testing log level and context fields, asserting no unexpected errors appear during a test
Observability
Datadog is not just a dashboard—it's a test surface. This guide covers asserting APM trace structure in tests, using Datadog Synthetics as CI quality gates, enforcing p99 latency thresholds before deployment, and mocking the Datadog agent so unit tests stay fast and isolated. Key Takeaways APM traces
DevOps
Flux automates GitOps deployments, which means a broken Kustomization or HelmRelease can silently fail to reconcile — or worse, reconcile incorrectly — without anyone noticing until a production incident. This guide covers the full Flux testing stack: flux build for local manifest rendering, flux diff for live drift detection, envtest for controller