AWS - HelpMeTest

LLM Testing

AWS Bedrock Model Evaluation: Automated Testing and Quality Metrics

AWS Bedrock Model Evaluation lets you run structured quality assessments against Bedrock-hosted models — comparing outputs against ground truth, scoring on custom criteria, and tracking results across model versions. Here's how to set up automated evaluation pipelines using the Bedrock API and boto3. What Bedrock Model Evaluation Offers Bedrock

API Testing

AWS Cognito Testing Guide: User Pools, Identity Pools, JWT Validation, and Amplify

AWS Cognito provides user authentication, authorization, and identity management for AWS-hosted applications. Testing Cognito integrations is critical — a misconfigured user pool can lock users out, a broken Lambda trigger can block all signups, and a wrong JWT audience claim can silently reject valid users. This guide covers testing Cognito user

DevOps

AWS CDK Testing Patterns: Assertions, Snapshots, and Integration Tests

AWS CDK generates CloudFormation templates from TypeScript, Python, or Java code. The CDK Assertions library lets you test the generated template — verifying resource configurations without deploying anything. CDK snapshot tests catch unintended template changes. CDK integ tests validate actual deployments in real AWS accounts. Together, these three patterns give you

Testing

Object Storage Testing Best Practices: Mocking vs Real Buckets, CORS, CDN Invalidation, and Encryption

Object storage tests fall into three tiers: unit tests with mocks (fast, free, no network), integration tests against local emulators (slower, but real behavior), and end-to-end tests against real cloud buckets (slowest, costs money, but catches IAM and CDN issues nothing else will). The key is knowing which tier each

Testing

Testing S3 Multipart Uploads: Chunked Uploads, Resumable Uploads, and Large File Handling

S3 multipart upload is a three-phase protocol: initiate (get an upload ID), upload parts (minimum 5MB each), and complete (assemble the parts). Testing it requires verifying each phase independently, handling aborts, and testing resumable upload state management. moto supports multipart upload end-to-end. Key Takeaways Multipart upload is a three-phase transaction.

Testing

Amazon Redshift Testing: Unit Tests, Data Validation, and CI Integration

Testing Redshift pipelines is expensive and slow when done against a live cluster. This guide covers running Redshift-compatible tests against LocalStack, validating COPY/UNLOAD workflows, asserting on distribution and sort key behavior, and integrating Great Expectations for ongoing data quality monitoring — all in a CI pipeline that costs nothing per

Testing

AWS S3 Testing Guide: Unit Testing with moto, Presigned URLs, Versioning, and Lifecycle Policies

Testing AWS S3 without hitting real buckets is straightforward with moto, a Python library that intercepts boto3 calls and simulates S3 in memory. This guide covers unit testing bucket operations, presigned URL generation, versioning, and lifecycle policies with runnable examples. Key Takeaways moto replaces real AWS calls in tests. Decorate

Testing

AWS Messaging Testing Best Practices: DLQ, Retry, Idempotency, and Message Ordering

The failure modes of SQS, SNS, and EventBridge are well-known: duplicate delivery, out-of-order messages, poison pills that fill DLQs, and missing idempotency guards that corrupt state. This post covers the testing patterns that catch these bugs before they reach production. Key Takeaways Test idempotency explicitly. SQS delivers messages at least

Testing

LocalStack Integration Testing: Full SQS + SNS + Lambda Stack Setup

moto is great for unit tests but stops at service boundaries. When you need SQS triggering Lambda, SNS fan-out to multiple SQS queues, or Lambda updating DynamoDB — you need LocalStack. This guide sets up a full LocalStack integration test environment with Docker Compose, pytest fixtures, and reusable infrastructure helpers. Key

Testing

AWS EventBridge Testing Guide: Event Rules, Schema Validation, and Integration Tests

EventBridge is AWS's serverless event bus. Testing it means verifying that events match the right rules, rules route to the right targets, schemas validate correctly, and cross-bus or cross-account flows work as expected. moto supports EventBridge for unit tests; LocalStack handles the full cross-service integration. Key Takeaways Test

Testing

AWS SNS Testing Strategies: Topics, Fan-Out Patterns, and DLQ Handling

SNS is a pub/sub service where one publisher sends to a topic and multiple subscribers receive. Testing SNS means verifying: messages reach the right subscribers, message filters work correctly, delivery failures go to the DLQ, and fan-out to SQS queues behaves as expected. moto handles all of this in-process

Testing

AWS SQS Testing Guide: Unit and Integration Testing with LocalStack, moto, and boto3

Testing SQS consumers and producers without hitting real AWS is the only way to run fast, isolated tests. moto gives you an in-process mock for unit tests. LocalStack gives you a full local AWS stack for integration tests. This guide covers both, with patterns for FIFO queues, dead-letter queues, visibility