Testing
Syrupy is the most popular snapshot testing plugin for pytest. It stores snapshots in __snapshots__/ directories alongside your test files, uses YAML-like .ambr format for human-readable diffs, and integrates with pytest's assertion rewriting for clear failure messages. This guide covers installation, writing snapshot tests, updating snapshots, and handling
Testing
AI agent pipelines need CI/CD just like any software — but standard pipelines don't account for LLM non-determinism, evaluation costs, or model-level regressions. This guide covers building CI/CD pipelines specifically for AI agent systems: fast deterministic checks, LLM evaluation gates, model upgrade workflows, safety guardrails, and production
Testing
Retrieval-Augmented Generation (RAG) systems are notoriously hard to test. Unlike deterministic software where the same input always produces the same output, RAG pipelines combine a retrieval step (vector search, keyword search, or hybrid) with an LLM generation step — both of which introduce variability. A RAG system can appear to work
Testing
RAG evaluation frameworks exist because manual inspection doesn't scale. You can't read 10,000 (question, answer, context) triples and judge whether the model is hallucinating, whether the retrieved context is relevant, or whether the answer actually addresses the question. RAGAS and TruLens automate this judgment using
Testing
Vector databases are infrastructure. Like any database, they can be misconfigured, queried incorrectly, or updated in ways that corrupt your index. Unlike relational databases, the failure modes are subtle: incorrect metadata filters silently narrow results, wrong namespace routing returns data from a different index, stale vectors from deleted documents still
Testing
Embeddings are the foundation of semantic search, RAG systems, and recommendation engines. When embedding behavior is wrong — whether because the model changed, the preprocessing changed, or the similarity calculation is buggy — everything downstream breaks silently. Search returns irrelevant results. RAG systems hallucinate. Recommendations get weird. Testing embeddings requires thinking about
Testing
Chunking is one of the most impactful and least-tested parts of a RAG pipeline. How you split documents into chunks directly determines retrieval quality: too small and chunks lose context; too large and they include irrelevant content that confuses the LLM; wrong boundaries and questions about sentence-end information miss the
Testing
LangChain and LlamaIndex are composable systems. A RAG chain is a sequence of components: a retriever, a prompt template, an LLM, an output parser. Each component can fail, and their integration can fail in ways that none of the individual components would catch in isolation. Integration testing for RAG chains
Testing
PDF testing requires a layered approach: text content assertions, page structure checks, visual regression diffs, and accessibility validation. This guide covers strategies that work regardless of how your PDFs are generated — Puppeteer, WeasyPrint, iText, PDFBox, ReportLab, or any other tool — and shows how to combine them into a CI-ready pipeline.
Testing
Multi-agent systems (LangGraph workflows, CrewAI crews) are dramatically harder to test than single agents. State flows between agents, handoffs can fail silently, and emergent behaviors arise from agent interactions. This guide covers testing strategies for multi-agent orchestration: state validation, handoff testing, agent isolation, and failure injection. Single agents are hard
Testing
The .NET ecosystem has two mature approval/snapshot testing libraries: ApprovalTests.Net (the original) and Verify (the modern alternative with better async support and scrubbing). Both capture output to files and compare on subsequent runs. This guide covers both libraries with practical C# examples for NUnit and xUnit. Key Takeaways
Testing
Rate limiting in multi-tenant SaaS serves two goals: protecting infrastructure and enforcing plan limits. Both can break in ways that are hard to catch without explicit tests. The failure modes are distinct. Infrastructure protection fails when one tenant can starve other tenants — their traffic exhausts rate limit headroom that should