Testing
LangSmith gives you production observability for LLM applications — full request traces, cost tracking, latency breakdowns, and human annotation queues. Combined with its evaluation layer, you can compare prompt versions, run automated evaluators, and catch regressions before they reach users. Why Tracing Matters for LLM Applications An LLM application isn'
Testing
Testing pandas code requires different patterns than testing application code. DataFrames are complex objects—you rarely want to assert on the entire DataFrame at once. Instead, assert on specific columns, row counts, data types, and statistical properties. Use pytest fixtures for reusable test data, pandas.testing for DataFrame comparisons, and
Testing
SAML SSO testing requires testing both SP-initiated and IdP-initiated flows, SAML assertion validation (signature, conditions, audience), and error handling for invalid assertions. Most teams only test the happy path through the browser UI. This guide covers automated testing of SAML assertion handling, IdP simulation for integration tests, and
Testing
ServiceNow's Automated Test Framework (ATF) is a built-in, no-license-required testing tool that supports UI tests, server-side script tests, and API tests. This guide covers ATF setup, test writing patterns, and CI/CD integration using the ServiceNow REST API. What Is ServiceNow ATF? The Automated
Testing
Promptfoo is the standard tool for prompt regression testing and LLM red-teaming. Define test cases in YAML, run them against multiple models simultaneously, and catch prompt regressions before they reach production. The red-team mode automatically probes for jailbreaks, prompt injection, and safety failures. The Problem Promptfoo Solves Every
Testing
Session management testing covers four areas: cookie security attributes (HttpOnly, Secure, SameSite), CSRF protection, token refresh behavior, and session lifecycle (creation, invalidation, concurrent sessions). These aren't glamorous tests, but session vulnerabilities — session fixation, missing CSRF protection, cookies over HTTP — are consistently in the OWASP Top 10. This guide
Testing
Data pipeline orchestrators (Airflow, Prefect, Dagster) each have different testing models, but the principle is the same: unit-test your business logic in isolation, integration-test the orchestration layer with real or containerized dependencies. This guide covers testing patterns for all three orchestrators plus shared practices that apply to any
Testing
Keycloak testing requires a dedicated test realm to isolate test users and configuration from production. The Keycloak REST Admin API handles test setup and teardown — creating users, resetting passwords, and managing roles programmatically. For E2E tests, token generation via the Direct Access Grant bypasses browser login. This guide covers Keycloak
Testing
SAP testing requires tools that understand SAP's proprietary GUI, transaction codes, and BAPI interfaces. This guide covers the three main automation approaches — UTFx, Worksoft Certify, and SAP's built-in eCATT — plus how to integrate them into CI pipelines. SAP Testing Challenges Testing SAP is fundamentally different
Testing
Ragas gives you a rigorous metric suite for RAG pipelines: faithfulness, answer relevancy, context precision, and context recall. Each metric isolates a different failure mode — bad retrieval vs. bad generation vs. incomplete recall. This guide shows you how to compute them, interpret scores, and integrate Ragas into CI. The RAG
Testing
JWT testing requires testing your token validation logic — not just the happy path (valid token accepted), but also expiry, wrong algorithm, tampered signatures, missing claims, and algorithm confusion attacks. This guide covers creating test JWTs with PyJWT and jsonwebtoken, testing validation middleware, mocking JWT verification for unit tests, and the
Testing
Great Expectations (GX) is a Python library for defining and running data quality assertions against DataFrames, files, and databases. You define Expectations (e.g., "this column should be non-null"), group them into Expectation Suites, and run them via Checkpoints. Failed Expectations produce HTML Data Docs reports showing