Google Actions Testing: pytest, Local Fulfillment, and the Actions Console
Google Actions testing spans three environments: the Actions Console simulator for interactive exploration, ngrok tunnels for local fulfillment development, and automated pytest suites for webhook logic and conversation flow regression. This guide covers all three with production-ready code.
Google Actions (built with Actions Builder or the Actions SDK v3) presents a unique testing challenge: the platform handles NLU and conversation management, while your fulfillment webhook handles business logic and dynamic responses. Your test strategy needs to cover both sides — the platform behavior and your webhook code — using different tools for each.
Understanding the Testing Split
When a user says something to your Action, the request flows through Google's NLU engine, gets matched to a scene and intent, and then optionally calls your webhook. This means:
- Platform behavior (intent matching, scene transitions, slot filling) — test with the Actions Console simulator or the
gactionsCLI - Webhook logic (dynamic responses, external API calls, data validation) — test with Flask test client and pytest
- End-to-end flows — test with the Acts on Google Python client library or conversation log analysis
Getting this split wrong leads to either over-testing the platform (Google tests their own NLU; you don't need to) or under-testing your webhook (where your bugs actually live).
Actions Console Simulator
The Actions Console at console.actions.google.com provides a simulator where you can run complete conversations against your deployed Action. This is useful for exploratory testing and quick regression checks, but it's not automatable.
For systematic testing, use the gactions CLI to push your project and run test interactions. First, install the CLI and authenticate:
# Install gactions CLI
curl -OL https://dl.google.com/gactions/v3/release/gactions_linux_amd64.tar.gz
tar -xzf gactions_linux_amd64.tar.gz
<span class="hljs-built_in">sudo <span class="hljs-built_in">mv gactions /usr/local/bin/
<span class="hljs-comment"># Authenticate
gactions login
<span class="hljs-comment"># Push your Action project
gactions push --project-id my-action-project-id
<span class="hljs-comment"># Deploy to production
gactions deploy preview --project-id my-action-project-idLocal Fulfillment with ngrok
During development, you want your local webhook server to receive Actions requests without deploying. ngrok creates a public HTTPS tunnel to your local server:
# Start your local fulfillment server
python fulfillment.py <span class="hljs-comment"># runs on port 5000
<span class="hljs-comment"># In another terminal, start ngrok
ngrok http 5000
<span class="hljs-comment"># Copy the https URL (e.g., https://abc123.ngrok.io)
<span class="hljs-comment"># Set this as your webhook URL in Actions Console > WebhookThen configure your action's webhook URL in actions.yaml or the console to point to your ngrok URL. The Actions Console simulator will now hit your local server.
Flask Webhook: Structure and Testing
The Actions SDK v3 webhook uses a specific JSON schema. Here's a properly structured Flask webhook with all the touchpoints you need to test:
# fulfillment.py
from flask import Flask, request, jsonify
import json
app = Flask(__name__)
@app.route('/webhook', methods=['POST'])
def webhook():
req = request.get_json(silent=True)
handler = req.get('handler', {}).get('name', '')
intent = req.get('intent', {}).get('name', '')
scene = req.get('scene', {}).get('name', '')
params = req.get('intent', {}).get('params', {})
session_params = req.get('session', {}).get('params', {})
if handler == 'validate_booking_date':
return handle_validate_date(params, session_params)
elif handler == 'confirm_booking':
return handle_confirm_booking(session_params)
elif handler == 'cancel_booking':
return handle_cancel(session_params)
else:
return jsonify({"prompt": {"override": False, "firstSimple": {"speech": "Sorry, I didn't understand that."}}})
def handle_validate_date(params, session_params):
date_param = params.get('date', {})
date_value = date_param.get('resolved') or date_param.get('original')
if not date_value:
return jsonify({
"prompt": {
"override": False,
"firstSimple": {"speech": "What date would you like?"}
},
"scene": {"next": {"name": "CollectDate"}}
})
if not is_future_date(date_value):
return jsonify({
"prompt": {
"override": False,
"firstSimple": {"speech": f"Sorry, {date_value} is in the past. Please choose a future date."}
},
"scene": {"next": {"name": "CollectDate"}}
})
return jsonify({
"prompt": {"override": False},
"session": {"params": {"booking_date": date_value}},
"scene": {"next": {"name": "CollectTime"}}
})Now test this webhook in complete isolation from the Actions platform:
# tests/test_fulfillment.py
import pytest
import json
from unittest.mock import patch
from fulfillment import app
@pytest.fixture
def client():
app.config['TESTING'] = True
with app.test_client() as c:
yield c
def make_actions_request(handler, intent_params=None, session_params=None):
"""Build a Google Actions webhook request payload."""
return {
"handler": {"name": handler},
"intent": {
"name": "actions.intent.MAIN",
"params": intent_params or {},
"query": "test query",
},
"scene": {"name": "BookingScene", "slotFillingStatus": "FINAL"},
"session": {
"id": "test-session-id",
"params": session_params or {},
"typeOverrides": [],
"languageCode": "en",
},
"user": {"locale": "en-US", "params": {}},
"home": {},
"device": {"capabilities": ["SPEECH", "RICH_RESPONSE"]},
}
def test_validate_date_success(client):
payload = make_actions_request(
handler="validate_booking_date",
intent_params={"date": {"original": "tomorrow", "resolved": "2026-05-18"}}
)
response = client.post('/webhook',
data=json.dumps(payload),
content_type='application/json')
assert response.status_code == 200
data = response.get_json()
# Verify scene transition to CollectTime
assert data["scene"]["next"]["name"] == "CollectTime"
# Verify date stored in session
assert data["session"]["params"]["booking_date"] == "2026-05-18"
def test_validate_date_past_date(client):
payload = make_actions_request(
handler="validate_booking_date",
intent_params={"date": {"original": "yesterday", "resolved": "2026-05-16"}}
)
response = client.post('/webhook',
data=json.dumps(payload),
content_type='application/json')
data = response.get_json()
# Should stay on CollectDate
assert data["scene"]["next"]["name"] == "CollectDate"
speech = data["prompt"]["firstSimple"]["speech"]
assert "past" in speech.lower()
def test_validate_date_missing(client):
payload = make_actions_request(
handler="validate_booking_date",
intent_params={}
)
response = client.post('/webhook',
data=json.dumps(payload),
content_type='application/json')
data = response.get_json()
assert data["scene"]["next"]["name"] == "CollectDate"
def test_confirm_booking_creates_record(client):
payload = make_actions_request(
handler="confirm_booking",
session_params={
"booking_date": "2026-05-20",
"booking_time": "19:00",
"party_size": 2,
}
)
with patch('fulfillment.create_booking_record') as mock_create:
mock_create.return_value = {"id": "BK-12345", "status": "confirmed"}
response = client.post('/webhook',
data=json.dumps(payload),
content_type='application/json')
data = response.get_json()
mock_create.assert_called_once_with(
date="2026-05-20", time="19:00", party_size=2
)
speech = data["prompt"]["firstSimple"]["speech"]
assert "BK-12345" in speech or "confirmed" in speech.lower()Automated Conversation Testing with the Python Client
For end-to-end automated testing against the actual Actions platform, you can use the google-assistant-sdk or the newer Actions API to run test conversations programmatically:
# tests/test_conversation_flows.py
import pytest
import subprocess
import json
import time
class ActionsSimulator:
"""Wrapper around gactions CLI for programmatic conversation testing."""
def __init__(self, project_id: str, locale: str = "en-US"):
self.project_id = project_id
self.locale = locale
def send(self, utterance: str, session_id: str = None) -> dict:
"""Send an utterance and return the response."""
cmd = [
"gactions", "simulator",
"--project-id", self.project_id,
"--locale", self.locale,
"--input", utterance,
]
if session_id:
cmd.extend(["--session", session_id])
result = subprocess.run(cmd, capture_output=True, text=True)
return json.loads(result.stdout)
@pytest.fixture(scope="session")
def simulator():
return ActionsSimulator(project_id="my-action-project-id")
def test_booking_happy_path(simulator):
session_id = f"test-{int(time.time())}"
# Invoke action
response = simulator.send("Talk to My Booking Action", session_id)
assert "welcome" in response["text"].lower()
# Start booking
response = simulator.send("book a table for two", session_id)
assert "date" in response["text"].lower()
# Provide date
response = simulator.send("next Friday", session_id)
assert "time" in response["text"].lower()
# Provide time
response = simulator.send("seven pm", session_id)
assert "confirm" in response["text"].lower()
# Confirm
response = simulator.send("yes", session_id)
assert "confirmed" in response["text"].lower() or "booking" in response["text"].lower()Conversation Log Analysis
Production Google Actions generate conversation logs in Google Cloud Logging. Analyzing these logs reveals patterns that tests don't catch — common drop-off points, misunderstood utterances, and error rates by intent.
from google.cloud import logging_v2
from google.cloud.logging_v2 import DESCENDING
import json
from datetime import datetime, timedelta
def analyze_action_logs(project_id: str, hours_back: int = 24):
client = logging_v2.Client(project=project_id)
filter_str = (
f'resource.type="assistant_action_project" '
f'timestamp>="{(datetime.utcnow() - timedelta(hours=hours_back)).isoformat()}Z"'
)
intent_counts = {}
error_turns = []
for entry in client.list_entries(filter_=filter_str, order_by=DESCENDING, max_results=1000):
payload = entry.payload
if isinstance(payload, dict):
intent = payload.get("queryResult", {}).get("intent", {}).get("displayName")
if intent:
intent_counts[intent] = intent_counts.get(intent, 0) + 1
if payload.get("responseCode") and int(payload["responseCode"]) >= 400:
error_turns.append(payload)
print(f"Intent distribution (last {hours_back}h):")
for intent, count in sorted(intent_counts.items(), key=lambda x: -x[1]):
print(f" {intent}: {count}")
print(f"\nError turns: {len(error_turns)}")
return intent_counts, error_turnsCI/CD Integration
Combine webhook unit tests with a deployment smoke test:
name: Google Actions Tests
on: [push, pull_request]
jobs:
webhook-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: '3.12' }
- run: pip install flask pytest pytest-mock
- run: pytest tests/test_fulfillment.py -v --tb=short
deploy-and-smoke-test:
needs: webhook-tests
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- uses: google-github-actions/auth@v2
with: { credentials_json: '${{ secrets.GCP_SA_KEY }}' }
- run: gactions push --project-id ${{ secrets.ACTIONS_PROJECT_ID }}
- run: gactions deploy preview --project-id ${{ secrets.ACTIONS_PROJECT_ID }}
- run: pytest tests/test_smoke.py -vHelpMeTest can run your Google Actions conversation flows against production on a schedule, giving you continuous visibility into whether your Action responds correctly after any platform or webhook update.
