Gauge

Building Custom Gauge Plugins: Messages API and Beyond

HelpMeTest

HelpMeTest

— 8 min read
Building Custom Gauge Plugins: Messages API and Beyond

Gauge plugins communicate with the Gauge runner over a socket using Protocol Buffers. There are three plugin types: language runners (execute step implementations), reporting plugins (generate reports), and general plugins (extend functionality). Building a custom plugin means implementing a small socket server that speaks the Gauge protobuf message format. The Messages API is well-defined and stable, making it possible to build plugins in any language that has protobuf support.

Key Takeaways

  • Gauge plugins communicate via a Unix/TCP socket using Protocol Buffers messages
  • Three plugin types: language runner, reporting plugin, general plugin
  • The plugin manifest (plugin.json) declares type, entry point, and compatible Gauge versions
  • Reporting plugins receive execution lifecycle events: suite start/end, spec start/end, scenario results
  • Language runner plugins are complex — contribute to existing runners rather than building new ones
  • General plugins (like custom notifiers or exporters) are the most accessible entry point for custom development
  • The gauge.proto file defines all message types; generated Go/Python/Java code is available

Why Build a Custom Gauge Plugin?

The built-in Gauge plugins cover most needs: HTML reports, XML reports, JSON export, and language runners for Java, Python, JavaScript, Ruby, and C#. Custom plugins become relevant when you need:

  • Custom reporting: Push results to Slack, PagerDuty, or an internal dashboard
  • Test result storage: Save results to a database for trend analysis
  • Integration with proprietary tools: Export to your company's test management system (TestRail, Zephyr, qTest)
  • Custom notifications: Alert specific channels based on which tags failed
  • Execution hooks at the infrastructure level: Spin up/tear down external services around test runs

This post focuses on building a general-purpose plugin — the most accessible type — that receives execution events and does something useful with them.

How Gauge Plugin Communication Works

When Gauge starts, it reads the manifest.json of each installed plugin, launches the plugin process, and establishes a socket connection. The plugin and Gauge exchange Protocol Buffer messages over this socket throughout the test run.

The message flow looks like this:

Gauge Runner                    Your Plugin
     |                               |
     |-- SuiteExecutionResult -----> |
     |-- SpecExecutionResult ------> |
     |-- ScenarioExecutionResult --> |
     |-- StepExecutionResult ------> |
     |-- SuiteExecutionResult -----> |  (final, with aggregate stats)
     |                               |
     |<-- Empty (acknowledgement) ---|

The protobuf definitions live in the gauge-proto repository. Every message type is defined there, including ExecutionResult, SpecInfo, ScenarioInfo, FailureScreenshot, and ProtoStep.

Plugin Architecture

Plugin Manifest

Every plugin needs a plugin.json manifest:

{
  "id": "slack-reporter",
  "version": "1.0.0",
  "name": "Slack Reporter",
  "description": "Posts Gauge test results to a Slack channel",
  "pluginPath": ".",
  "type": "reporting",
  "pluginsApiVersion": "1.0.0",
  "gauge_version_support": {
    "minimum": "1.0.0",
    "maximum": "1.99.99"
  },
  "run": {
    "linux": {
      "x86": "bin/slack-reporter"
    },
    "darwin": {
      "x86": "bin/slack-reporter",
      "arm": "bin/slack-reporter-arm64"
    },
    "windows": {
      "x86": "bin/slack-reporter.exe"
    }
  }
}

The type field can be "reporting", "language", or "" (general). Reporting plugins receive execution result messages. General plugins can define their own lifecycle.

Plugin Types in Detail

Language Runner Plugins (type: "language"): Receive step execution requests, resolve them to step implementations, execute the code, and return results. These are the most complex — they implement a full RPC protocol. Unless you need to support a new programming language, avoid building these from scratch.

Reporting Plugins (type: "reporting"): Receive execution lifecycle events after each spec, scenario, and suite completes. These are the right type for result reporting, export, and notification plugins.

General Plugins (type: ""): Receive a connection and can implement any behavior. Used for tools that need to be co-located with the Gauge process but have their own protocol.

Building a Reporting Plugin in Go

Go is the natural choice for Gauge plugins — the Gauge runner itself is written in Go, and the official plugins use Go. But any language with protobuf support works.

Project Structure

slack-reporter/
├── plugin.json
├── main.go
├── slack.go
├── go.mod
└── go.sum

Dependencies

go mod init github.com/yourorg/gauge-slack-reporter
go get google.golang.org/protobuf
go get github.com/getgauge/gauge-proto/go/gauge_messages

Main Plugin Loop

The Gauge runner starts your plugin with a --port flag specifying the socket port to connect to:

// main.go
package main

import (
    "flag"
    "fmt"
    "log"
    "net"
    "os"
    "strconv"
    
    gm "github.com/getgauge/gauge-proto/go/gauge_messages"
    "google.golang.org/protobuf/proto"
)

var port = flag.Int("port", 0, "Port to connect to Gauge runner")

func main() {
    flag.Parse()
    
    if *port == 0 {
        log.Fatal("Port not specified. Use --port flag.")
    }
    
    conn, err := net.Dial("tcp", fmt.Sprintf("localhost:%d", *port))
    if err != nil {
        log.Fatalf("Failed to connect to Gauge runner: %v", err)
    }
    defer conn.Close()
    
    log.Printf("Connected to Gauge runner on port %d", *port)
    
    slackConfig := &SlackConfig{
        WebhookURL: os.Getenv("SLACK_WEBHOOK_URL"),
        Channel:    os.Getenv("SLACK_CHANNEL"),
    }
    
    handler := &PluginHandler{slack: slackConfig}
    
    for {
        msg, err := readMessage(conn)
        if err != nil {
            log.Printf("Connection closed: %v", err)
            return
        }
        
        handler.handle(msg, conn)
    }
}

func readMessage(conn net.Conn) (*gm.Message, error) {
    // Gauge messages are length-prefixed: 4-byte big-endian length + protobuf bytes
    var length uint32
    buf := make([]byte, 4)
    if _, err := conn.Read(buf); err != nil {
        return nil, err
    }
    length = uint32(buf[0])<<24 | uint32(buf[1])<<16 | uint32(buf[2])<<8 | uint32(buf[3])
    
    data := make([]byte, length)
    if _, err := conn.Read(data); err != nil {
        return nil, err
    }
    
    msg := &gm.Message{}
    if err := proto.Unmarshal(data, msg); err != nil {
        return nil, fmt.Errorf("failed to unmarshal message: %w", err)
    }
    
    return msg, nil
}

func writeMessage(conn net.Conn, msg *gm.Message) error {
    data, err := proto.Marshal(msg)
    if err != nil {
        return err
    }
    
    length := len(data)
    header := []byte{
        byte(length >> 24),
        byte(length >> 16),
        byte(length >> 8),
        byte(length),
    }
    
    if _, err := conn.Write(append(header, data...)); err != nil {
        return err
    }
    return nil
}

Handling Execution Events

// handler.go
package main

import (
    "log"
    "net"
    
    gm "github.com/getgauge/gauge-proto/go/gauge_messages"
)

type PluginHandler struct {
    slack        *SlackConfig
    failedSpecs  []string
    totalScenarios int
    failedScenarios int
}

func (h *PluginHandler) handle(msg *gm.Message, conn net.Conn) {
    switch msg.MessageType {
    
    case gm.Message_SuiteExecutionResult:
        h.onSuiteResult(msg.SuiteExecutionResult)
    
    case gm.Message_SpecExecutionResult:
        h.onSpecResult(msg.SpecExecutionResult)
    
    case gm.Message_ScenarioExecutionResult:
        h.onScenarioResult(msg.ScenarioExecutionResult)
    
    case gm.Message_KillProcessRequest:
        log.Println("Received kill signal, shutting down")
        return
    }
    
    // Always acknowledge
    ack := &gm.Message{
        MessageType: gm.Message_Empty,
        MessageId:   msg.MessageId,
        Empty:       &gm.Empty{},
    }
    if err := writeMessage(conn, ack); err != nil {
        log.Printf("Failed to send acknowledgement: %v", err)
    }
}

func (h *PluginHandler) onSpecResult(result *gm.SpecExecutionResult) {
    spec := result.SpecResult
    if spec.Failed {
        h.failedSpecs = append(h.failedSpecs, spec.ProtoSpec.FileName)
        log.Printf("FAILED: %s", spec.ProtoSpec.FileName)
    }
}

func (h *PluginHandler) onScenarioResult(result *gm.ScenarioExecutionResult) {
    h.totalScenarios++
    if result.ScenarioResult.Failed {
        h.failedScenarios++
    }
}

func (h *PluginHandler) onSuiteResult(result *gm.SuiteExecutionResult) {
    suiteResult := result.SuiteResult
    
    passed := int(suiteResult.SpecsExecuted) - int(suiteResult.SpecsFailed)
    
    message := &SlackMessage{
        Passed:   passed,
        Failed:   int(suiteResult.SpecsFailed),
        Skipped:  int(suiteResult.SpecsSkipped),
        Duration: suiteResult.ExecutionTime,
        FailedSpecs: h.failedSpecs,
        Success:  !suiteResult.Failed,
    }
    
    if err := h.slack.Send(message); err != nil {
        log.Printf("Failed to send Slack notification: %v", err)
    }
}

Slack Notification Implementation

// slack.go
package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
    "strings"
)

type SlackConfig struct {
    WebhookURL string
    Channel    string
}

type SlackMessage struct {
    Passed      int
    Failed      int
    Skipped     int
    Duration    int64
    FailedSpecs []string
    Success     bool
}

func (s *SlackConfig) Send(msg *SlackMessage) error {
    if s.WebhookURL == "" {
        return fmt.Errorf("SLACK_WEBHOOK_URL not set")
    }
    
    emoji := ":white_check_mark:"
    color := "#36a64f"
    status := "All tests passed"
    
    if !msg.Success {
        emoji = ":x:"
        color = "#e01e5a"
        status = fmt.Sprintf("%d spec(s) failed", msg.Failed)
    }
    
    failedList := ""
    if len(msg.FailedSpecs) > 0 {
        failedList = "\n*Failed specs:*\n• " + strings.Join(msg.FailedSpecs, "\n• ")
    }
    
    durationSec := msg.Duration / 1000
    
    payload := map[string]interface{}{
        "channel": s.Channel,
        "attachments": []map[string]interface{}{
            {
                "color": color,
                "title": fmt.Sprintf("%s Gauge Test Results", emoji),
                "text": fmt.Sprintf(
                    "*Status:* %s\n*Passed:* %d | *Failed:* %d | *Skipped:* %d\n*Duration:* %ds%s",
                    status, msg.Passed, msg.Failed, msg.Skipped, durationSec, failedList,
                ),
                "mrkdwn_in": []string{"text"},
            },
        },
    }
    
    body, err := json.Marshal(payload)
    if err != nil {
        return err
    }
    
    resp, err := http.Post(s.WebhookURL, "application/json", bytes.NewBuffer(body))
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    
    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("Slack returned status %d", resp.StatusCode)
    }
    
    return nil
}

Building and Installing

# Build for current platform
go build -o bin/slack-reporter .

# Cross-compile for Linux
GOOS=linux GOARCH=amd64 go build -o bin/slack-reporter-linux .

# Install into Gauge's plugin directory
GAUGE_PLUGIN_DIR=$(gauge env | grep GAUGE_HOME | cut -d= -f2)/plugins/slack-reporter/1.0.0
mkdir -p "$GAUGE_PLUGIN_DIR"
cp plugin.json "$GAUGE_PLUGIN_DIR/"
cp -r bin/ "$GAUGE_PLUGIN_DIR/"

After installation, verify:

gauge install slack-reporter --from /path/to/slack-reporter
gauge --version  # Should list slack-reporter

Building a Plugin in Python

For teams more comfortable with Python, the same pattern applies using the grpcio and protobuf packages. The Gauge team maintains Python protobuf bindings in the gauge-proto repository.

# main.py
import socket
import struct
import sys
import os
from gauge_messages import messages_pb2 as gm

def read_message(sock):
    header = sock.recv(4)
    if len(header) < 4:
        return None
    length = struct.unpack('>I', header)[0]
    data = b''
    while len(data) < length:
        chunk = sock.recv(length - len(data))
        if not chunk:
            return None
        data += chunk
    msg = gm.Message()
    msg.ParseFromString(data)
    return msg

def write_message(sock, msg):
    data = msg.SerializeToString()
    header = struct.pack('>I', len(data))
    sock.sendall(header + data)

def acknowledge(sock, message_id):
    ack = gm.Message()
    ack.messageType = gm.Message.Empty
    ack.messageId = message_id
    write_message(sock, ack)

def main():
    port = int(sys.argv[sys.argv.index('--port') + 1])
    
    with socket.create_connection(('localhost', port)) as sock:
        print(f"Connected on port {port}", flush=True)
        
        results = {'total': 0, 'failed': 0}
        
        while True:
            msg = read_message(sock)
            if msg is None:
                break
            
            if msg.messageType == gm.Message.ScenarioExecutionResult:
                results['total'] += 1
                if msg.scenarioExecutionResult.scenarioResult.failed:
                    results['failed'] += 1
            
            elif msg.messageType == gm.Message.SuiteExecutionResult:
                # Send to your reporting system
                print(f"Suite complete: {results}", flush=True)
                post_to_dashboard(results)
            
            elif msg.messageType == gm.Message.KillProcessRequest:
                break
            
            acknowledge(sock, msg.messageId)

def post_to_dashboard(results):
    import urllib.request, json
    url = os.environ.get('DASHBOARD_URL')
    if not url:
        return
    data = json.dumps(results).encode()
    req = urllib.request.Request(url, data=data, 
                                  headers={'Content-Type': 'application/json'})
    urllib.request.urlopen(req)

if __name__ == '__main__':
    main()

Using the Gauge Messages API from Step Implementations

Beyond full plugins, the Messages API is also accessible from within step implementations to push data into the current execution context. This is lighter weight than a full plugin.

In Java:

import com.thoughtworks.gauge.Gauge;

@Step("Log the order summary for order <orderId>")
public void logOrderSummary(String orderId) {
    Order order = orderService.getOrder(orderId);
    
    // This text appears in the Gauge HTML report inline with the step
    Gauge.writeMessage("Order ID: %s", orderId);
    Gauge.writeMessage("Total: $%.2f", order.getTotal());
    Gauge.writeMessage("Items: %d", order.getItems().size());
    
    // Capture a table in the report
    StringBuilder table = new StringBuilder("| Item | Price |\n|------|-------|\n");
    for (OrderItem item : order.getItems()) {
        table.append(String.format("| %s | $%.2f |\n", item.getName(), item.getPrice()));
    }
    Gauge.writeMessage(table.toString());
}

In JavaScript (gauge-js):

step('Log the API response for <endpoint>', async (endpoint) => {
    const response = await fetch(`${process.env.API_URL}${endpoint}`);
    const body = await response.json();
    
    gauge.message(`Status: ${response.status}`);
    gauge.message(`Response: ${JSON.stringify(body, null, 2)}`);
    
    if (!response.ok) {
        throw new Error(`${endpoint} returned ${response.status}`);
    }
});

Publishing Your Plugin

If your plugin is useful to others, publish it to the Gauge plugin registry.

  1. Host the plugin: Create a GitHub release with binaries for Linux, macOS, and Windows (both x86_64 and arm64 for macOS).
  2. Create a plugin descriptor: Submit a PR to gauge-repository with your plugin's metadata:
{
  "name": "slack-reporter",
  "description": "Posts Gauge test results to Slack",
  "versions": [
    {
      "version": "1.0.0",
      "gaugeVersion": ">=1.0.0",
      "url": "https://github.com/yourorg/gauge-slack-reporter/releases/download/v1.0.0/slack-reporter-{os}-{arch}.zip",
      "zipMd5": "..."
    }
  ]
}
  1. Document the environment variables: Users need to know what to set (SLACK_WEBHOOK_URL, SLACK_CHANNEL, etc.).

Once merged, users can install with:

gauge install slack-reporter

Debugging Your Plugin

Gauge starts plugins with stdout/stderr connected to the terminal. Use logging liberally during development:

log.Printf("[slack-reporter] Received %s message", msg.MessageType)

To test your plugin in isolation without a full Gauge run, write a small test harness that opens a socket, sends sample protobuf messages, and checks the plugin's response:

// test-harness/main.go
func main() {
    // Start listening for plugin connection
    listener, _ := net.Listen("tcp", ":0")
    port := listener.Addr().(*net.TCPAddr).Port
    
    // Start plugin in a subprocess
    cmd := exec.Command("./bin/slack-reporter", "--port", strconv.Itoa(port))
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    cmd.Start()
    
    conn, _ := listener.Accept()
    
    // Send a mock SuiteExecutionResult
    msg := &gm.Message{
        MessageType: gm.Message_SuiteExecutionResult,
        MessageId:   1,
        SuiteExecutionResult: &gm.SuiteExecutionResult{
            SuiteResult: &gm.ProtoSuiteResult{
                SpecsExecuted: 10,
                SpecsFailed:   2,
                ExecutionTime: 45000,
                Failed:        true,
            },
        },
    }
    writeMessage(conn, msg)
    
    // Wait for acknowledgement
    ack, _ := readMessage(conn)
    fmt.Printf("Got ack: %v\n", ack.MessageType)
}

Summary

Gauge's plugin system is a well-defined socket-based protocol using Protocol Buffers. The barrier to building a useful plugin is lower than it appears: you need to handle a small set of message types (spec result, scenario result, suite result, kill), respond with acknowledgements, and do something useful with the data. Reporting plugins that push results to Slack, databases, or test management tools are the most practical starting point. Start with the Go implementation for the closest alignment with the Gauge ecosystem, or use Python if that fits your team better.

Read more

Start now free