Tsung vs JMeter: Choosing the Right Load Testing Tool

Tsung vs JMeter: Choosing the Right Load Testing Tool

JMeter and Tsung are both open-source load testing tools, both mature, both capable of generating significant concurrent load. But they take fundamentally different approaches — and the differences matter when you're trying to push high concurrency on limited hardware.

This comparison focuses on the practical trade-offs for teams choosing between them.

Quick Summary

Dimension Tsung JMeter
Language Erlang Java
Config format XML XML (GUI-generated)
GUI Minimal Full-featured IDE
Concurrency model Erlang actors Java threads
Resource efficiency Very high Moderate
Max users (single node) 200k–500k 10k–50k
Distributed mode Built-in Built-in (more complex)
Protocols HTTP, WS, XMPP, DB HTTP, WS, JMS, many more
Plugins Few Hundreds
Learning curve Steep (XML-only) Moderate (GUI helps)
Reporting Basic HTML Extensive, plugin-based
Community Small Very large

Concurrency and Resource Efficiency

This is where Tsung has its clearest advantage.

JMeter uses one OS thread per virtual user. At 1,000 concurrent users, JMeter is running 1,000 threads. At 10,000 users, it's running 10,000 threads. Thread context switching overhead is significant, and memory consumption scales linearly — each Java thread typically uses 512KB–1MB of stack space.

Tsung uses Erlang lightweight processes (not OS threads). An Erlang process uses roughly 2KB of memory at startup. Tsung can run 100,000+ concurrent virtual users on a single node without the context switching overhead of JMeter's thread-per-user model.

Practical benchmark reference:

A typical c5.2xlarge AWS instance (8 vCPU, 16GB RAM):

  • JMeter: ~5,000–10,000 concurrent users before degradation
  • Tsung: ~50,000–100,000 concurrent users on the same hardware

If you need to generate high concurrency cheaply, Tsung wins. If your tests stay under 5,000 concurrent users, the difference doesn't matter much.

Configuration and Scripting

Both tools use XML as their primary configuration format, but the authoring experience is very different.

JMeter has a visual GUI that makes test creation accessible to non-programmers:

  • Drag and drop components
  • Right-click to add assertions and extractors
  • Live result viewers during test execution
  • Generates the XML file you edit or run

Tsung has no meaningful GUI (there's a basic web monitor, but no test editor). All configuration is hand-written XML:

<!-- JMeter: complex element structures but GUI-generated -->
<!-- Tsung: hand-written but more readable -->

<session name="api_test" probability="100" type="ts_http">
  <request>
    <http url="/api/users" method="GET" version="1.1"/>
  </request>
  <thinktime value="1"/>
</session>

Tsung's XML is arguably more readable than JMeter's when written by hand, but JMeter's GUI makes it accessible to QA engineers who don't want to hand-author XML.

Distributed Mode

Both tools support distributed load generation across multiple nodes.

Tsung uses Erlang clustering, which is built into the language. Setup requires matching Erlang cookies and SSH access — once configured, it's seamless:

<clients>
  <client host="injector1.example.com" cpu="8" maxusers="100000"/>
  <client host="injector2.example.com" cpu="8" maxusers="100000"/>
</clients>

JMeter uses a controller/worker model with RMI. It works but has more moving parts:

  • Start jmeter-server on each worker node
  • Configure RMI ports and SSL certificates
  • More firewall configuration
  • Workers must have the same JMeter version and plugins

For small distributed setups (2–5 nodes), both are manageable. For larger clusters, Tsung's Erlang clustering tends to be more reliable.

Protocol Support

JMeter supports far more protocols out of the box and via plugins:

  • HTTP/HTTPS
  • WebSockets (via plugins)
  • FTP
  • JDBC (database load testing)
  • JMS (message queues)
  • LDAP
  • SMTP/POP3
  • TCP generic
  • Native mobile app testing (via plugins)

Tsung has built-in support for:

  • HTTP/HTTPS
  • WebSockets
  • XMPP (Jabber)
  • PostgreSQL
  • MySQL
  • LDAP

If you need to load test a JMS queue or an FTP server, JMeter has the plugin for it. Tsung doesn't.

Reporting

JMeter has extensive reporting options:

  • Built-in HTML dashboard report (jmeter -g results.jtl -o dashboard/)
  • Listeners during test: active charts, response viewers, summary tables
  • Plugin-based: InfluxDB exporter for Grafana, Kibana integration
  • Aggregate reports, transaction response times, error detail

Tsung has basic built-in reporting:

  • HTML report via tsung_stats.pl
  • Charts: RPS, response time, concurrent users, errors
  • No real-time dashboard during test (just the minimal web monitor)
  • No built-in plugin ecosystem for external monitoring

For teams that want detailed reporting and real-time visibility, JMeter's ecosystem is richer. For teams that need raw throughput numbers and a simple HTML output, Tsung's reporting is sufficient.

Scripting and Dynamic Scenarios

JMeter supports:

  • BeanShell scripting (Java)
  • Groovy scripting (preferred, faster)
  • JSR223 elements anywhere in the test plan
  • Pre/Post Processors for request manipulation

Tsung supports:

  • Erlang functions for dynamic data
  • dyn_variable with regex extraction
  • CSV data injection
  • Random number generation via built-in functions

For complex scenarios with conditional logic, JMeter's Groovy scripting is significantly more capable. Tsung's dynamic variables cover the common cases but struggle with complex response parsing.

Example: conditional flow in JMeter with Groovy:

// JMeter JSR223 Sampler
def response = prev.getResponseDataAsString()
def json = new groovy.json.JsonSlurper().parseText(response)

if (json.status == 'pending') {
    // Set variable for next request
    vars.put('endpoint', '/api/poll')
} else {
    vars.put('endpoint', '/api/result')
}

Equivalent in Tsung requires Erlang or creative XML workarounds.

Scenario Recording

JMeter has a built-in HTTP(S) proxy recorder:

  1. Start the recorder
  2. Configure your browser to use JMeter as a proxy
  3. Browse your application
  4. JMeter captures all requests as a test plan

Tsung has no built-in recorder. You write scenarios by hand or use external tools to capture and convert HAR files.

For quickly building tests against existing applications, JMeter's recorder is a significant time-saver.

When to Choose Tsung

  • Very high concurrency on limited hardware: You need 100k+ concurrent users without paying for many JMeter nodes
  • Long-running sustained tests: Tsung's low memory footprint makes it suitable for multi-hour tests
  • Simple HTTP scenarios: Your test is GET/POST requests without complex conditional logic
  • Cost-sensitive environments: Fewer injector machines needed for equivalent load

When to Choose JMeter

  • Team includes non-programmers: The GUI makes JMeter accessible to QA engineers without coding experience
  • Complex scenarios with conditional logic: Groovy scripting handles branching, dynamic calculations, complex extractions
  • Many protocols: Testing JDBC, JMS, FTP, or other non-HTTP protocols
  • Rich reporting requirements: Built-in dashboard, Grafana integration, Kibana dashboards
  • Existing JMeter investment: Test plans already written, team expertise already in JMeter
  • Short test runs with moderate concurrency: Under 10k users, the resource efficiency advantage disappears

Alternatives Worth Considering

Before committing to either, it's worth knowing the current landscape:

  • k6: JavaScript-based, very popular, cloud hosting via Grafana k6 Cloud, better than both for developer-friendly workflows
  • Gatling: Scala DSL, code-first, better reporting than Tsung, good Scala team fit
  • Locust: Python, developer-friendly, easy scenario logic, moderate concurrency ceiling
  • Artillery: Node.js/YAML, easy to get started, cloud execution option

For new projects in 2024, k6 or Gatling often make more sense than either Tsung or JMeter unless you have specific reasons to choose them.

Summary

Tsung wins on raw concurrency efficiency. JMeter wins on features, ecosystem, and accessibility.

If you're load testing at very high concurrency (>20k users) and want to minimize hardware costs, Tsung is worth the configuration overhead.

If your team includes QA engineers who aren't comfortable writing XML by hand, or if you need complex scenario logic, protocol variety, or rich reporting, JMeter's feature set is worth the resource overhead.

For most teams starting fresh, k6 or Gatling are worth evaluating before either Tsung or JMeter — both have modernized APIs and better developer experience.

Read more

Start now free