Advanced TestProject Features: AI-Powered Test Creation and Addons
TestProject's basic recorder and dashboard were entry points. The platform's advanced capabilities — self-healing element detection, the addon development SDK, OpenSDK internals, and test data strategies — are what made it genuinely production-grade for teams willing to go deeper. This guide covers those advanced features in detail.
How TestProject's Self-Healing AI Worked
TestProject marketed "AI-powered" self-healing, but the mechanism was deterministic rather than neural. When the recorder captured an element interaction, it stored a ranked list of locator strategies derived from the element's attributes:
{
"locators": [
{ "type": "id", "value": "submit-btn", "score": 100 },
{ "type": "css", "value": ".checkout-form button[type='submit']", "score": 85 },
{ "type": "xpath", "value": "//button[contains(@class,'btn-primary') and text()='Place Order']", "score": 70 },
{ "type": "xpath", "value": "//form[@id='checkout']//button[last()]", "score": 50 }
]
}During execution, the agent tried locators in score order. If the first failed (element not found), it tried the next. When a lower-scored locator succeeded, the agent logged a "self-heal" event and — if auto-correction was enabled — prompted you to update the primary locator in the dashboard.
The "AI" was in the initial scoring: the recorder analyzed element attributes, ARIA labels, visual position, and DOM context to assign confidence scores. This was heuristic-based, not learned from test outcomes.
Practical implication: Self-healing caught attribute changes well (an ID renamed, a class changed) but failed on structural changes (an entire component rebuilt with different DOM structure). Teams that understood this used it appropriately — as a buffer against minor UI changes, not a replacement for maintaining tests.
The OpenSDK Internals
The OpenSDK was open source (MIT licensed). Understanding its internal architecture explains both its power and its constraints.
The core was a ProxyDriver that wrapped the standard WebDriver:
# Simplified OpenSDK internals
class ChromeDriver(webdriver.Chrome):
def __init__(self, token=None, project_name=None, job_name=None, **kwargs):
# Start local proxy on random port
self._proxy = AgentProxy(token=token)
self._proxy.start()
# Configure Chrome to route through proxy
options = kwargs.get('options', ChromeOptions())
options.add_argument(f'--proxy-server=localhost:{self._proxy.port}')
super().__init__(options=options, **kwargs)
def find_element(self, by, value):
# Wrap with reporting
self._proxy.log_step(f"Find element: {by}={value}")
try:
element = super().find_element(by, value)
self._proxy.log_step_result(passed=True, screenshot=self._screenshot())
return ElementProxy(element, self._proxy)
except NoSuchElementException:
# Try alternative locators
for locator in self._proxy.get_alternative_locators(by, value):
try:
element = super().find_element(*locator)
self._proxy.log_heal_event(original=(by, value), used=locator)
return ElementProxy(element, self._proxy)
except NoSuchElementException:
continue
self._proxy.log_step_result(passed=False)
raiseThe AgentProxy was a local HTTP server that:
- Forwarded WebDriver commands to the real browser driver
- Intercepted results for reporting
- Communicated with the TestProject cloud service for report upload and locator alternatives
This architecture meant the OpenSDK required the TestProject agent to be running locally — you couldn't use the SDK without the agent, even for purely local execution without cloud reporting.
Building Custom Addons
TestProject's addon SDK let you package reusable test actions as Java classes and publish them to the marketplace. Custom addons were the correct solution for:
- Organization-specific test patterns (logging into your internal SSO, navigating your custom component library)
- Third-party service interactions (querying your database, calling internal APIs)
- Data transformations (parsing response bodies, generating test data)
A minimal addon looked like this:
// Java addon: waits for a specific API response before proceeding
@Addon(name = "Wait For API Response", description = "Polls an endpoint until expected response")
public class WaitForApiResponseAddon implements WebAction {
@Parameter(description = "API endpoint URL")
public String url;
@Parameter(description = "Expected response substring")
public String expectedContent;
@Parameter(description = "Maximum wait seconds", defaultValue = "30")
public int timeoutSeconds;
@Override
public ExecutionResult execute(AddonHelper helper) {
long deadline = System.currentTimeMillis() + (timeoutSeconds * 1000L);
while (System.currentTimeMillis() < deadline) {
try {
String response = HttpClient.get(url);
if (response.contains(expectedContent)) {
helper.getReporter().result("Found expected content in response");
return ExecutionResult.PASSED;
}
Thread.sleep(2000);
} catch (Exception e) {
helper.getReporter().result("Request failed: " + e.getMessage());
}
}
helper.getReporter().result("Timeout: expected content not found after " + timeoutSeconds + "s");
return ExecutionResult.FAILED;
}
}The addon was packaged as a JAR and uploaded to TestProject's marketplace. Once uploaded, it appeared in the recorder's action list and in the OpenSDK's addon proxy — callable from both no-code tests and SDK tests.
Test Data Management Strategies
TestProject had no built-in test data management, but it supported parameterized tests through data-driven execution. A job could be configured with a CSV data source, and each row became a separate test run:
# SDK approach to data-driven testing
import csv
from src.testproject.sdk.drivers import webdriver
with open('test_data.csv') as f:
reader = csv.DictReader(f)
for row in reader:
driver = webdriver.Chrome()
try:
driver.get("https://app.example.com/login")
driver.find_element(By.ID, "email").send_keys(row['email'])
driver.find_element(By.ID, "password").send_keys(row['password'])
driver.find_element(By.ID, "submit").click()
expected = row['expected_outcome']
actual = driver.find_element(By.CSS_SELECTOR, ".dashboard-title").text
assert actual == expected, f"Expected {expected}, got {actual}"
finally:
driver.quit()The dashboard showed separate execution records per data row, making it easy to identify which test data combination failed.
Dynamic Element Strategies in the OpenSDK
For highly dynamic UIs where even multiple locators weren't reliable, the SDK supported custom element resolution:
from src.testproject.sdk.drivers import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
# Wait for element by text content — more stable than attribute-based locators
def element_by_text(text):
return EC.presence_of_element_located(
(By.XPATH, f"//*[normalize-space(text())='{text}']")
)
# Wait for element by ARIA role + label — most semantic/stable
def accessible_button(label):
return EC.element_to_be_clickable(
(By.CSS_SELECTOR, f"[role='button'][aria-label='{label}']")
)
submit_btn = wait.until(accessible_button("Submit order"))
submit_btn.click()This pattern worked within the OpenSDK the same way it worked in raw Selenium — the proxy captured the interaction regardless of how you located the element.
Test Suite Organization and Parallel Execution
On the free plan, test jobs ran sequentially. Advanced users structured their job configurations to maximize the value of sequential execution:
- Smoke job: 5-10 critical path tests, ran on every CI trigger
- Regression job: Full test suite, ran nightly
- Feature job: Tests scoped to a specific feature area, ran on relevant PRs
The dashboard let you set up dependencies between jobs — a failing smoke job could block regression from starting, saving CI minutes.
For parallel execution without paying for TestProject's cloud, teams ran multiple agents on separate machines, each configured with different aliases, and triggered jobs targeting specific agents. This required orchestration external to TestProject but was functional.
The AI Test Generation Feature
TestProject's AI test generation (distinct from self-healing) was a late-platform feature that analyzed your application's DOM and suggested test steps:
- You navigated to a URL in the recorder
- The AI scanned the page structure, identified interactive elements, and proposed a test flow
- You accepted, rejected, or modified the suggestions
The generated tests were conservative — they covered the most obvious interactions (form fill, button click, navigation) without semantic understanding of what the actions meant. The output quality depended heavily on how semantically the application's HTML was written (ARIA labels, meaningful IDs, descriptive text content).
In practice, the AI suggestions saved 20-40% of recording time on well-structured applications and provided little value on applications with generated attribute names (id="ng-binding-47").
What These Features Look Like in 2024 Alternatives
The self-healing and AI generation features that TestProject pioneered are now standard across the market:
- Playwright: Auto-waiting eliminates the need for most self-healing (elements are waited for automatically)
- Modern platforms: AI test generation from natural language descriptions has superseded recorder-based generation
HelpMeTest takes the AI generation concept to its logical conclusion — instead of recording interactions and having AI suggest steps, you write the test intent in plain English ("Log in as admin and verify the dashboard shows recent orders") and the platform executes it. No locators, no self-healing needed, no DOM analysis required.
The tradeoff: you lose the fine-grained control of specifying exact locator strategies, which matters for testing interactions that depend on specific DOM structure. For functional correctness testing where intent matters more than implementation, natural language is faster to write and maintain.
Key takeaways:
- TestProject's self-healing was heuristic-based locator fallback, not learned AI
- The addon SDK (Java) let teams package reusable actions deployable to both no-code and SDK tests
- Data-driven testing worked via CSV data sources configured at the job level
- Free parallel execution required running multiple agents on separate machines
- Modern alternatives have absorbed these patterns — Playwright eliminated most self-healing need; HelpMeTest eliminates locators entirely