Getting Started with BitBar: From Zero to First Test

Getting from "I have a BitBar account" to "I have test results from a real device" involves more steps than most getting-started guides show. This walkthrough covers each step in order, including the parts that typically trip up first-time users.

Before You Start

BitBar requires a few things before you can run any tests:

1. A BitBar account — BitBar is available through SmartBear's sales process. There's a free trial for evaluation.
2. An API key — found in your BitBar account settings under "My Account" > "API Access"
3. An app to test — an APK for Android testing or IPA for iOS testing
4. Test code — an Appium test suite, Espresso tests, or XCUITest tests

This guide uses Appium with Java for Android, as it's the most common starting path.

Account Setup

After logging into BitBar, the first thing to collect is your API key.

Navigate to: My Account (top right avatar) > API Access

Your API key is a long alphanumeric string. Store it as an environment variable — never hardcode it in source files:

export BITBAR_API_KEY="your-api-key-here"

Add this to your shell profile (.bashrc, .zshrc, or .fish config) to persist it across sessions.

Understanding BitBar's Concepts

Before uploading anything, understand the organizational concepts:

Files: APKs, IPAs, and test APKs you upload to BitBar's storage. Every run references one or more files.

Projects: Logical groupings of test runs. Create one project per application you're testing.

Device Groups: Collections of devices, either public (shared with all customers) or private (reserved for your account). Your subscription determines which device groups you have access to.

Test Runs: Execution of a test suite against a device group. Each run produces results per device, with videos, logs, and screenshots.

Frameworks: BitBar needs to know which test framework your tests use — Appium, Espresso, XCUITest, etc. — to configure the execution environment correctly.

Uploading Your App

BitBar requires your app to be uploaded to their file storage before a test run can reference it.

Via the dashboard: Go to My Account > My Files > Upload file. Drag your APK there.

Via API:

FILE_RESPONSE=$(curl -s -X POST \
  -H "Authorization: Bearer $BITBAR_API_KEY" \
  -F "file=@/path/to/your-app.apk" \
  https://cloud.bitbar.com/api/me/files)

FILE_ID=$(echo $FILE_RESPONSE | python3 -c "import sys, json; print(json.load(sys.stdin)['id'])")
echo "Uploaded file ID: $FILE_ID"

Note the file ID — you'll need it when creating a test run.

For Espresso tests: You need two files — the app APK and the test APK. Upload both and note both IDs.

# Upload main app
APP_FILE_ID=$(curl -s -X POST \
  -H "Authorization: Bearer $BITBAR_API_KEY" \
  -F "file=@app/build/outputs/apk/debug/app-debug.apk" \
  https://cloud.bitbar.com/api/me/files | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")

# Upload test APK
TEST_FILE_ID=$(curl -s -X POST \
  -H "Authorization: Bearer $BITBAR_API_KEY" \
  -F "file=@app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk" \
  https://cloud.bitbar.com/api/me/files | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")

echo "App ID: $APP_FILE_ID, Test ID: $TEST_FILE_ID"

Setting Up an Appium Project

Create a project in BitBar to organize your test runs.

PROJECT_RESPONSE=$(curl -s -X POST \
  -H "Authorization: Bearer $BITBAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "My Android App", "type": "APPIUM_ANDROID_SERVER"}' \
  https://cloud.bitbar.com/api/me/projects)

PROJECT_ID=$(echo $PROJECT_RESPONSE | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
echo "Project ID: $PROJECT_ID"

Project types include:

• APPIUM_ANDROID_SERVER — Appium for Android
• APPIUM_IOS_SERVER — Appium for iOS
• ANDROID — Espresso
• IOS — XCUITest

Finding Available Devices

Before triggering a run, check which devices are available to your account:

curl -s \
  -H "Authorization: Bearer $BITBAR_API_KEY" \
  "https://cloud.bitbar.com/api/me/devices?limit=20&sort=displayName+asc" \
  | python3 -c "
import sys, json
devices = json.load(sys.stdin)['data']
for d in devices:
    print(f\"{d['id']:6} | {d['displayName']:40} | {d['osType']} {d['softwareVersion']['releaseVersion']}\")
"

Note device IDs for the devices you want to target. For your first run, pick one specific device rather than a group — it simplifies debugging if something goes wrong.

Writing Your First Appium Test

Here's a minimal Appium test that works with BitBar. It installs your app and verifies the main screen loads.

Maven dependency setup (pom.xml):

<dependencies>
    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.6.0</version>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>7.8.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Test class:

import io.appium.java_client.android.AndroidDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.*;
import java.net.URL;

public class BitBarFirstTest {
    
    private AndroidDriver driver;
    private static final String API_KEY = System.getenv("BITBAR_API_KEY");
    private static final String BITBAR_URL = "https://appium.bitbar.com/wd/hub";
    
    @BeforeClass
    public void setUp() throws Exception {
        DesiredCapabilities caps = new DesiredCapabilities();
        
        // BitBar credentials and configuration
        caps.setCapability("bitbar_apiKey", API_KEY);
        caps.setCapability("bitbar_device", "Samsung Galaxy A52");  // Specific device name
        caps.setCapability("bitbar_project", "My Android App");
        caps.setCapability("bitbar_testrun", "First Test Run");
        
        // App configuration
        caps.setCapability("bitbar_app", FILE_ID);  // The file ID from upload step
        
        // Appium capabilities
        caps.setCapability("platformName", "Android");
        caps.setCapability("automationName", "UiAutomator2");
        caps.setCapability("appActivity", "com.yourapp.MainActivity");
        caps.setCapability("appPackage", "com.yourapp");
        caps.setCapability("newCommandTimeout", 90);
        
        driver = new AndroidDriver(new URL(BITBAR_URL), caps);
    }
    
    @Test
    public void mainScreenLoads() {
        // Replace with actual element from your app
        boolean mainScreenVisible = driver
            .findElement(By.id("com.yourapp:id/main_container"))
            .isDisplayed();
        
        Assert.assertTrue(mainScreenVisible, "Main screen should be visible after launch");
    }
    
    @Test
    public void loginFlowWorks() {
        driver.findElement(By.id("com.yourapp:id/email_field"))
              .sendKeys("test@example.com");
        driver.findElement(By.id("com.yourapp:id/password_field"))
              .sendKeys("testpassword");
        driver.findElement(By.id("com.yourapp:id/login_button"))
              .click();
        
        // Wait for dashboard
        new WebDriverWait(driver, Duration.ofSeconds(10))
            .until(ExpectedConditions.visibilityOfElementLocated(
                By.id("com.yourapp:id/dashboard_view")));
        
        Assert.assertTrue(
            driver.findElement(By.id("com.yourapp:id/dashboard_view")).isDisplayed(),
            "Dashboard should appear after successful login"
        );
    }
    
    @AfterClass
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Running the Test

Run against BitBar with Maven:

BITBAR_API_KEY=$BITBAR_API_KEY mvn test -Dtest=BitBarFirstTest

The test will:

1. Connect to BitBar's Appium endpoint
2. BitBar allocates a physical Samsung Galaxy A52
3. Your app is installed on the device
4. Test commands execute
5. Results, video, and logs are available in your BitBar dashboard

Understanding Your First Results

After the run completes, the BitBar dashboard shows:

Test Run Overview:

• Pass/fail status
• Device used
• Duration
• Screenshot count

Device Session Details:

• Full video recording of the test session
• Device logs (logcat for Android)
• Appium server logs
• Screenshots at each step

Common first-run failures and what they mean:

Interactive Device Access for Debugging

When automated tests fail and you need to understand why, BitBar's live testing feature lets you interact with a real device in your browser.

Go to Live Testing > select a device > click Start Session. You'll see a browser-based view of the actual device with touch input support. This is invaluable for:

• Verifying element selectors before writing tests
• Reproducing failures interactively
• Understanding device-specific rendering issues

Next Steps After Your First Test

Add more devices: Once your test runs reliably on one device, add more to your run. Create a device group with a few Android versions and manufacturers.

Add to CI: Trigger BitBar runs from your CI pipeline rather than locally. See the CI/CD integration guide for Jenkins and GitHub Actions examples.

Build up the test suite: Start with smoke tests covering critical flows — login, core feature, checkout. Add depth over time.

Set up notifications: BitBar can send webhook notifications when runs complete. Wire these to Slack or your team's notification system so failures are visible immediately.

What This Setup Costs

BitBar pricing is not public, but for planning:

• Free trials are available for evaluation
• Production pricing is enterprise-tier, quote-based
• Cost scales with parallel sessions and device-minutes

For a team running a test suite of 50 tests against 5 devices once per day, model approximately 2-4 hours of device-time daily. Get a quote from SmartBear sales before committing budget.

If cost is a concern at this stage, AWS Device Farm's per-minute pricing ($0.17/device-minute) may be more predictable for low-volume usage, with no upfront commitment.

Conclusion

BitBar's first-test experience involves more setup than testing tools with local device support, but the pattern is consistent: upload your app, configure Appium capabilities, run tests, review results in the dashboard. Once the first run works, the path to a real CI-integrated device testing suite is incremental from there. Each step — more devices, CI integration, parallel execution — builds on the same foundation.