Stability is a key metric in app store rankings and user retention. For a startup, maintaining low crash rates is essential. In this post, we’ll look at how we implemented Firebase Robo Tests in our automated CI/CD pipeline to uncover critical bugs before shipping them to production.
The Challenge
Manual testing is slow, expensive, and error-prone. In our Flutter application, asynchronous operations (like processing real-time geo-location logs and loading complex 3D GLTF models) created subtle race conditions that were hard to reproduce manually. We needed a system that could explore the entire app graph automatically on real devices and report uncaught exceptions.
Enter Firebase Test Lab & Robo Tests
Firebase Robo Tests use a crawler that automatically navigates your application by analyzing its user interface structure. It taps buttons, fills forms, swipes screens, and records screenshot flows alongside device logcats.
Because Flutter renders UI on its own canvas (skia/impeller) rather than using native Android UI controls, classic test crawlers sometimes struggle to find interactive widgets. However, by enabling the accessibility labels in Flutter, we can expose semantic nodes directly to Android's accessibility framework, which Robo uses to index the screen.
CI/CD Pipeline Integration
We integrated the test suite directly into our GitHub Actions workflow:
name: Test Lab Execution
on:
push:
branches: [ main ]
jobs:
test-lab:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
- name: Install Flutter
uses: subosito/flutter-action@v2
with:
flutter-version: '3.19.x'
- name: Build Android APK
run: flutter build apk --debug
- name: Authenticate with GCP
uses: google-github-actions/auth@v1
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
- name: Run Firebase Robo Test
run: |
gcloud firebase test android run \
--type robo \
--app build/app/outputs/flutter-apk/app-debug.apk \
--device model=redfin,version=30,locale=en,orientation=portrait \
--timeout 180s
Creating Custom Robo Scripts
To test authenticated flows, we configured a robo.config file to automatically log in our crawler. We supplied it with credentials and defined specific element resource-names for text entry:
[
{
"eventType": "VIEW_TEXT_CHANGED",
"replacementText": "test_crawler@omnee.ca",
"elementDescriptors": [
{
"resourceId": "email_input_field"
}
]
},
{
"eventType": "VIEW_TEXT_CHANGED",
"replacementText": "password123",
"elementDescriptors": [
{
"resourceId": "password_input_field"
}
]
},
{
"eventType": "VIEW_CLICKED",
"elementDescriptors": [
{
"resourceId": "sign_in_button"
}
]
}
]
Results
Integrating automated Robo runs on every push resulted in:
- A 60% reduction in production crashes within the first month.
- Automatic detection of null pointer exceptions during model loads.
- Full UI logs, videos, and stack traces accessible directly from our pull requests.