UI testing in Android has always been controversial for many reasons. Tests are slow because they must run in emulators or real devices. They can be flaky for many different reasons. Test code is usually hard to read and maintain because of Espresso and Android APIs. UI testing is a pain in the ass, and it will continue to be, I’m sorry.

But still, in my experience, these kinds of tests are the most useful ones for Android development. While unit tests help us during development, UI tests give us peace of mind during releases, knowing that the most important features of the app won't just stop working for everybody.

In this post, I want to share some of the work we’ve been doing at InfoJobs the past few years to make UI testing less painful.

Our main efforts went to

• Reduce flakiness
• Make tests more semantic
• Write simpler and maintainable tests

Mandatory disclaimer: There are many approaches you can take to UI testing, and most of them are right. I’m not going to tell you how you should do it. I’m going to show what has worked for us and how we do it, so you can hopefully take something from it.

Beginning with UI tests and Espresso

Espresso is Google's official UI testing framework. It can perform actions and check conditions on views. So a really simple UI test could look like this:

@Test
fun mySimpleTest() {
  onView(withId(R.id.greet_button)).perform(click())
  onView(withText("Hello Steve!")).check(matches(isDisplayed()))
}

But in our day-to-day, we want to cover more complex features and flows involving multiple screens. So a real test could become something like:

@Test
fun shouldShowRecommendedOffers_whenComingToHomeBackAfterLogin() {
  // Perform search
  onView(withId(R.id.toolbar)).perform(click())
  onView(withId(R.id.keyword_field)).perform(replaceText("Java"))
  onView(withId(R.id.search_button)).perform(click())
  // Log in
  onView(withId(R.id.searchResultLoginButton)).perform(click())
  onView(withId(R.id.loginEmailField)).perform(replaceText("email@example.com"))
  onView(withId(R.id.loginPasswordField)).perform(replaceText("password"))
  onView(withId(R.id.loginButton)).perform(click())

  // Go back
  onView(isRoot()).perform(pressBack());

  // Assert header text
  onView(withText("10 ofertas recomendadas")).check(matches(isDisplayed()))
}

It’s not the prettiest code, but it works. This is our starting point. Let me tell you what we did to improve it from here.

Barista

Espresso is ugly

A bit of history. When we started using Espresso to write our UI tests years ago we had to learn its API. And it wasn’t pretty. Don’t get me wrong, it’s very powerful and flexible. We can do almost anything to views. But it’s not pretty. The code above is barely readable.

Tip: Check out this awesome cheat sheet in the official docs to understand how the Espresso API works.

At the time, a couple of backend engineers were learning Android development and working on some features for the app. They wanted to cover the features with UI tests, but having to learn and remember the syntax was a problem. So they suggested adding static methods for common actions that they used over and over again. Like clickOn(R.id.button) and assertDisplayed(“Some text”).

I must confess that I didn’t agree with this idea at the time. I believed that we should get used to the Espresso API so we could understand how it works and do more complex things when needed. Luckily my friend Roc was also on the Android Team and convinced me to play along. So we tried it for a while. And oh boy, was I wrong. It was a great decision, it improved a lot the readability of our tests.

Espresso is dumb

Ok, maybe “dumb” is unfair. Let’s say it’s “strict”. For example, we would often need to interact with elements inside scrolls. Espresso fails if you perform an action on a View below a scroll. So we need to tell it to scroll first.

fun click(@IdRes id: Int){
  onView(withId(id)).perform(scrollTo(), click())
}

But there is a catch! Espresso will fail if the view is not inside a ScrollView. Bummer!

From the point of view of the test, I don’t care if the view is inside a scroll or not, I just want the test to click it as if it was an user using the app. So we came up with a trick:

fun click(@IdRes id: Int){
  try {
    onView(withId(id)).perform(click())
  } catch (e: PerformException) {
    onView(withId(id)).perform(scrollTo(), click())
  }
}

Another common example is having a ViewPager with the same layout on multiple pages.

A simple click() action will fail with

androidx.test.espresso.AmbiguousViewMatcherException: 'with id: id/button' matches multiple views in the hierarchy.

As you see, Espresso is strict. If it gets confused and doesn't know what to do, it will fail. It forces you to be more explicit about your actions. There's nothing wrong with that, it's a valid approach. But we were comfortable assuming that, if there are multiple matching views, we only want to click the one that is displayed:

fun click(@IdRes id: Int){
  // I omitted the try/catch from the previous example for brevity
    onView(allOf(isDisplayed(), withId(id))).perform(click())
}

This kind of code could be considered a bad practice in production code. But in testing code we found it makes our tests simpler and more robust since they would not depend on variable things like the screen size or future layout changes. It was totally worth it!

With this same philosophy we created a class named EspressoUtils, with many more similar static methods. It kept growing as we added new utilities for new scenarios we wanted to test. Over time we noticed that even long tests were much more readable using our custom methods than the Espresso API, but more importantly, easier and faster to write.

We added click(id), click(stringRes), openDrawer(), acceptDialog(), assertDisplayed("String"), and many more. Writing new UI tests with EspressoUtils was a delight. We would copy and paste this file internally from project to project until we realized we should share it with everyone as a library. So we moved it to an independent project, renamed it to Barista (great name Sergi!), designed a beautiful logo (thanks Diego!), added tests for all the interactions and assertions (thanks Roc for so many hours!) and published it. Barista was born!

Introducing Barista

Using Barista we removed a lot of the boilerplate from Espresso, making the code easier to read and write. Time has passed and Barista has grown a lot. Today it contains many more interactions and assertions. By making acceptable assumptions it is smarter, it can decide to scroll the screen when needed, it can deal with ViewPagers that contain similar layouts, it interacts with ListView and RecyclerView indistinctively, it allows making custom assertions with a couple of lines thanks to assertAny, it contains some useful Test Rules to fight flaky tests like cleaning Shared Preferences and Databases between tests, and more. Take a look at the Readme, you will be surprised by how much stuff it can do with little code.

androidTestImplementation('com.adevinta.android:barista:4.2.0') {
  exclude group: 'org.jetbrains.kotlin' // Only if you already use Kotlin in your project
}

repositories {
    google()
}

We are super proud to have reached 1.2K stars in GitHub and 1 million downloads in Bintray. And it makes us especially happy seeing a lot of people in the community using, contributing and enjoying it.

But the story doesn't end here. Keep reading to see what else we did to improve our tests.

Page Objects

Barista was just the tip of the iceberg. It’s not just a useful library. It changed our approach to UI testing, it gave us the mindset of writing simpler and easier to read tests. Page Objects are a step in a similar direction, they make test code more semantic and easier to write.

The idea of using Page Objects started many years ago. QA engineers and web developers were used to a concept named “Fragment”. Basically, they were Java abstractions over the actions and assertions that could be done on each page from the perspective of the user (yea, not those fragments). When some backend engineers started working on the Android app they imported those ideas to our shiny Espresso tests. We later learned (thanks to Pedro) that this pattern was already a thing called Page Object. Great! It wasn’t a bad idea after all! 😃

(image from https://martinfowler.com/bliki/PageObject.html)

Our implementation of these Page Objects has evolved over time. We now call them Screen Objects, but that's not relevant. After many iterations and agreements among our Android Team, the result looks like this:

@Test
fun shouldShowRecommendedOffers_whenComingBackAfterLogin() {
  HomeListScreenObject()
    .clickSearchToolbar()
    .search("Java")
    .clickFloatingLoginButton()
    .fillCredentialsAndLogIn()
    .assertThat { headerContains("10 ofertas recomendadas") }
}

class HomeListScreenObject : NavigationScreenObject() {

  fun clickSearchToolbar(): SearchScreenObject {
    clickOn(R.id.toolbar)
    return SearchScreenObject()
  }

  fun assertThat(assertionBlock: HomeListScreenAssertions.() -> Unit): HomeListScreenObject {
    HomeListScreenAssertions().assertionBlock()
    return this
  }
}

class HomeListScreenAssertions : BaseScreenAssertions(R.id.homeListRoot) {
  fun headerContains(text: String) {
    assertDisplayed(text)
  }
}

class SearchResultScreenObject : BaseScreenObject {

  fun clickFloatingLoginButton(): LoginScreenObject {
    clickOn(R.id.searchResultLoginButton)
    return LoginScreenObject()
  }

  fun clickFilters(): SearchFiltersScreenObject {
    clickOn(R.id.searchResultFilterButton)
    return SearchFiltersScreenObject()
  }

  fun assertThat(assertionBlock: SearchResultScreenAssertions.() -> Unit): SearchResultScreenObject {
    SearchResultScreenAssertions().assertionBlock()
    return this
  }
}

class SearchResultScreenAssertions : BaseScreenAssertions(R.id.searchResultRoot) {

  fun createAlertButtonIsDisplayed() {
    assertDisplayed(R.id.searchResultCreateAlertButton)
  }

  fun searchBarContains(text: String) {
    assertAnyView(
      viewMatcher = withParent(withId(R.id.toolbar)),
      condition = withText(text)
    )
  }
}

The test code can be read out loud to any person following those steps in front of the app and they would understand what to do. You don’t see any Espresso code or Android resources ID. You only see concepts from the user perspective.

Here are some general rules that we follow:

• One PageObject per screen: There is a Page Object class for each screen in our app. The concept of Screen is subjective, it can be an Activity, a Fragment inside the BottomBar, a complex dialog, etc. When in doubt we ask ourselves: “what does the user perceive as a screen in this case?”.
• Actions: A PageObject contains functions for the actions that the user can perform on the current screen. They can be very concrete like clickSaveButton(), or wider like fillFormFields(). They can also contain parameters, like search("Java").
• Navigation: Each function returns the PageObject of the next screen after the action is performed. If the user stays on the same page, it returns itself. If the user navigates to a different screen, it returns the PageObject of that screen. This way we can fluently write actions across multiple screens, with the help of the IDE’s autocomplete.
• Assertions: We have an assertThat builder function in every PageObject that returns the Assertions of this PageObject. This Assertions object contains functions to assert different things on the screen.
• Be pragmatic: We don’t write every possible action or assertion for every screen. We add them as we need them. Remember that this is still code that must be maintained.

No fancy framework here, we try to keep it as simple as we can. These abstractions let us write UI tests really fast and keep them very semantic and readable. Modifying a feature or testing a new one is a breeze.

Barista is hidden in the implementation of the PageObject functions, so we still benefit from its magic. Sometimes we'll implement more complex actions or assertions that cannot be written with Barista and we'll write some custom Espresso code. Barista and Espresso are not exclusive!

Utilities

The last piece of our UI testing story is a series of smaller utilities that we've been adding over time. We grouped them in a InstrumentationTest interface, and I'm going to show some of them because I consider they help a lot with test readability.

Launch

This function takes an Intent and a lambda block, launches the Activity with the Intent, runs the code from the lambda, and then closes the Activity.

If it sounds awfully familiar to an ActivityScenario, it is because our launch function is just a wrapper around ActivityScenario to make the code a little bit easier to read and write.

@Test
fun openHome() {
  givenCountryIsSelected()
  givenUserIsNotLogged()

  launch(intentFactory.home.create(appContext)) {
    assertDisplayed(R.id.homeOffersList)
  }
}

@Test
fun openHome() {
  givenCountryIsSelected()
  givenUserIsNotLogged()

  ActivityScenario
    .launch<Activity>(intentFactory.home.create(appContext))
    .use { scenario: ActivityScenario<Activity> ->
      assertDisplayed(R.id.homeOffersList)
    }
}

fun launch(intent: Intent, testBlock: ActivityScenario<Activity>.() -> Unit) {
  ActivityScenario.launch<Activity>(intent).use { scenario ->
    scenario.testBlock()
  }
}

Before this, we were using the old ActivityTestRule (now deprecated). The test code didn’t look too bad, but it had some design issues that have been solved thanks to ActivityScenario. To mention some of its advantages, it provides fine-grained control of when the Activity gets launched and destroyed, we can easily launch different Activities on the same test, it lets you interact with the Lifecycle and even run code on the Activity’s Thread without sketchy tricks. There is an ActivityScenarioRule, but I would personally recommend avoiding it in favor of ActivityScenario, in order to have better control over the Activity or Activities in your test.

DeepLinks

Small and silly trick, but we added a launchDeepLink function to simplify testing deeplinks, hiding boilerplate code and workarounds.

@Test
fun offerSearchOpensWithDeepLink() {
  launchDeepLink("https://www.infojobs.net/ofertas-trabajo/barcelona") {
    SearchResultScreenObject()
      .assertThat {
        screenIsDisplayed()
        searchBarContains("Barcelona")
      }
  }
}

fun launchDeepLink(uri: String, testBlock: ActivityScenario<Activity>.() -> Unit) {
  val deepLinkIntent = Intent(Intent.ACTION_VIEW, Uri.parse(uri))
  // Component required because of a bug in ActivityScenario: https://github.com/android/android-test/issues/496
  deepLinkIntent.component = ComponentName(appContext.packageName, DeepLinkActivity::class.java.name)
  return launch(deepLinkIntent, testBlock)
}

IntentFactory

Another common task when dealing with Activities is creating Intents. In our app, we have an IntentFactory class what we use to create Intents to any screen. This class is injected by Koin, because it might depend on other dependencies like Feature Flags or A/B tests.

By having a commong InstrumentationTest interface we provide an easy shortcut instead of manually injecting it on every test eveery time. Again, a very small improvement that adds up!

interface InstrumentationTest {
  val intentFactory
    get() = KoinContextHandler.get().get<IntentFactory>()
  // ...
}

Rules

I'm sure you usually apply some JUnit rules to your instrumentation tests, don't you? We can apply a bunch of common test rules to all our tests. And we have many of them!

@get:Rule
val instrumentationRules: TestRule
  get() = RuleChain.outerRule(
    FlakyTestRule().allowFlakyAttemptsByDefault(FLAKY_ATTEMPTS))
    // ↓ All rules below flakyTestRule will be repeated if the test fails
    .around(FinishAllActivitiesRule())
    .around(DisableAnimationsRule())
    .around(IdlingUseCaseRule())
    .around(ReloadKoinRule())
    .around(ClearFilesRule().includeFilesMatching("/(.*).json"))
    .around(ClearCacheRule())
    .around(ClearPreferencesRule())
    .around(ClearDatabaseRule())
    .around(ResetLocaleRule())
    .around(RESTMockRule())
    .around(ScreenshotOnFailureRule())

private val FLAKY_ATTEMPTS: Int
  get() = if (BuildConfig.IS_CI) 7 else 1

And more!

I can't list every single little helpful feature we have for our tests, but I'd say these are the most impactful ones. Mocking the API, testing custom views, testing event tagging or screenshot testing to name a few. Maybe we'll share some more of these in the future!

The takeaway from this last part is that there is a lot of verbosity associated with UI tests that can be hidden away to make them easier to read, write and modify. These small things only save a few lines of code in each test. But they work together to make you forget about the testing framework, focus on what you really care about in the test, and let the existing infrastructure handle the rest.

Conclusion

UI tests can be a pain sometimes. But investing some time into them can make your life much easier, you'll soon find yourself writing new tests without effort.

I showed many different things here. You might like some, you might dislike others. I told our story because context is important. Find what works best for you and your team, and don't be afraid to iterate your solutions.

Writing a test for a button: 4 minutes.
Finding why the button hasn't worked in production for a week because you didn't write that test: 4 hours.
Write more UI tests! Don't trust Android!

Last but not least, I wrote this post but the contents and learnings are a product of many people that have worked on our Android Team. Thank you Roc, Jose, Rubén, Sergi, Bernat and many more contributors!

If you have any questions about the post or want me to expand on some part, please feel free to leave a comment below or ping me on Twitter @sloydev.

Thanks for reading!