google / mobly-bundled-snippets

Snippets to allow Mobly tests to control Android devices by exposing a simplified Android API.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Document a test strategy for new snippets

yschimke opened this issue · comments

With this PR #158, there is a question "What are the breakages from bumping the targetSDK?"

It would be good to have a pattern for new snippets at least, to demonstrate that the snippet works on some particular device.

Useful when

  • changing the code of the snippet
  • changing the build such as targetSdk
  • testing against some new device

Question from @xpconanfan

How you would confirm the API actually behaved though?
It sounds like end to end or integration tests? Who would provide the HW to run those on?

run on emulator (GMD) to test or FTL devices?

GMD here https://android-developers.googleblog.com/2021/10/whats-new-in-scalable-automated-testing.html

Similar integration with FTL https://developer.android.com/studio/preview/features#ftl-gmd

As for what to test, I think that depends on the snippet, it might be enough to test against known state of the emulator.