A library to help auto-generate screenshot tests from Composable Previews with any screenshot testing library: JVM-based (i.e. Paparazzi, Roborazzi) as well as Instrumentation-based (i.e. Shot, Dropshots, Android-Testify, etc.)
Composable Preview Scanner | Showkase | Compose Preview Screenshot Testing | |
---|---|---|---|
Independent of AGP version | ✅ | ✅ | ❌ |
Library-agnostic solution | ✅ | ✅ | ❌1 |
Scans previews in "main" source | ✅ | ✅ | ❌2 |
Preview Infos available | ✅ | ❌3 | ✅ |
Specific Config (e.g. for Libs) available | ✅4 | ❌ | ❌ |
Compose Multiplatform support | *✅5 | ❌6 | ❌ |
1 Compose Preview Screenshot Testing is a standalone solution based on LayoutLib, whereas ComposablePreviewScanner and Showkase provide Composables' infos so you can run screenshot tests with your favourite screenshot testing library.
2 Compose Preview Screenshot Testing requires to put the previews in a brand-new "screenshotTest" source. ComposablePreviewScanner and Showkase only work with previews in the "main" source, so if you want to have the previews used for screenshot tests separate, I recommend to create an extra module for them and their screenshot tests and do not include that module in :app.
3 Showkase components only hold information about the Composable, but not about the Preview Info (i.e. ApiLevel, Locale, UiMode, FontScale...).
4 ComposablePreviewScanner supports adding extra lib-config (e.g. Paparazzi's Rendering Mode or Roborazzi's compare options) in the form of annotations that are additionally added to the preview. You can check how in the examples below in Jvm Screenshot Tests and Instrumentation Screenshot Tests respectively.
5 ComposablePreviewScanner provides some Compose Multiplatform support: It can scan @Preview
s in common
, as described in the Compose-Multiplatform section. Compose-Desktop @Preview
s are still not supported out of the box because of this issue, but you can check the Compose-Desktop support section for a workaround.
6 Showkase: Compose Multiplatform Support
ComposablePreviewScanner also works with:
@PreviewParameters
- Multi-Previews, including
@PreviewScreenSizes
,@PreviewFontScales
,@PreviewLightDark
, and@PreviewDynamicColors
. - private
@Previews
(from version 0.1.3 on)
but does not work with
@Previews
that are not located in the "main" source@Previews
inside classes, as reported in this issue
Add jitpack to your root build.gradle file:
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
dependencies {
// jvm tests (e.g. with Roborazzi & Paparazzi)
testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// instrumentation tests (e.g. with Shot, Dropshots & Android-Testify)
debugImplementation("com.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// compose multiplatform (jvm-targets), from 0.2.0
testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
}
For compose-desktop support, check this issue and its workaround.
Direct links
- Jvm Screenshot Tests
1.1 Paparazzi
1.2 Roborazzi - Instrumentation Screenshot Tests
- Compose Desktop Support
The API is pretty simple:
AndroidComposablePreviewScanner() // or CommonComposablePreviewScanner(), see Compose Multiplatform section
.scanPackageTrees("your.package", "your.package2")
// options to filter scanned previews
.excludeIfAnnotatedWithAnyOf(
ExcludeForScreenshot::class.java,
ExcludeForScreenshot2::class.java
)
.includeAnnotationInfoForAllOf(
ScreenshotConfig::class.java,
ScreenshotConfig2::class.java
)
.includePrivatePreviews() // Otherwise they are ignored
.filterPreviews {
// filter by any previewInfo: name, group, apiLevel, locale, uiMode, fontScale...
previewInfo -> previewInfo.apiLevel == 30
}
// ---
.getPreviews()
There are 2 more options to scan previews:
- All Packages:
scanAllPackages()
. This might require a huge amount of memory since it would scan not only in a set of packages, but in all packages used in your app/module (i.e. also in its transitive dependencies). This is in 99% of the cases unnecessary, and scanning the main package trees of your module should be sufficient. - From a file for faster times:
2.1.scanFile(jsonFile: File)
. The file had been previously generated by using ScanResultDumper().dumpScanResultToFile(jsonFile)
2.2.scanFile(inputStream: InputStream)
. This is useful for Instrumentation tests as you can see in its corresponding section.
You can find executable examples here
Let's say we want to enable some custom Paparazzi Config for some Previews, for instance a maxPercentDifferent value
- Define your own annotation for the Lib config.
annotation class PaparazziConfig(val maxPercentDifferent: Double)
- Annotate the corresponding Previews accordingly (you do not need to annotate all):
@PaparazziConfig(maxPercentDifferent = 0.1F)
@Preview(uiMode = Configuration.UI_MODE_NIGHT_YES)
@Composable
fun MyComposable(){
...
}
- Include your annotation info in the Preview
object ComposablePreviewProvider : TestParameter.TestParameterValuesProvider {
override fun provideValues(): List<ComposablePreview<AndroidPreviewInfo>> =
AndroidComposablePreviewScanner()
.scanPackageTrees("my.package", "my.package2")
.includeAnnotationInfoForAllOf(PaparazziConfig::class.java)
... // any other filtering option
.getPreviews()
}
- Map the PreviewInfo and PaparazziConfig values. For instance, you can use a custom class for that.
object PaparazziPreviewRule {
fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): Paparazzi =
Paparazzi(
deviceConfig = PIXEL_4A.copy(
nightMode =
when(preview.previewInfo.uiMode and Configuration.UI_MODE_NIGHT_MASK == Configuration.UI_MODE_NIGHT_YES) {
true -> NightMode.NIGHT
false -> NightMode.NOTNIGHT
}
... // other configurations
),
maxPercentDifference = preview.getAnnotation<PaparazziConfig>()?.maxPercentDifference ?: 0F
)
}
- Create the corresponding Parameterized Test:
@RunWith(TestParameterInjector::class)
class PreviewTestParameterTests(
@TestParameter(valuesProvider = ComposablePreviewProvider::class)
val preview: ComposablePreview<AndroidPreviewInfo>,
) {
@get:Rule
val paparazzi: Paparazzi = PaparazziPreviewRule.createFor(preview)
@Test
fun snapshot() {
paparazzi.snapshot {
preview()
}
}
}
- Run these Paparazzi tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordPaparazziDebug
You can find executable examples here
Let's say we want to enable some custom Roborazzi Config for some Previews, for instance a maxPercentDifferent value
- Define your own annotation for the Lib Config.
annotation class RoborazziConfig(val comparisonThreshold: Double)
- Annotate the corresponding Previews accordingly:
@RoborazziConfig(comparisonThreshold = 0.1)
@Preview(uiMode = Configuration.UI_MODE_NIGHT_YES)
@Composable
fun MyComposable(){
...
}
- Map the PreviewInfo and RoborazziConfig values. For instance, you can use a custom class for that.
object RoborazziOptionsMapper {
fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): RoborazziOptions =
preview.getAnnotation<RoborazziConfig>()?.let { config ->
RoborazziOptions(
compareOptions = CompareOptions(resultValidator = ThresholdValidator(config.comparisonThreshold))
)
} ?: RoborazziOptions()
}
object RobolectricPreviewInfosApplier {
fun applyFor(preview: ComposablePreview<AndroidPreviewInfo>) {
val uiMode = nightMode =
when(preview.previewInfo.uiMode and Configuration.UI_MODE_NIGHT_MASK == Configuration.UI_MODE_NIGHT_YES) {
true -> "+night"
false -> "+notnight"
}
RuntimeEnvironment.setQualifiers(uiMode)
... // other configurations
}
}
Check the following link for a full list of Robolectric device qualifiers and this blog post on how to set the cumulative Qualifiers dynamically
- Create the corresponding Parameterized Test:
@RunWith(ParameterizedRobolectricTestRunner::class)
class PreviewParameterizedTests(
private val preview: ComposablePreview<AndroidPreviewInfo>,
) {
companion object {
// Optimization: This avoids scanning for every test
private val cachedPreviews: List<ComposablePreview<AndroidPreviewInfo>> by lazy {
AndroidComposablePreviewScanner()
.scanPackageTrees("your.package", "your.package2")
.filterPreviews { previewParams -> previewParams.apiLevel == 30 }
.includeAnnotationInfoForAllOf(RoborazziConfig::class.java)
.getPreviews()
}
@JvmStatic
@ParameterizedRobolectricTestRunner.Parameters
fun values(): List<ComposablePreview<AndroidPreviewInfo>> = cachedPreviews
}
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [30]) // same as the previews we've filtered
@Test
fun snapshot() {
RobolectricPreviewInfosApplier.applyFor(preview)
captureRoboImage(
roborazziOptions = RoborazziOptionsMapper.createFor(preview)
) {
preview()
}
}
}
- Run these Roborazzi tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziDebug
Android does not use the standard Java bytecode format and does not actually even have a runtime classpath. Therefore, the simplest way to support instrumentation tests, is to...
- run the scan in a unit test & save it in a file accessible by instrumentation tests e.g. in assets
class SaveScanResultInAssets {
@Test
fun `task -- save scan result in assets`() {
val scanResultFileName = "scan_result.json"
ScanResultDumper()
.scanPackageTrees("my.package")
.dumpScanResultToFileInAssets(
fileName = scanResultFileName
)
assert(
assetsFilePath(
fileName = scanResultFileName
).exists())
}
}
Ensure that the .json with the scan result is up-to-date before executing the instrumentation screenshot tests. For instance, execute that test always before your instrumentation screenshot tests. Ideally, this scanning could be done via a Gradle Plugin in the future instead of by running it in a unit test.
- Now proceed to prepare your Composable Preview Tests with, for instance, Dropshots.
Let's say we want to enable some custom Dropshots Config for some Previews, for instance a maxPercentDifferent value.
- Define your own annotation
annotation class DropshotsConfig(val comparisonThreshold: Double)
- Annotate the corresponding Previews accordingly:
@DropshotsConfig(comparisonThreshold = 0.15) @Preview(uiMode = Configuration.UI_MODE_NIGHT_YES) @Composable fun MyComposable(){ ... }
- Map the PreviewInfo and DropshotsConfig values. For instance, you can use a custom class for that. To map the Preview Info values, I recommend to use the ActivityScenarioForComposableRule of AndroidUiTestingUtils
object DropshotsPreviewRule { fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): Dropshots = preview.getAnnotation<DropshotsConfig>()?.let { config -> Dropshots( resultValidator = ThresholdValidator(config.comparisonThreshold)) ) } ?: Dropshots() } object ActivityScenarioForComposablePreviewRule { fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): ActivityScenarioForComposableRule { val uiMode = nightMode = when(preview.previewInfo.uiMode) { Configuration.UI_MODE_NIGHT_YES -> UiMode.NIGHT else -> UiMode.DAY } return ActivityScenarioForComposableRule( config = ComposableConfigItem( uiMode = uiMode ... // other configurations ) ) } }
- Create the corresponding Parameterized Test:
@RunWith(ParameterizedTestRunner::class) class PreviewParameterizedTests( private val preview: ComposablePreview<AndroidPreviewInfo>, ) { companion object { private val cachedPreviews: List<ComposablePreview<AndroidPreviewInfo>> by lazy { AndroidComposablePreviewScanner() .scanFile(getInstrumentation().context.assets.open("scan_result.json")) .includeAnnotationInfoForAllOf(DropshotsConfig::class.java) .getPreviews() } @JvmStatic @ParameterizedTestRunner.Parameters fun values(): List<AndroidComposablePreview<PreviewInfo>> = cachedPreviews } @get:Rule val dropshots: Dropshots = DropshotsPreviewRule.createFor(preview) @get:Rule val activityScenarioForComposable: ActivityScenarioForComposableTestRule = ActivityScenarioForComposablePreviewTestRule.createFor(preview) @Test fun snapshot() { activityScenarioForComposableRule.onActivity { it.setContent { preview() } } dropshots.assertSnapshot( view = activityScenarioForComposable.activity.waitForComposeView() ) } }
- Run these Dropshots tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:connectedAndroidTest -Pdropshots.record
- Define your own annotation
WARNING Beware that Locale Strings in Preview Infos, unlike AndroidUiTestingUtils, use The BCP-47 tag but with + instead of - as separators, and have the prefix b+. Therefore, the BCP-47 tag "zh-Hans-CN" would be written as "b+zh+Hans+CN" instead. So for this case, you'd have to convert locale "b+zh+Hans+CN" to "zh-Hans-CN" in order to use it with AndroidUiTestingUtils
ComposablePreviewScanner also provides a class to customize the name of the generated screenshots based on its Preview Info. By default, it does not include the Preview Info in the screenshot file name if it is the same as its default value, but it can be configured to behave differently. That means, for @Preview(showBackground = false), showBackground would not be included in the screenshot file name since it is the default.
fun createScreenshotIdFor(preview: ComposablePreview<AndroidPreviewInfo>) =
AndroidPreviewScreenshotIdBuilder(preview)
// Paparazzi screenshot names already include className and methodName
// so ignore them to avoid them duplicated what might throw a FileNotFoundException
// due to the longName
.ignoreClassName()
.ignoreMethodName()
.ignoreForId("heightDp")
.ignoreForId("widthDp")
.overrideDefaultIdFor(
previewInfoName = "showBackground",
applyInfoValue = {
when (it.showBackground) {
true -> "WITH_BACKGROUND"
false -> "WITHOUT_BACKGROUND"
}
}
)
.build()
and then in your test
@Test
fun snapshot() {
paparazzi.snapshot(
name = createScreenshotIdFor(preview)
) {
preview()
}
}
So, for the following Preview
class MyClass {
@Preview(widthDp = 33, heightDp = 33, fontScale = 1.5f)
@Composable
fun MyComposable(){
...
}
}
createScreenshotIdFor(preview) will generate the following id: "MyClass.MyComposable.FONT_1_5f_WITHOUT_BACKGROUND"
This library is written on top of ClassGraph, an uber-fast parallelized classpath scanner.
Classpath can scan everything that is available either at bytecode level or at runtime.
This is also the case of annotations without retention or with either AnnotationRetention.BINARY
or AnnotationRetention.RUNTIME
, like Android Composable Previews
package androidx.compose.ui.tooling.preview
...
@Retention(AnnotationRetention.BINARY)
annotation class Preview(
...
)
However, those with AnnotationRetention.SOURCE
are not visible to Classgraph. Such annotations are mainly used for IDE tooling, and that is the case for the Compose-Desktop Preview annotation.
package androidx.compose.desktop.ui.tooling.preview
...
@Retention(AnnotationRetention.SOURCE)
annotation class Preview
Since Compose Multiplatform 1.6.0, Jetbrains has added support for @Preview
s in common
. ComposablePreviewScanner can also
scan such Previews when running on any jvm-target, like
- Android
- Desktop
- Jvm
ComposablePreviewScanner provides a CommonComposablePreviewScanner
for that purpose.
Assuming that you have:
- some Compose Multiplatform
@Previews
defined incommon
- some Android screenshot tests in place
Here is how you could also run screenshot tests for those Compose Multiplatform @Previews
together, for instance, with Roborazzi (would also work with Paparazzi or any Instrumentation-based library).
-
Add
:jvm
dependency from ComposablePreviewScanner 0.2.0+testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
-
Add an additional Parameterized screenshot test for these Compose Multiplatform
@Previews
@RunWith(ParameterizedRobolectricTestRunner::class)
class PreviewParameterizedTests(
private val preview: ComposablePreview<CommonPreviewInfo>,
) {
companion object {
// Optimization: This avoids scanning for every test
private val cachedPreviews: List<ComposablePreview<CommonPreviewInfo>> by lazy {
CommonComposablePreviewScanner()
.scanPackageTrees("your.package", "your.package2") // those where your common @Previews are located
.includeAnnotationInfoForAllOf(RoborazziConfig::class.java)
.getPreviews()
}
@JvmStatic
@ParameterizedRobolectricTestRunner.Parameters
fun values(): List<ComposablePreview<CommonPreviewInfo>> = cachedPreviews
}
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [30]) // same as the previews we've filtered
@Test
fun snapshot() {
captureRoboImage(
roborazziOptions = RoborazziOptionsMapper.createFor(preview)
) {
preview()
}
}
}
- Run these screenshot tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziDebug
As we've seen in the previous section How it works, Compose-Desktop previews are still not visible to ClassGraph since they use AnnotationRetention.SOURCE
.
There is already an open issue to change it to AnnotationRetention.BINARY
, which would allow ClassGraph to find them.
In the meanwhile, it is also possible to workaround this limitation with ComposablePreviewScanner as follows.
-
Add
:jvm
dependency from ComposablePreviewScanner 0.2.0+ and use Roborazzi, since it is the only Screenshot Testing Library that supports Compose-DesktoptestImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
-
Configure Roborazzi as described in the corresponding "Multiplatform support" section
-
Define your own annotation. The only condition is not to define
AnnotationRetention.SOURCE
package my.package.path @Target(AnnotationTarget.FUNCTION) annotation class VisibleForScreenshotTesting
-
Annotate the Desktop Composables you want to generate screenshot tests for with this annotation, e.g.
class MyClass {
@VisibleForScreenshotTesting
@Preview // It'd also work without this annotation
@Composable
fun MyDesktopComposable() {
...
}
}
- Create the test. Unfortunately,
kotlin.test
doesn't provide any means for parameterized tests, so we'll need to iterate through all previews. That has the disadvantage that, if it fails to record/verify one of the preview screenshots, subsequent screenshots won't be recorded/verified (hard asserts).
class DesktopPreviewScreenshotTests {
@OptIn(ExperimentalTestApi::class)
@Test
fun desktopPreviews() {
ROBORAZZI_DEBUG = true
val listComposables = JvmAnnotationScanner(
annotationToScanClassName = "my.package.path.VisibleForScreenshotTesting"
)
.scanPackageTrees("my.package.tree1", "my.package.tree2")
.getPreviews()
listComposables.forEach { composablePreview ->
runDesktopComposeUiTest {
setContent {
composablePreview()
}
onRoot().captureRoboImage(
filePath = "$composablePreview.png", // optional, works without it too
)
}
}
}
}
- Run these Roborazzi tests by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziJvm (if using the Kotlin Jvm Plugin)
Composable Preview Scanner logo modified from one by Freepik - Flaticon