Generating common code

Question

Generating common code

belyaev-mikhail opened this issue 3 years ago · comments

If I understand it correctly, "supporting MPP" currently means that KSP can run on all platforms separately, but what if I want to generate common code?
Currently running ksp on an MPP project with common code results in generating the exact same generated code for each platform separately, without any common code generated (or able to generate).
Is generating common code in plans?

Dario Valdespino · Answer 1 · Mon Aug 16 2021 20:32:01 GMT+0800 (China Standard Time)

I think the kspKotlinMetadata task takes care of generating common code. However I've been having issues with it since it isn't resolving annotations properly. Maybe I'm not using it right though.

Eduardo Fonseca · Answer 2 · Wed Aug 25 2021 05:55:04 GMT+0800 (China Standard Time)

Hi @belyaev-mikhail. I'm facing a similar issue, but when trying to generate JS code while processing JVM code. My issue is that CodeGenerator does not support specifying the target platform. If this was possible, would it also work for your use case?

Mikhail Belyaev · Answer 3 · Sat Aug 28 2021 04:19:20 GMT+0800 (China Standard Time)

Hi @belyaev-mikhail. I'm facing a similar issue, but when trying to generate JS code while processing JVM code. My issue is that CodeGenerator does not support specifying the target platform. If this was possible, would it also work for your use case?

Maybe, depends on the way it's implemented. Basically, one could run ksp on any platform and generate code to common part, that would work for me, but for true MPP projects where there exists some some code in all supported platforms' folders, it may not work.

Eduardo Fonseca · Answer 4 · Mon Aug 30 2021 06:33:51 GMT+0800 (China Standard Time)

but for true MPP projects where there exists some some code in all supported platforms' folders, it may not work.

I'm not sure I understand why it may not work. Can you explain?

Mikhail Belyaev · Answer 5 · Mon Aug 30 2021 20:27:11 GMT+0800 (China Standard Time)

Because if you have, say, a library that contains: a common part, a JVM-specific part and a JS-specific part, you definitely want to generate common code from common part and common part only, which is impossible atm.

Eduardo Fonseca · Answer 6 · Tue Aug 31 2021 09:21:37 GMT+0800 (China Standard Time)

Well, it's possible, just not that straightforward. I was able to achieve common-only execution of KSP with the following Gradle configuration:

tasks.withType<com.google.devtools.ksp.gradle.KspTaskJS> {
  enabled = false
}

tasks.withType<com.google.devtools.ksp.gradle.KspTaskJvm> {
  enabled = false
}

tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile> {
  dependsOn(tasks.withType<com.google.devtools.ksp.gradle.KspTaskMetadata>())
}

Eva Tatarka · Answer 7 · Sat Oct 30 2021 04:40:09 GMT+0800 (China Standard Time)

Note: been testing with the new ksp* configurations on 1.0.1-RC, it seems like the metadata task isn't hooked up correctly? I'd expect:

dependencies {
    add("kspMetadata", "...")
}

would generate common code, but instead it does not run. Was able to get it working by hooking up task dependencies & source sets manually:

kotlin.sourceSets.commonMain {
    kotlin.srcDir("build/generated/ksp/commonMain/kotlin")
}
tasks.withType<org.jetbrains.kotlin.gradle.dsl.KotlinCompile<*>>().all {
    if (name != "kspKotlinMetadata") {
        dependsOn("kspKotlinMetadata")
    }
}

Alex Vanyo · Answer 8 · Sat Nov 13 2021 06:50:12 GMT+0800 (China Standard Time)

Another note for how the kspKotlinMetadata task works in conjunction with other platform tasks:

kspKotlinMetadata runs only against commonMain, which is expected. If you have an entirely pure project (where all code is in commonMain) then @evant 's code snippet above can created common code.

However, let's say you have a non-pure project, where some platform(s) have unique code (code underneath jvmMain, jsMain, etc.). That code won't be processed by kspKotlinMetadata, which also makes sense. If you try to add in processing for platform specific code (such as by adding kspJvm), the platform-specific code gets processed, but the commonMain code gets processed again. For a standard codegen processor, that usually results in duplicate classes for everything that was generated twice.

If each platform only processed code unique to that plaform, and generated code at the same level, I think this situation would be more natural. Under that scenario, kspKotlinMetadata (or maybe that should be kspCommon/kspCommonMain?) would just analyze commonMain code (and output it generated/ksp/commonMain), and kspJvm would just analyze jvmMain code (and output it in generated/ksp/jvmMain)

This is only a guess, but I'm guessing that would also help the situation for hierarchical source sets, where you might want to generate desktopMain code for instance.

Alex Vanyo · Answer 9 · Wed Nov 24 2021 03:00:46 GMT+0800 (China Standard Time)

Currently, there is also no ksp task for generating code from commonTest

Salomon BRYS · Answer 10 · Tue Dec 14 2021 23:46:28 GMT+0800 (China Standard Time)

Currently, there is also no ksp task for generating code from commonTest

This breaks our use case and forces us to revert to processing only "kspJvmTest" and having the generated directory be added to the commonTest source set.

Marie · Answer 11 · Sun Apr 10 2022 00:17:15 GMT+0800 (China Standard Time)

The code snippet above is not working for me. Is there a better way for generating common code?

Michael Rittmeister · Answer 12 · Sun Apr 10 2022 01:21:28 GMT+0800 (China Standard Time)

Another note for how the kspKotlinMetadata task works in conjunction with other platform tasks:

kspKotlinMetadata runs only against commonMain, which is expected. If you have an entirely pure project (where all code is in commonMain) then @evant 's code snippet above can created common code.

However, let's say you have a non-pure project, where some platform(s) have unique code (code underneath jvmMain, jsMain, etc.). That code won't be processed by kspKotlinMetadata, which also makes sense. If you try to add in processing for platform specific code (such as by adding kspJvm), the platform-specific code gets processed, but the commonMain code gets processed again. For a standard codegen processor, that usually results in duplicate classes for everything that was generated twice.

If each platform only processed code unique to that plaform, and generated code at the same level, I think this situation would be more natural. Under that scenario, kspKotlinMetadata (or maybe that should be kspCommon/kspCommonMain?) would just analyze commonMain code (and output it generated/ksp/commonMain), and kspJvm would just analyze jvmMain code (and output it in generated/ksp/jvmMain)

This is only a guess, but I'm guessing that would also help the situation for hierarchical source sets, where you might want to generate desktopMain code for instance.

kspKotlinMetadata doesn't exist anymore

However what does exist is "kspCommonMainMetadata`, but you cannot add any dependency to that configuration because it can't be resolved

Gabriel Souza · Answer 13 · Mon Apr 17 2023 04:21:52 GMT+0800 (China Standard Time)

The workaround apparently is outdated. The metadata task and configuration apparently has a new name. This is currently solution that I found for Kotlin 1.8.10 + (maybe is the same for older versions)

dependencies  {
  add("kspCommonMainMetadata", your-ksp-dependency)
}

tasks.withType<org.jetbrains.kotlin.gradle.dsl.KotlinCompile<*>>().all {
    if(name != "kspCommonMainKotlinMetadata") {
        dependsOn("kspCommonMainKotlinMetadata")
    }
}

kotlin.sourceSets.commonMain {
    kotlin.srcDir("build/generated/ksp/metadata/commonMain/kotlin")
}

Pavel Lahoda · Answer 14 · Sun Apr 23 2023 19:12:00 GMT+0800 (China Standard Time)

The workaround apparently is outdated. The metadata task and configuration apparently has a new name. This is currently solution that I found for Kotlin 1.8.10 + (maybe is the same for older versions)
tasks.withType<org.jetbrains.kotlin.gradle.dsl.KotlinCompile<*>>().all {
    if(name != "kspCommonMainKotlinMetadata") {
        dependsOn("kspCommonMainKotlinMetadata")
    }
}

tasks.withType() doesn't work for me if I have KMM with Android and iOS platforms. Getting error:

Cannot change attributes of dependency configuration ':shared:iosX64ApiElements' after it has been resolved

(and bunch of others for another ios* components)
There is additional note in the error:

Consumable configurations with identical capabilities within a project (other than the default configuration) must have unique attributes, but configuration ':shared:releaseFrameworkIosFat' and [configuration ':shared:debugFrameworkIosFat'] contain identical attribute sets. Consider adding an additional attribute to one of the configurations to disambiguate them.

But not sure how to achieve this. Any ideas?

Jonathan Bailey · Answer 15 · Tue May 02 2023 17:24:44 GMT+0800 (China Standard Time)

I had some trouble with some build tasks being run unnecessarily. I found only applying the dependsOn conditionally helped. Disclaimer: I'm primarily an iOS dev, I got here by process of elimination, I'm not really sure why the workarounds in this thread are needed, nor exactly what they do. But maybe this helps someone.

tasks.withType<KotlinCompile<*>>().all {
    if (name.startsWith("compileKotlinIos")) { // the remaining suffix is the target eg simulator, arm64, etc
        dependsOn("kspCommonMainKotlinMetadata")
    }
}

Leon Kiefer · Answer 16 · Sat May 13 2023 01:03:18 GMT+0800 (China Standard Time)

We are trying to generate expect/actual declarations with KSP. We are able to generate expect declaration in common code and actual declarations in the specific platforms, however the gradle tasks dependencies are not correct, so kotlin tries to compile the generated actual code without generating the expected declaration in common first, resulting in an compilation error.

If we try to apply the solution from #567 (comment) we get the error as described in #567 (comment). We are using kotlin 1.8.0.

Pavel Shurmilov · Answer 17 · Wed Aug 30 2023 19:51:48 GMT+0800 (China Standard Time)

Note that this workaround can lead to this problem with apple targets, the solution is to replace all with configureEach in the script

Kazem Moridi · Answer 18 · Fri Sep 01 2023 17:15:41 GMT+0800 (China Standard Time)

The workaround apparently is outdated. The metadata task and configuration apparently has a new name. This is currently solution that I found for Kotlin 1.8.10 + (maybe is the same for older versions)
dependencies  {
  add("kspCommonMainMetadata", your-ksp-dependency)
}

tasks.withType<org.jetbrains.kotlin.gradle.dsl.KotlinCompile<*>>().all {
    if(name != "kspCommonMainKotlinMetadata") {
        dependsOn("kspCommonMainKotlinMetadata")
    }
}

kotlin.sourceSets.commonMain {
    kotlin.srcDir("build/generated/ksp/metadata/commonMain/kotlin")
}

I used this with unsafe gradle cache and this error triggerd

Configuration cache state could not be cached: field `__libraries__` of task `:i18n:kspCommonMainKotlinMetadata` of type `com.google.devtools.ksp.gradle.KspTaskMetadata`: error writing value of type 'org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection'
> Querying the mapped value of task ':i18n:transformCommonMainDependenciesMetadata' property 'transformedLibrariesIndexFile' before task ':i18n:transformCommonMainDependenciesMetadata' has completed is not supported

with gradle cache I mean adding this code to gradle.properties

org.gradle.unsafe.configuration-cache=true
# Use this flag sparingly, in case some of the plugins are not fully compatible
org.gradle.unsafe.configuration-cache-problems=warn

Felix Wiemuth · Answer 19 · Thu Apr 18 2024 01:34:43 GMT+0800 (China Standard Time)

Unfortunately, none of the workarounds seem to work with KSP 1.9.21-1.0.15. For the solution with adding the task dependency (#567 (comment)) I get the same error also without unsafe caching:

Configuration cache state could not be cached: field `__libraries__` of task `:my-project:kspCommonMainKotlinMetadata` of type `com.google.devtools.ksp.gradle.KspTaskMetadata`: error writing value of type 'org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection'
> Querying the mapped value of provider(java.util.Set) before task ':my-other-project:allMetadataJar' has completed is not supported

I use a KSP processor added as project dependency. The my-other-project in the error message is another dependent module though.

My current workaround to make the generated files available in commonMain is to symlink build/generated/ksp/commonMain/kotlin to build/generated/ksp/jvm/jvmMain/kotlin. I have an annotation in a file in commonMain which results in code being generated in jvmMain which I suppose is the case because I am using and executing the code from a test in jvmTest. However, also here code generation is not always run. It seems that making changes in some other dependent module triggers it, but then again changes to files in the commonMain module again don't, resulting in the generated files being deleted.

How does one properly specify the task dependency with KSP 1.9.21?