Feature request: Support KSP
davidjwiner opened this issue · comments
For Kotlin projects, Dagger currently makes use of KAPT. This is generally quite slow since it requires generating Java stubs before the annotation processing step.
KSP offers an alternative to this by making it easier for annotation processors like Dagger to more directly use Kotlin's compiler plugin infrastructure. Based on initial benchmarks, using KSP can speed up build times by as much as 2x.
Just updating this to give a sense of where we are at. This is a priority for us that we are looking into. There are some complications though mostly due to the SPI plugins which expose the JSR-269 APIs through our public interface. There’s also the fact that our annotation processor code base is huge.
Because we still need to support our javac users, we cannot switch entirely to the KSP APIs which leaves us in a bit of a pickle since the places these are exposed are deep in our core model classes. I don’t think there is a possible way to get around this being a breaking API change, but we’ll need to figure out the right API to switch to that can support both KSP and javac SPI users.
Anyway, all this is to say, this is a high priority and we are working on this, but it is also a complicated thing to rollout for a codebase our size.
Can you use compiler-processing from Room to simultaneously support both? SPI users can, for now at least, be told they're unsupported in KSP (which seems fair considering the SPI API is javac-based anyway).
Thanks for the pointer, we are already in discussion with that team about that as a possibility =)
Hi any update on this? Thanks!
Still working on it. We're preparing a new model that will wrap both javac and KSP types like in this recent change: 30fa0f3
SPI users can, for now at least, be told they're unsupported in KSP
I've opened a separate issue #2802 for that to track status and discuss possible solutions as my team is concerned about KSP support in SPI.
@Jeffset, once we're ready to support KSP in Dagger, we will likewise support KSP in the SPI plugins as well. I think we can probably close #2802 , as this issue should track both.
Just to give you more details on our plans for the SPI plugin:
The new SPI model currently exists, and we are beginning to use it internally in Dagger. However, it is not yet available for users to write their own plugins.
The new model will be a thin wrapper around the javac/ksp types. For example, DaggerElement is a wrapper around Element in javac and KSDeclaration in KSP. If you need access to the javac or KSP types directly in your plugin, you will be able to call .compiler() to get which environment you're currently in (i.e. javac or KSP), and then call .javac() or .ksp() respectively to get access to the types.
Feel free to let us know if you have any concerns.
KSP is officially 1.0!
https://android-developers.googleblog.com/2021/09/accelerated-kotlin-build-times-with.html
My last dep that uses kapt is dagger + hilt! Do we have any estimated timeline? Just curious if this is a "likely by the end of 2021" type of feature request, or a "likely by the end of 2022" type of feature request. 😅
Dagger maintainers are pretty good about posting updates, let's trust a push system rather than nagging about something they are obviously aware of and working on.
Updating here since it has been some time. We've been making progress on the KSP migration, which includes making changes to Room's xprocessing libraries which we are using. Things were recently delayed for a while as the migration uncovered and exacerbated an existing issue which is the main change of the 2.41 release we just did. This issue caused a large delay not only in pulling people off the migration but also because we were afraid of making the problem worse by continuing the migration.
For that reason, it is still hard to give any sort of estimate as it is really the unknowns that we're going to hit that will take the bulk of the time. Just that one issue discovered set us back a couple months. With that said, now that the 2.41 release is finally out, we are now unblocked to continue the migration work.
any news here?
Just click this button
any update on the progress?
KSP support is something my team urgently needs. To the point that we are considering finding a solution other than dagger.
@byates-fleetian, please take a look at @ZacSweers’s blog post to help keep the discussion about the actual content of the feature rather than trying to put pressure on the maintainers 🙏
(I’m not a maintainer/contributer)
Considering the number of votes on this, which I would put around 4-500, a better question might be:
Is there any way we (the community) can help?
@TWiStErRob As usual in the OSS community – PRs are welcome ;)
Quick Update:
We’ve recently wrapped up the bulk of our migration to XProcessing -- a library that abstracts away the Javac APIs to allow Dagger’s processor to run in both Javac and KSP environments (see commits here).
However, along the way, there was a number of places in our codebase where we had to fall back to using Javac when there was no XProcessing equivalent. Thus, we’re now working with the XProcessing team to add support for these missing features.
Unfortunately, it’s unlikely that contributing PRs will help much in this phase of the project. The bulk of the work will be dedicated to designing the APIs, so any PRs would just shift our time from designing APIs to reviewing designs for APIs. In addition, this will require some level of coordination between the XProcessing and Dagger codebases, which will be easier to coordinate from our side.
However, once the bulk of that work is done, we will need some help testing with real projects using KSP, so we’ll definitely let you all know once we get there.
it's been a long time since last update. Any planned ETA?
This is quite a blocker for compose. To use 1.3.0 I need Kotlin 1.7, but that removes kapt. I'm sure I can't be the only one. Is a resolution of this issue close?
This is quite a blocker for compose. To use 1.3.0 I need Kotlin 1.7, but that removes kapt. I'm sure I can't be the only one. Is a resolution of this issue close?
Kapt (annotation processor) is not removed at all in Kotlin 1.7, it is the Kotlin Android Extensions that are removed (a way to use Android View directly from activity/fragment) In Kotlin 1.8 actually!
in Kotlin 1.7, it is the Kotlin Android Extensions that are removed
It's besides the point here, but even those are only removed by in 1.8.
Even considering all previous (fair) statements that kapt is not removed (AFAIK there is no plan to remove kapt, it will go into "maintenance mode" as ksp is a clear successor, but it is not getting removed anytime soon) - compose compiler version is not bound to other compose libraries. You can upgrade everything to 1.3.0 and leave compiler on 1.2.0. Source here And you can check compiler-kotlin compatibility chart here
Quick Update:
We've recently finished migrating the bulk of Dagger's tests to run on Javac, KAPT, and KSP (see commits here).
However, we still need to add basic coverage for Kotlin sources (our existing tests are mostly Java sources) which we expect to uncover more bugs, and we also need to address some API issues with using Dagger's SPI plugin with KSP.
Note that the goal here is not to have perfect coverage before releasing a first alpha version, but we do want to find and fix the most basic/common issues first to save both users’ time filing issues and our time triaging issues (in addition, it's often faster/easier for us to debug things internally since we have access to the source code). Also note that this is only an update for Dagger (not Hilt) -- Hilt will also need to be migrated to KSP once the work on migrating Dagger stabilizes.
Thanks a ton for keeping us updated. I have a question. When KSP support is added to dagger, will that also apply to dagger-android?
Thanks a ton for keeping us updated. I have a question. When KSP support is added to dagger, will that also apply to dagger-android?
dagger-android is Hilt.
Also note that this is only an update for Dagger (not Hilt) -- Hilt will also need to be migrated to KSP once the work on migrating Dagger stabilizes.
No, @Jean-Daniel, dagger-android is com.google.dagger:dagger-android with @ContributesAndroidInjector being the most known entry point. This predates com.google.dagger:hilt-android by 5 years.
Unfortunately we still use the dagger-android artifact that @TWiStErRob mentioned. If there is any additional work required to support KSP in dagger-android, I don't expect the Guava / dagger team to work on it. I just want confirmation so that I can say that KSP is 1 more reason to move away from dagger-android.
Unfortunatelly our 300-fragment big project also adopted dagger-android. For good or for worse, we heavily depend on
dagger.android feature of components per fragment, and the hilt is totally opposite. Migration will require extreme effort and missing KSP support hardly justifies it, so I still have poor man hope to see KSP support in dagger-android as well 🤞
Our plan is to eventually support KSP with dagger-android, but it would likely come after support for Dagger and Hilt.
just wondering when it will be done) maybe there is progress tracker or something like this?
maybe XProcessing progress could help
I am wondering if dagger migrated to ksp it will support kotlin native? Or will it helps dagger to support kotlin native in the future?
It should open the door for that in the future at least, that's the biggest advantage of ksp 😁
+1
Hoping some good news this IO 🤞
I/O mentioned this progress, hope we could see the progress updates and ETA soon!
ive been playing around with dagger snapshot since #3857 piqued my curiosity. looks like snapshot has ksp support?!
Sorry to bother all subscribers.
KAPT to KSP is one thing, but can we expect KMM soon after that or it is not planned yet ?
If a community member wants to help with KSP migration, will his pull requests be accepted?
For instance, I can take several issues from this list https://github.com/google/dagger/issues?q=XProcessing+is%3Aopen
@IlyaGulya there's only 3 issues there, the rest are PRs. See the "merge" vs circle icons on the left. Anything opened by copybara is an externalised changeset from their internal systems.
@IlyaGulya Thanks for the offer, we really appreciate it. Unfortunately, it probably isn't so easy for community members to contribute to the KSP migration. We're tracking a lot of the issues we're discovering internally because they have stacktraces/information from internal codebases. Also, a lot of issues are actually across different parts of the stack, not just in Dagger, and so a lot of the work comes from coordinating fixes across those other projects. Additionally, there are extra complications from the process to push code from our internal systems out here that just makes it difficult for us to take large code contributions from the community in general.
@bcorso is planning to update this thread soon with more details on where we're at and some of the issues we're currently working on. We know this has been going on for a long time, so we really appreciate all of your patience and support here.
KSP Update:
Hey all, it’s been a while since the last update, so thanks for being patient. In the past few months we’ve been collaborating with the KSP team to enable testing of Dagger’s KSP processor within Google so that we can test the processor on the many applications and libraries that are already using Dagger within Google's codebase.
As a result of that work, we found a number of bugs that we’re still working through (Note: these aren’t just bugs in Dagger, but at multiple layers including kotlinc, KSP, XProcessing, and even the testing infra pipeline itself). I'll likely have another update once we've finished debugging and triaging those issues and have a better sense of which ones will block the official release.
However, there is one breaking change to Dagger's API that will require changes from users to safely enable KSP, so I’ve added more details on that below. We're hoping that these changes will cause minimal churn for users and will provide an overall better experience when using Dagger in Kotlin, but please let us know if you have concerns.
Breaking changes required to use Dagger KSP:
With KSP, Dagger’s processor inspects the Kotlin source itself (rather than intermediate java stubs as when using KAPT). This makes it difficult to ensure correct behavior when switching from KAPT to KSP since the Kotlin types can elide use-site variance that were previously encoded explicitly in KAPT’s Java stubs. This is especially important in Dagger because Dagger currently relies on the exact type (wildcards included) to match a provided binding type to its requested types.
To fix this, our initial approach was to try to create a mapping function that would allow us to transform a KSP type to its equivalent KAPT type (basically trying to infer the correct variance/wildcards of each type argument). While this mostly works, it’s also error-prone since there’s no good way to ensure we’ve correctly inferred all variances. In addition, getting the mapping wrong can lead to subtle (even silent) functional changes when migrating from KAPT to KSP (e.g. with Dagger's multibindings), which can prevent production applications from safely making the switch to KSP. In fact, even once a user has switched to KSP, future changes to the mapping (e.g applying a fix) has a liability of accidentally regressing other untested cases, making it risky to change. While eventually, we could add enough tests to get this mostly right, it’s a risk that we’d rather not subject our clients to.
For these reasons, we have decided to change the way Dagger matches it's provided binding types to its request types, which should make it safe for clients to migrate to Dagger KSP:
- First, we’ll start by introducing a new flag that prevents users from providing a binding with the same invariant type, e.g
@Provides Foo<Bar>and@Provides Foo<? extends Bar>. If you are currently relying on this feature you will need to migrate those usages, e.g. by adding a qualifier to distinguish between the types instead of wildcards. We don’t anticipate that this flag will affect most users, and we hope that this change will actually lead to less confusion for users that are affected. - Next, we’ll introduce another flag (which requires the previous flag to be enabled) that allows users to request a binding via any of its type’s variances, e.g. injecting
@Inject Foo<Bar>or@Inject Foo<? extends Bar>would both be allowed (assuming the provided type can be assignable to them).
As an additional benefit, once the changes above have been applied, Kotlin users will no longer need to add @JvmSuppressWildcards and @JvmWildcards to the types in their Kotlin sources just to satisfy Dagger bindings.
We’re hoping to roll out these flags (with further instructions on how to enable them) out within the next few months. At least initially, these flags will be optional for users who don't plan to migrate to KSP.
Waiting for the good news, our team will move to KSP once Dagger is supported.
Any news here?
Hi all, Dagger 2.48 includes the alpha release of the Dagger and Hilt KSP processors. Please see the release notes for more details.
Also, as an update on https://github.com/google/dagger/issues/2349#issuecomment-1549890357, the @JvmSuppressWildcards and @JvmWildcard annotations will still be needed in your Kotlin sources until the second part of that change is completed. Currently, we have only implemented the first part of the proposed change (i.e. we’ve banned providing a binding with the same invariant type), as that was required for the KSP release.
Unfortunately, we had to put the second part of that change on hold as it was not needed for the KSP release, and it will likely take a good amount of time to address some of the unexpected issues we ran into while trying to implement it.
Hi all, Dagger 2.48 includes the alpha release of the Dagger and Hilt KSP processors. Please see the release notes for more details.
Also, as an update on https://github.com/google/dagger/issues/2349#issuecomment-1549890357, the
@JvmSuppressWildcardsand@JvmWildcardannotations will still be needed in your Kotlin sources until the second part of that change is completed. Currently, we have only implemented the first part of the proposed change (i.e. we’ve banned providing a binding with the same invariant type), as that was required for the KSP release. Unfortunately, we had to put the second part of that change on hold as it was not needed for the KSP release, and it will likely take a good amount of time to address some of the unexpected issues we ran into while trying to implement it.
Thanks for the alpha release, great job.
Does it mean that the "2nd part" is gonna generate Kotlin instead of Java code?
@FunkyMuse, for the "2nd part" I mean this:
"Next, we’ll introduce another flag [...] that allows users to request a binding via any of its type’s variances, e.g. injecting
@Inject Foo<Bar>or@Inject Foo<? extends Bar>would both be allowed (assuming the provided type can be assignable to them)."
Ideally, the implementation of that feature would not require generating Kotlin source, but admittedly that is part of what makes it difficult. Fwiw, I know generating Kotlin source is of some interest due to KMP; however, just to be upfront, it's not something we are currently working on at the moment.