Allow @ConfigurationProperties binding for immutable POJOs
sdeleuze opened this issue · comments
Currently, it seems we are forced to use Kotlin classes with mutable nullable properties and default constructor with @ConfigurationProperties while idiomatic Kotlin code would be using classes with immutable properties initialized via constructor. I think there is a way to supporting that by leveraging kotlin-reflect library like jackson-module-kotlin do.
More concretely, in MiXiT app I would like to be able to convert this MixitProperties class implementation to:
@ConfigurationProperties("mixit")
class MixitProperties(
val baseUri: String,
val admin: Credential,
val drive: Drive
)
class Credential(
val username: String,
val password: String
)
class Drive(
val fr: DriveDocuments,
val en: DriveDocuments
)
class DriveDocuments(
val sponsorform: String,
val sponsor: String,
val speaker: String,
val press: String
)
We could imagine to support optional properties by using nullable types, like val sponsorform: String? for example.
I will be happy to help. I have also already worked with @apatrida who maintains Jackson Kotlin module, he may provide us some guidance I think.
We currently use Spring Framework's binder (that's also used in the MVC world) and it does not support binding via constructor. I guess that's the problem you want to solve?
We're rewriting the binder for Boot 2.0, perhaps @philwebb has some plans about that?
@jhoeller just pinged me to mention SPR-15199. Would be pretty cool if we could use that somehow but I guess the best course of action is our binder and 2.0 anyway...
Having said that, perhaps we can reuse some bits?
There are two models that work with Kotlin. The first, is that a plugin model to the binder let's Kotlin describe the constructors and properties providing information about which can be used to construct the class (or a combination of constructor + properties set after). The second, is that a source of all settable things is made available and the decision is made by a Kotlin instantiator that does the work based on what it has available.
The best support considers all of:
- what properties are set in the constructor
- what properties are writeable after construction
- for any property if it is nullable
- for any property in the constructor has a default value that can be used if it is absent
Kotlin reflection has a callBy method on the constructor that can be used in the case that default values are being used, otherwise it should use call method.
So the binder should give the opportunity to a plugin to do the introspection and instantiation so that the plugin can do the best job for the language.
Binding examples for Kotlin:
Jackson Kotlin Module (pulls parameter name information, then hands it back along with actual parameters to a custom instantiator)
JDBI 3 (the Kotlin module handles all binding tasks when the target is a Kotlin class)
If there is a branch to track for the new binder, I can look at it and see what needs to be done to support Kotlin, and add that support.
The experimental binder code is here, but it's still very much a work in progress. The plan is to make a binder that's specifically designed for binding from our configuration properties. It'll allow us to do some nice things like track the line/column number that a property was loaded from.
The Binder class is pretty complete, and there are quite a few tests. It deals with Collections, Arrays, Maps and Scalar types directly. When it finds something it can't handle it delegates to BeanBinder implementations.
We currently have a JavaBeanBinder that works with mutable classes but the plan is to add a couple more. We'd like to have a ValueObjectBeanBinder that binds using constructor arguments (this should work well with Kotlin).
We've also got plans for an InterfaceBeanBinder. The idea would be that you can so something like this:
@ConfigurationProperties(prefix="mine")
public interface MyProperties {
@DefaultValue("spring")
public String getName();
public int getAge();
}Which would be used to generate a proxy class with the appropriate implementation.
The first step is to get the binder working with our existing mutable configuration property beans. We'll then look at extending the support.
Quick update here: #8868 has been merged, the new binder is in master.
@apatrida Since the new binder is now in master, any chance you could have a look to identify what needs to be done to provide first class support for immatable data classes in Kotlin?
Is this planned for the 2.0 release or for later in the roadmap? @ConfigurationProperties are a bit confusing without it.
To summarize the discussion I just had with @sdeleuze we have interface binding on the pipe for 2.0 (see #1254). This issue is really about supporting constructor binding as well and it brings a number of challenges. One of them is that we need to upgrade the annotation processor to be able to detect and generate the metadata in such a case.
I bet Java developers will be interested by this feature as well and will use it with Lombok which will force us to detect all of these cases and I anticipate it is a lot of work. And supporting constructor binding without the metadata support is a no-go on my end.
I am flagging this one for team attention so that the rest of the team has a chance to review this.
@serandel Can you please expand a bit on what you find confusing at the moment?
@wilkinsona Of course. :)
Compare this class from my project (please, ignore the missing ElasticsearchSource definition):
@Component
@ConfigurationProperties
class ElasticsearchConfiguration {
var origin : ElasticsearchSource? = null
var destination : ElasticsearchSource? = null
override fun toString() = """
|ElasticsearchConfiguration:
|origin=$origin
|destination=$destination
""".trimMargin()
}
to the version I tried first:
@Component
@ConfigurationProperties
data class ElasticsearchConfiguration(val origin: ElasticsearchSource, val destination :ElasticsearchSource?)
Several features are dropped:
- properties have to be mutable
- properties have to be nullable
or use lateinit - equals, hashCode and toString are not included, and have to be written by hand (as I did in one case, just for debugging)
- copy and componentN are not included neither
From a Kotlin point of view, this type of configuration is obviously an immutable data class. There is some (very small, I concede) cognitive dissonance not being able to use a value type instead of a regular class for a limitation in the framework. Plus, the component is open to same rogue code altering a property.
I would really love to have this issue sorted out, as it would lead to beautiful, terse, expressive and idiomatic Kotlin data classes. :)
I just updated the previous comment. I thought that lateinit var would work... and it doesn't. :(
So, furthermore, we lose the distinction between optional and required properties. Perhaps it could be somewhat softened by using @required annotations and validating, I don't know, but it's not ideal.
Comment updated
2 remarks on the provided use case:
- IMO we don't really care if that's a
dataclass or not - I don't think you should inject beans in such class
That said, I think what makes @ConfigurationProperties confusing in Kotlin is is that immutable classes initialized at constructor level are a super common pattern (and a best practice) in Kotlin and a perfect use case here.
Kotlin null-safety makes this even more needed because you can only support that correctly via constructor based injection. Otherwise you force developers to write code like properties.admin!!.username!! with additional null safety operators where just properties.admin.username should be needed.
A regular Java ValueObjectBeanBinder that binds using constructor arguments should support Kotlin immutable classes (including data classes) without additional effort. The main point to check will be how it behaves with null-safety. So such @ConfigurationProperties should work:
@ConfigurationProperties("mixit")
class MixitProperties(
val baseUri: String,
val admin: Credential
)
class Credential(
val username: String,
val password: String?
)A more advanced use case that will require specific Kotlin support will be classes with optional parameters and default values like this one since from a bytecode POV 2 constructors are generated:
@ConfigurationProperties("mixit")
class MixitProperties(
val baseUri: String = "http://localhost:8080",
val admin: Credential
)
class Credential(
val username: String = "root",
val password: String?
)I should be able to provide the relevant utils method as part of https://jira.spring.io/browse/SPR-15673 to deal with this use case, making possible for Spring Boot to leverage it.
FYI Spring Framework 5.0 RC3 BeanUtils now fully supports Kotlin classes with immutable properties and optional parameters + default values while still providing API and implementation relevant for regular Java beans via the new BeanUtils#findPrimaryConstructor(Class) method and BeanUtils.instantiateClass(Constructor, Object...) which now supports Kotlin classes with optional parameters + default values.
Notice that Spring Data is about to add proper support for Kotlin constructor as well, so such support in Spring Boot for @ConfigurationProperties is the last missing bit to get proper support for typical Spring + Kotlin application based on immutable POJOs which are idiomatic in Kotlin (and a best practice worth to promote IMO).
For what I have understood from previous discussion, and since we now have the confirmation that Kotlin support for immutable POJOs for @ConfigurationProperties is mainly a small extension of the Java support for that, should we modify the title and scope of this issue to "Allow @ConfigurationProperties binding for immutable POJOs" in order to have an equivalent of #1254 but applied to POJO binding?
I know you have a huge todo for Spring Boot 2.0, but with the guidance of somebody aware of the new @ConfigurationProperties infrastructure, maybe I could help in contributing Java and Kotlin immutable POJO PR? I guess that could also be a way to make sure that current design is a good fit with immutable classes and eventually adapt it before 2.0 API freeze.
I think the code changes for the binding won't be that tricky. The annotation processor that generates the meta-data might be a bit harder.
There are more side effect to it. Like annotations we can only put on the getter that we'll have to accept on fields. Non scalar (nested) properties is also a challenge as soon as it's not an inner class.
We discussed this one at our team call last week and it's still going to be a challenge in its current form. @ConfigurationProperties annotated types becomes beans ultimately. Either because the type was provided explicitly or because a bean is exposed manually.
Regardless, a bean has a lifecycle and the constructor is one of them. While we don't advocate for injecting anything in a @ConfigurationProperties object, there's nothing stopping you from doing that. Concretely, it means that we can't afford to use the constructor as a mean to pass the environment properties.
That's why I see interface-based binding as a very good way forward: we keep control over the object's lifecyle and things are more consistent with what we have now.
Of course, the other options would be to stop allowing a @ConfigurationProperties type to be a bean which is quite harsh and brings other challenges (binding of 3rd party object we build in an auto-config being the most obvious one).
After discussing with @snicoll, I agree this feature requires to stop allowing a @ConfigurationProperties to be regular beans as well.
And in fact I see at least 3 advantages to remove such capability:
- It would enable
@ConfigurationPropertiesbinding for immutable POJOs (solve this issue) which is nice to have in Java but the most idiomatic way to support such feature in Kotlin - It would make class and interface based
@ConfigurationPropertiesbindings more consistent (in a sense interface based@ConfigurationPropertiesbinding is a kind of workaround to avoid having the bean injection issue) - It would stop to confuse users by providing a more error-prone and opinionated solution (I see so many people confused by these double properties + bean injection capabilities on Stack Overflow).
I may be biased and there would be side effects as explained by @snicoll, but I think that would be a good change even if a breaking one, so I vote 👍 for removing such support from Boot 2.0.
To move forward about the original need for Kotlin + Boot developers regardless of what Boot team decide on this issue, my point of view is that the most important point is to provide a solution to the current situation where both escaped @Value("\${foo}") and nullable mutable setter/field based @ConfigurationProperties are 2 far from ideal possible solutions .
This can be solved by this issue (ideal solution) but also by interface binding if Kotlin interfaces with val properties are supported (I will had a comment on this issue with more details about Kotlin bytecode generated in such case in order to evaluate if such support will be doable or not).
Spring Cloud allows users to override @Bean @ConfigurationProperties. In other words, some are marked with @ConditionalOnMissingBean. We do have rare cases where we inject things into beans and take advantage of the spring bean lifecycle.
How arduous would it be to update Cloud so that wasn't the case? We've decided in Boot that it was a mistake to allow people to override a @ConfigurationProperties beans. Overriding ServerProperties was particularly problematic, IIRC. It isn't explicitly prevented at the moment, but we're moving in that direction.
Unsure, the cases users have asked for are usually in places like providing hostnames and ipaddresses to the eureka instance configuration. We can certainly go in a different direction for Finchley if needed. We inject config beans into other classes all over the place. What would be the mechanism if it wasn't a spring bean? @dsyer thoughts?
@EnableConfigurationProperties will create a bean for all the classes listed. That's the recommended approach for creating a bean from a @ConfigurationProperties class
As long as they have a bean lifecycle I don't think we should care how the bean definitions were created. It will mean some changes in Spring Cloud, but they ought to be cosmetic for users (as long as they don't override the @Bean).
Just a reminder: we do still want to rebind anything mutable (and I think that ought to be a Spring Boot feature but that's a separate topic). So we need to be able to identify all the beans that are @ConfigurationProperties.
Any news on the immutable feature for ConfigurationProperties?
@wlsc We'd still like to do it, but we're focusing on other 2.0 issues at the moment.
I have just seen that #1254 has been postponed to 2.1, I am sure for good reasons, but while providing a way to handle immutable and non-nullable configuration properties is probably nice to have in Java, this is from my point of view a critical issue in to solve in Boot 2.0 to provide a decent development experience with Kotlin. And that not just my own advice, a lot of developers working with Spring Boot + Kotlin gives that feedback.
After more thoughts and a discussion with @bclozel about that, I tend to think that trying to provide exactly the same feature for Java + Kotlin is a dead end. #1254 was not an ideal solution in Kotlin, that's maybe not a bad thing it has been postponed after all.
Previously, @snicoll accurately said:
That's why I see interface-based binding as a very good way forward: we keep control over the object's lifecyle and things are more consistent with what we have now.
I do think Kotlin data classes which are specifically designed to hold data, are suitable for @ConfigurationProperties, even if in Java world interfaces are maybe the right answer. Also such restriction can make it easier to implement, since data classes are pure data and Spring Framework 5 provides now full support to instantiate such class.
My proposal (from a @bclozel idea ^^) is to provide support of @ConfigurationProperties limited to Kotlin data classes. Kotlin KClass provides a isData property that could allow us to identify them easily and limit the support only to this kind of class.
I would like to contribute it via a pull-request with your guidance.
Update: the solution proposed bellow does not work with nested classes so in practice is pretty useless :-(
Good news, while working on this issue, I found that the no-arg plugin allows to use Kotlin mutable @ConfigurationProperties data classes with non-nullable properties.
Concretely, with the following configuration:
plugins {
id "org.jetbrains.kotlin.plugin.noarg" version kotlinVersion
}
noArg {
annotation("org.springframework.boot.context.properties.ConfigurationProperties")
}It is possible to use the following mutable data classes:
@ConfigurationProperties("mixit")
data class MixitProperties(
var baseUri: String,
var contact: String,
var drive: Drive,
var aes: Aes) {
data class Drive(
var fr: DriveDocuments,
var en: DriveDocuments)
data class DriveDocuments(
var sponsorform: String,
var sponsor: String,
var speaker: String,
var press: String)
data class Aes(
var initvector: String,
var key: String)
}This issue is still needed to avoid the noArg trick and support immutable @ConfigurationProperties, but at least it provides a reasonably usable way to use @ConfigurationProperties with Kotlin + Spring Boot 2.0 regardless if we fix this issue in time for 2.0 or not.
In parallel, I am working on a Kotlin BeanBinder that supports immutable classes (including support for data classes) using Kotlin reflection. I will share the code later when I have something usable.
Apology, the proposed no-arg workaround proposed above does not work with nested classes, it does not raise any error but nested classes instances are null, so I am back on working on Kotlin BeanBinder in high priority mode in order to get it work before Boot 2 RC1.
@sdeleuze Boot 2 RC1 comes out tomorrow, is this feature going to be included?
@asarkar No. This issue has no target milestone. It's unlikely we'll be able to get to it until 2.1.
@serandel My latest tests with lateinit var seems to show it works as expected, could you please check again against Spring Boot 2.0.0.RC1 and Kotlin 1.2.20?
Is there any current solution to this I've tried a bunch of different variations of the above and get various errors
@ryan-barker-zefr No, we've not started working on this yet. We want to get it in Boot 2.1 if possible.
@ryan-barker-zefr For now just use lateinit var as described in the reference documentation.
Just a note of warning though, the metadata does not seem to be generated so there will be no autocomplete and no documentation unless I am missing something.
@vojtapol thanks but that warning has nothing to do with the task at hand and we'd like to keep things focused. AFAIK, kapt support works. If it does not, please create a separate issue with a sample that reproduces the issue.
@vojtapol Even if that's surprising, Kapt is not yet integrated in IDEA so you need to build your project on command line (for example with ./gradlew clean build if you use Gradle) to have your metadata generated.
Thank you @sdeleuze . Yes, I tried to manually invoke kapt through Maven in my case. It did generate the metadata file however only top-level properties were there. All nested properties were missing. Maybe it's a bug in kapt or in the kapt-maven-plugin.
I have not tested with Maven, but with Gradle nested properties are generated correctly in MiXiT project.
Finally, I think I figured out the problem of kapt only generating metadata of top-level properties. I will share it here in hopes it helps someone.
Preconditions:
- Use Maven
- Have a mixed Kotlin + Java project, (
@ConfigurationPropertiesfile is written in Kotlin) - Have
spring-boot-configuration-processoron the classpath
The problem
In the recommended Maven build setup for Kotlin maven-compiler-plugin is set to run after kotlin-maven-plugin. When the kapt goal on kotlin-maven-plugin runs the metadata is correctly generated.
Afterwards, maven-compiler-plugin runs its compile goal which also has annotation processing and if spring-boot-configuration-processor is on the classpath it will automatically discover it, run it and OVERRIDE the metadata with only top level properties.
Solution:
Either
- somehow prevent
maven-compiler-pluginfrom processing annotations
or
- remove
spring-boot-configuration-processorfrom dependencies
hell no, seems like much more easier just use java for any ConfigurationProperties components...
I'm wondering if the JDK12 record JEP could be a nice match here for Java (see this talk as well).
I am not sure if it's even needed anymore. Without kapt this will not work and kapt keeps breaking on every new kotlin release in our Maven build. Nobody from JetBrains is really maintaining it as far as I can tell. I am so tired of kapt issues I am thinking about just going back to Java POJOs for configuration.
@vojtapol Could you provide a repro project of the issue you see in order to allow me to bring that to Kotlin team ?
Thanks a lot Boot team for fixing that!
Re-opening so that we can update the reference docs. I’d also like to take another look at the occasional need for @Autowired although I am not sure there’s a better alternative.
The documentation refers to @ConfigurationPropertyDefaultValue but the annotation was renamed to @DefaultValue.
In the doc there is info that this is still not supported...
That link is for the wrong version of Spring Boot.
This was updated properly: https://docs.spring.io/spring-boot/docs/2.2.0.BUILD-SNAPSHOT/reference/html/spring-boot-features.html#boot-features-kotlin-configuration-properties
Ok, but this is first in the Google... and says that this is current.
The current refers to the current, i.e. latest, Spring Boot release that is GA. Once 2.2 reaches that state, the current link will point to the documentation for 2.2.0.RELEASE.
@wilkinsona Does this work with Java aswell (or) it works just with kotlin?
I tried with the following configuration & it did not resolve properties. Not sure if I had any mistake
application.yml
config:
username: alpha
password: password
timeoutInSecs: 2ExternalSystemConfig.java
@Getter
@Builder
@AllArgsConstructor(onConstructor = @__(@JsonCreator))
@ToString
@ConfigurationProperties("config")
public class ExternalSystemConfig {
private String username;
private String password;
private int timeoutInSecs;
}DemoApplication.java
@Log4j2
@SpringBootApplication
@EnableConfigurationProperties(ExternalSystemConfig.class)
public class DemoApplication {
public static void main(String[] args) {
new SpringApplicationBuilder()
.sources(DemoApplication.class)
.web(WebApplicationType.NONE)
.run(args);
}
@Bean
InitializingBean configLogger(ExternalSystemConfig config) {
return () -> log.info("Config: {}", config);
}
}But the config object that gets injected have all values set to null
I'm using 2.2.0.BUILD-SNAPSHOT version for this
Any idea if I'm doing something wrong
Thanks!
If I make my object mutable, the binding works fine tho
This is how my (lombok) generated class looks like
@ConstructorProperties({"username", "password", "timeoutInSecs"})
@JsonCreator
public ExternalSystemConfig(String username, String password, int timeoutInSecs) {
this.username = username;
this.password = password;
this.timeoutInSecs = timeoutInSecs;
}@thekalinga Immutable configuration property binding works with both Java and Kotlin. If you believe you've found a situation where it's not working as expected, please open a new issue with a complete and minimal sample that reproduces the problem. A project zipped and attached to the new issue or hosted in a separate GitHub repository is ideal.
@wilkinsona Just opened an issue with test project