NoSuchMethodError after upgrade
martinvisser opened this issue · comments
After upgrading to version 4.19.2, 4.19.3 or 4.19.4 I get a NoSuchMethodError with regards to Pivotal's cfenv library. This library is a provided library on a Java Buildpack in Cloud Foundry, which means we cannot upgrade it ourselves.
I'm a bit surprised by a patch level upgrade a method now no longer exists, so would be awesome if someone can look into it.
The stack trace:
Caused by: java.lang.NoSuchMethodError: 'java.lang.Object com.cedarsoftware.util.io.JsonReader.jsonToJava(java.lang.String, java.util.Map)'
at io.pivotal.cfenv.core.JsonIoConverter.jsonToJavaWithListsAndInts(JsonIoConverter.java:39)
at io.pivotal.cfenv.core.CfEnv.parseVcapServices(CfEnv.java:62)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:44)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:40)
at io.pivotal.spring.cloud.config.client.ConfigClientOAuth2BootstrapRegistryInitializer.initialize(ConfigClientOAuth2BootstrapRegistryInitializer.java:53)
at org.springframework.boot.SpringApplication.lambda$createBootstrapContext$1(SpringApplication.java:350)
No issue when downgrading back to 4.19.1.
You are right in that this upgrade should likely be 4.20 or a 5.0 in that it does change api.
The new API for JsonIo is the static class JsonIo class. In this case, the equivalent API is JsonIo.toObjects(), with the String json as the 1st parameter, a ReadOptions containing the options settings (which used to be passed in a Map, and are now created with new ReadOptionsBuilder().setting1().setting2().build()). The 3rd argument is an optional root class type. You can pass null for this, or the root class type if known a priori.
Please let me know if you have any issues making this change. You can reach me at jdereg@gmail.com.
Thanks @jdereg for explaining! The only option for us to downgrade, because we cannot make a change in the buildpack over which we have no influence. I will create an issue there in the hopes it will be picked up.
I added back the deprecated API which will allow you to upgrade to 4.19.5 and work fine again. Please let me know if you have any issues. That API will stay throughout all 4.x versions, so no reason to worry about it going away. We may keep it in 5.x as well.
Yes, awesome! Works like a charm!
Thank you for the quick fix.
Martin, we found that because the cfenv jar includes 4.19.1 json-io code, and if someone has code running inside the contained environment with a newer version of json-io, both versions are "blended" together and will likely not work. We just released json-4.19.9 that completed "repackages" the code so that none of the classes nor any of the resources are in the same folder as the prior versions.
This removes the NoSuchMethodError or any other unusual errors you may have seen. It appears their build is supposed to "shade" the json-io jar, my understanding being that they intended to move the classes in the jar to their own package space to prevent these types of conflicts, however, the classes are in the same package space in their (cfenv) jar as they are in json-io. We repackage code and resources to eliminate the overlap.
That sounds like an interesting way of "shading" indeed, more like copying over the same classes.
I tried upgrading to 4.19.9, but now our deployment fails again though.
Caused by: java.lang.NoClassDefFoundError: com/cedarsoftware/util/io/JsonReader
at io.pivotal.cfenv.core.JsonIoConverter.jsonToJavaWithListsAndInts(JsonIoConverter.java:39)
Do I need to wait on cfenv being upgraded?
Hello!
Having a similar issue. Version 4.19.+ breaks compatibility with who was using 4.18.+
2024-03-20T07:17:20.830-03:00 [APP/PROC/WEB/0] [ERR] Caused by: java.lang.NoClassDefFoundError: com/cedarsoftware/util/io/JsonReader
2024-03-20T07:17:20.830-03:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.cfenv.core.JsonIoConverter.jsonToJavaWithListsAndInts(JsonIoConverter.java:39)
2024-03-20T07:17:20.830-03:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.cfenv.core.CfEnv.parseVcapServices(CfEnv.java:62)
2024-03-20T07:17:20.830-03:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.cfenv.core.CfEnv.(CfEnv.java:44)
Looks like JsonReader is not inside util/io/ anymore, the class seems to be moved to /util/ and that breaks whoever is using older versions.
We have added JsonReader and JsonWriter transition version classes to the com.cedarsoftware.util.io package that have the earlier static APIs. Those should bind correctly with your code and get you going again.
The class JsonIo is the main API now. It is packaged at com.cedarsoftware.io. We repackaged the entire libary, and added the transition classes, due to conflicts with a container vendor's usage of json-io. They have an older version of json-io, in the classpath, which, if your application uses json-io, depending on where your jars appear in the classpath, could pick up the vendors or your json-io.jar file. By repackaging our code, it "moves it" out of the way and no longer conflicts with the container vendor's version of classes.
Please let me know if 4.19.10 corrects the NoSuchMethod errors you were seeing.
Thanks again for the update @jdereg, but unfortunately there still is an issue during deployment on Cloud Foundry:
Caused by: java.lang.NoClassDefFoundError: com/cedarsoftware/util/io/JsonObject
at io.pivotal.cfenv.core.JsonIoConverter.jsonToJavaWithListsAndInts(JsonIoConverter.java:39)
at io.pivotal.cfenv.core.CfEnv.parseVcapServices(CfEnv.java:62)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:44)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:40)
at io.pivotal.spring.cloud.config.client.ConfigClientOAuth2BootstrapRegistryInitializer.initialize(ConfigClientOAuth2BootstrapRegistryInitializer.java:53)
We have added JsonReader and JsonWriter transition version classes to the com.cedarsoftware.util.io package that have the earlier static APIs. Those should bind correctly with your code and get you going again.
The class JsonIo is the main API now. It is packaged at com.cedarsoftware.io. We repackaged the entire libary, and added the transition classes, due to conflicts with a container vendor's usage of json-io. They have an older version of json-io, in the classpath, which, if your application uses json-io, depending on where your jars appear in the classpath, could pick up the vendors or your json-io.jar file. By repackaging our code, it "moves it" out of the way and no longer conflicts with the container vendor's version of classes.
Please let me know if 4.19.10 corrects the NoSuchMethod errors you were seeing.
Thanks for the answer.
I think what you guys are doing is a breaking change.
Look at my situation here, this dependency is being introduced indirectly by
'io.pivotal.spring.cloud:spring-cloud-services-starter-service-registry'
The version this lib is using (4.x) has some CVE vulnerabilities and we wanted to force an upgrade to the latest 4.x and that broke the code, since you guys moved classes from packages.
If you guys really want to move packages, this should probably be a 5.x version.
Thanks all!
I have provided a "transition" layer that is at the same package level as the older code APIs, so that existing code should bind to the transition layer APIs for getting up and running quickly. That code includes the 5.0 APIs, at their new location, so you can convert to that at your leisure. Unfortunately, in the transition layer, I left references to JsonObject. Those have been removed.
I created a 4.19.11 version. Please try that and see if it works in CloundFoundry for you. If it does, I will release the 4.20 version as the final version on this path.
I did a deployment with 4.19.11 and it seems to work as expected again. I don't know if there are specific use cases in Cloud Foundry which I cannot foresee, but it always failed in the beginning, which it now doesn't do anymore.
I spoke too soon, it does still fail with 11.
It again gave the same error as previously:
Caused by: java.lang.NoClassDefFoundError: com/cedarsoftware/util/io/JsonObject
at io.pivotal.cfenv.core.JsonIoConverter.jsonToJavaWithListsAndInts(JsonIoConverter.java:39)
at io.pivotal.cfenv.core.CfEnv.parseVcapServices(CfEnv.java:62)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:44)
at io.pivotal.cfenv.core.CfEnv.<init>(CfEnv.java:40)
at io.pivotal.spring.cloud.config.client.ConfigClientOAuth2BootstrapRegistryInitializer.initialize(ConfigClientOAuth2BootstrapRegistryInitializer.java:53)
If you include the cfenv.jar in your gradle build file, you will not see this issue. Cloud Foundry does not include their shaded (shadowed) version of json-io when you include cfenv.jar in your build.gradle file.
As you can see in the screenshot, the cfenv.jar is already on the classpath. As far as I can see there is no shading going on for json-io actually.
And looking at the JsonIoConverter code, my IDE also shows it cannot find those classes:
Which makes sense if I look at the latest version of json-io, because those classes were moved.
And I guess it's completely okay to move those classes, but it just means we cannot upgrade to 4.19.x (except 4.19.1 and 4.19.5) until cfenv itself is compatible with this version.
Cloud Foundry shaded a portion of the classes in the jar, but not all of them. We have repackage the class into com.cedarware.io.* which is a something we wanted to do long ago anyway, and we have submitted a defect to Cloud Foundry about the partial shading of json-io.jar.
With our code moved out of the way, you can upgrade to 4.19.x (latest). Update your import statements, and use the static methods on JsonIo.toObjects() and JsonIo.toJava() instead of the static methods on JsonReader and JsonWriter.
I think there might be some confusion, as I believe there is a cfenv-all (something like it at least) and "normal" cfenv jar. The normal one is a transitive dependency as seen in this screenshot, not one that does (partial) shading:
So cfenv uses 4.14, not 4.19. Because there was a CVE we (I am not a maintainer of the cfenv library) tried to upgrade to the latest json-io, which was then 4.19.1. That worked, but then it broke, got fixed again in 4.19.5, then broke again. java-cfenv:3.1.1 does not shade anything, it just uses code as seen in my previous screenshot. That is not our code, that is code from the java-cfenv library, hence I cannot change any import statements.
As I said in my previous comment, I'm totally fine with packages being moved around, but it wasn't expected in a patch level upgrade. If 4.19 isn't compatible with Spring Cloud Services, so be it, we just cannot upgrade to the latest version until they do.
I am not sure I am completely understanding what you are describing. With json-4.19.13+, the packaging is now com.cedarsoftware.io.*. It will remain at that location going forward.
With the code no longer sharing namespacing with any code from Cloud Foundary, and if you updated your code to use 4.19.13+, what issue do you believe would remain? Can you try this?
In our project we do not use json-io itself, but the version (4.14.0) we get via Spring Cloud Services contains a CVE, that's the only reason we want to upgrade it. But SCS (the java-cfenv jar specifically) does not work with 4.19. Is there a version of json-io that does not have the CVE as found in 4.14.0, but isn't 4.19.x?
I have released 4.14.3, which should not have any CVE issues and is the "maintenance" branch of the best version of the "old API" json-io. See the new updated readme on GitHub for json-io. This outlines the path forward for 4.14.x, 4.20.x, and 5.0.
Scroll down to the new "Releases" section. Also, the documentation for the 4.20.0 has been completely blown out.
Anybody else stumbling across this and trying to figure out the migration steps:
Migrating the objects from v4.17 → 4.19 and above
https://github.com/jdereg/json-io/blob/4.19.1/changelog.md
Migrating a JsonReader.CONSTANT to ReadOptions
https://github.com/jdereg/json-io/blob/4.24.0/src/main/java/com/cedarsoftware/io/JsonIo.java#L291
Migrating a JsonWriter.CONSTANT to WriteOptions
https://github.com/jdereg/json-io/blob/4.24.0/src/main/java/com/cedarsoftware/io/JsonIo.java#L391
Hello from https://github.com/pivotal-cf/java-cfenv !
Can we close this issue? I believe all is well now between java-cfenv shading and json-io since this commit got merged in?
Thanks!
Thank you to everyone who provided input and help to close this issue.