Serializing Optional not enabled by default since 2.16.0

[Robert Wenzel]

Search before asking

• I searched in the issues and found nothing similar.

Describe the bug

in version 2.15.4 the following code below succeeded

giving the output

{"test":{"empty":false,"present":true}}
{"test":{"empty":false,"present":true}}

(Obviously this value doesn't make sense, but it doesn't fail with a hard exception, breaking existing projects)

when using version a higher version e.g. 2.17.0 the following Exception occurs instead (in both cases the jackson-datatype-jdk8.jar of the same version is on the classpath)

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 optional type `java.util.Optional` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jdk8" to enable handling (through reference chain: java.util.ImmutableCollections$Map1["test"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1330)
	at com.fasterxml.jackson.databind.ser.impl.UnsupportedTypeSerializer.serialize(UnsupportedTypeSerializer.java:35)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:808)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:764)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:502)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:341)
	at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4799)
	at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:4040)
	at Jackson2_17_Test_ObjectMapper2.main(Jackson2_17_Test_ObjectMapper2.java:24)

I didn't find this default behavior change in the release notes, so I wonder whether this is an intended change.
As a workaround I can register the Jdk8Module manually (commented out in the code above), to produce the same results as in 2.15.4

Version Information

2.17.0

Reproduction

Run the example using the maven dependencies

• com.fasterxml.jackson.core:jackson-databind:2.15.4
• com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.15.4

after switching to 2.16.0, 2.16.1, 2.17.0 the same code below fails

• com.fasterxml.jackson.core:jackson-databind:2.17.0
• com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.17.0

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;

import java.util.Map;
import java.util.Optional;

public class Jackson2_17_Test_Optional {

    public static void main(String[] args) throws JsonProcessingException {

        Map<String, Optional<String>> object = Map.of(
                "test", Optional.of("optional value"));

        JsonMapper jsonMapper = JsonMapper.builder()
//                .addModule(new Jdk8Module()) // workaround, register jdk8 module manually
                .build();
        String value = jsonMapper.writeValueAsString(object);
        System.out.println(value);

        ObjectMapper objectMapper = new ObjectMapper();
//        objectMapper.registerModule(new Jdk8Module()); // workaround, register jdk8 module manually
        String value2 = objectMapper.writeValueAsString(object);
        System.out.println(value2);
    }
}

Expected behavior

giving the output as in 2.15.4, not throwing an exception

{"test":{"empty":false,"present":true}}
{"test":{"empty":false,"present":true}}

Additional context

even though we have a manual workaround it's difficult to track and adjust all usages by external libraries.

my apologies in case I missed the related documentation and this is the new expected behavior since 2.16.0

[Robert Wenzel]

I found this was changed by #4082
so I guess that's the intended behavior now... if that's the case, please confirm and close the issue

[Tatu Saloranta]

@cor3000 Correct: this is the new intended behavior, and exception indicates the root cause. It is unfortunate issue referred does not fully explain logic (or have good title) to make it easier to find the change.

But the goal was to prevent accidental serialization of Optional using structure that was never meant to be used by databind (but happens to "work" due to existence of boolean isEmpty() method).

So registering Java 8 types module is the fix, not workaround. Or, alternatively, registering custom serializer.