Deserialization of generic container (of Record type) using `EXTERNAL_PROPERTY` fails for "natural" types (`boolean`/`long`/`String`/`double`) because type information is missing

[alwins0n]

Describe the bug
Use case is serializing and deserializing a generic container which uses the type info in the container to deserialize the content.
In the case of primitive boxed types (Integer, String) no type information is writte to the output. While this is excepted behaviour the dual operation fails because jackson complains that this very type information is missing.

Version information
Testet on multiple. Latest was 2.14.1

To Reproduce
Minimal example test

public class JacksonTest {

  @Test
  public void serDeser() throws Exception {
    var mapper = new ObjectMapper().registerModule(new ParameterNamesModule());

    var strContainer = new Container<>(1, "Hello");
    var myContainer = new Container<>(1, new MyObject("foo", "bar"));

    var strContainerJson = mapper.writeValueAsString(strContainer);
    var myContainerJson = mapper.writeValueAsString(myContainer);

    System.out.println(strContainerJson);
    System.out.println(myContainerJson);

    // this works with the correct type in memory
    var myContainerDeser = mapper.readValue(myContainerJson, new TypeReference<Container<?>>() {
    });
    // this fails with an exception
    var strContainerDeser = mapper.readValue(strContainerJson, new TypeReference<Container<?>>() {
    });
  }


  record Container<T>(
    int id,
    @JsonTypeInfo(
      use = JsonTypeInfo.Id.CLASS,
      include = JsonTypeInfo.As.EXTERNAL_PROPERTY,
      property = "type"
    )
    T value
  ) {
    @JsonCreator
    Container {
    }
  }

  record MyObject(
    String foo,
    String bar
  ) {
    @JsonCreator
    MyObject {
    }
  }

}

where the last line of the test fails with an exception and the full output is

{"id":1,"value":"Hello"}
{"id":1,"value":{"foo":"foo","bar":"bar"},"type":"JacksonTest$MyObject"}

Missing external type id property 'type'
 at [Source: (String)"{"id":1,"value":"Hello"}"; line: 1, column: 24] (through reference chain: JacksonTest$Container["value"])
com.fasterxml.jackson.databind.exc.MismatchedInputException: Missing external type id property 'type'
 at [Source: (String)"{"id":1,"value":"Hello"}"; line: 1, column: 24] (through reference chain: JacksonTest$Container["value"])
	at app//com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at app//com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1781)
	at app//com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1797)
	at app//com.fasterxml.jackson.databind.deser.impl.ExternalTypeHandler.complete(ExternalTypeHandler.java:283)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeUsingPropertyBasedWithExternalTypeId(BeanDeserializer.java:1100)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeWithExternalTypeId(BeanDeserializer.java:941)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:350)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at app//com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
	at app//com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4730)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3677)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3645)
	at app//JacksonTest.serDeser(JacksonTest.java:25)

Expected behavior
No type information on serialization for built-in primitive json mappable types ( String, Boolean, Integer ), but no type information should be required for deserialisation of these types using EXTERNAL_PROPERTY

Additional context
I searched the issues for something similar but could not find anything. Sorry if this is a duplicate

[alwins0n]

May I ask, if this is worked on or whether contributions to fix the issue are welcome and if so, which branch to target?

[cowtowncoder]

At this point, contributions would be against 2.16. Contributions always welcome!

[cowtowncoder]

Quick note: not including type information for a small number of "natural" types:

• String
• Boolean
• Double
• Long

is not a bug but feature -- optimization to reduce inclusion of type information, mostly for common case of String. In hindsight this may not have been the best choice, but at this point it is a fundamental feature and including type information for these types would be considered bug.

So the question is not that of adding type information for this set of types, but that of making these types work too. So there may well be an issue, but one has to know how to address it.

Also note that this being for Record type may be part of the issue compared to regular POJOs: Records' handling is somewhat different tha n POJOs, technically speaking (not conceptually tho; both should work the same or very similarly).

[cowtowncoder]

Another note: this may well be due Java "Type Erasure" problem as well -- it is, in general, a bad idea to directly use generic types as root values because actual runtime type information is NOT available for the root value: type as seen by Jackson during serialization is Container<?> (i.e. Container<Object>) and NOT as one might except, Container<String>.
Deserialization is not problematic since caller must provide generic target type.

This can be worked around (but there is no "fix" to avoid it, due to Java runtime type erasure) by one of:

1. Use temporary type -- for POJOs it'd be class StringContainer extends Container<String> but, alas, that is NOT available for Records (one weakness with records)
2. Pass explicit write type via ObjectWriter:

mapper.writerFor(new TypeRerefence<Container<String>>() { })
   .writeValueAsString(containerInstance);

3. Wrap generic type in concrete wrapper; generic types are fine at any other level (actual type binding comes from Class, not runtime instance)

class StringContainerWrapper {
     public Container<String> contents;
}

This is sort of FAQ, really, the only twist being use of Record as a type which complicates work-around: Records and generic types do not mix as well as with POJOs.

[alwins0n]

Fwiw: i just converted the record to a class. it did not change a thing. referecne code below.

your proposed solution would be the same as mine: leave serialization the way it is and be able to handle missing type information on deser if the value in question is a json primitive.

the whole note around type erasure - i don't completely get it. in the case of the container class the type may be erased but the class/type of the generic element can always be inspected if present (this may be a problem if null - I did not consider that yet)

as for deser - you can see that i did in fact not provide a well formed target type. Container<?> must suffice and it does (except the currenly breaking primitives) - since the jsontypeinfo basically enumerates all cases anyway (or allows all, like my example, with class info) and thus the deser should be able to handle all of them.

static final class Container<T> {
    private final int id;
    @JsonTypeInfo(
        use = JsonTypeInfo.Id.CLASS,
        include = JsonTypeInfo.As.EXTERNAL_PROPERTY,
        property = "type"
      )
    private final T value;


      Container(int id, T value) {
        this.id = id;
        this.value = value;
      }

    public int getId() {return id;}

    public T getValue() {return value;}

}

[cowtowncoder]

If anyone has time, would probably make sense to check if this fails with 2.18.0 -- POJO/Record property introspection was rewritten in 2.18 which could help with this case.