Nested objects not being processed by `SerializerModifier` when class has custom Serializer (Jackson)

Question

Okay, so I've built a functional custom BeanSerializer, which limits the depth of serialization, and I can use it as shown by creating a SerializerModifier and adding it to module. This WORKS great and is called each time another nested field is encountered, creating the instance of DepthLimitedSerializer perfectly. HOWEVER, when I add a custom serializer to the nested class (using @JsonSerialize), then my modifySerializer method NEVER RUNS on the nested field!

Here is the simple class hierarchy, of which we will serialize the outer instance of Bar:

@Getter @Setter
public class BaseClass {
    private String id;
    private String someBaseProperty;
}

@Getter @Setter
//@JsonSerialize(using = FooSerializer.class)
public class Foo extends BaseClass {
    private String someFooProperty;

}

@Getter @Setter
public class Bar extends BaseClass {
    String someBarProperty;
    Foo fooOfBar;
}

And here is the simplified custom serializer to which you pass a maxDepth and if it reaches that depth, it ONLY serializes a single (id) field, otherwise, it simply calls the super:

public class DepthLimitedSerializer extends BeanSerializer {
    public static int DEFAULT_DEPTH = 2;
    private static final ThreadLocal<Integer> maxDepth = ThreadLocal.withInitial(() -> DEFAULT_DEPTH);
    private static final ThreadLocal<Integer> currentDepth = ThreadLocal.withInitial(() -> -1);

    public DepthLimitedSerializer(BeanSerializerBase src, int depth) {
        super(src);
        maxDepth.set(depth);
    }

    @Override
    protected void serializeFields(Object bean, JsonGenerator gen, SerializerProvider provider) throws IOException {
        if (maxDepth.get() < 0 || currentDepth.get() < maxDepth.get()) {
            currentDepth.set(currentDepth.get() + 1);

            super.serializeFields(bean, gen, provider);

            currentDepth.set(currentDepth.get() - 1);
        } else {
            try {
                Arrays.stream(_props).
                    filter(p -> p.getName().equals("id"))
                    .findFirst().orElseThrow()
                    .serializeAsField(bean, gen, provider);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
    }
}

In this simple test we populate a Foo and a Bar and serialize them:

class SerializeTest {
    @Test
    void testSerialization() throws JsonProcessingException {
        Foo foo = new Foo();
        foo.setId("fooID");
        foo.setSomeBaseProperty("someBaseValue");
        foo.setSomeFooProperty("foo property value");

        Bar bar = new Bar();
        bar.setId("barId");
        bar.setSomeBaseProperty("base of Bar");
        bar.setFooOfBar(foo);
        bar.setSomeBarProperty("bar property value");

        String depthOfZero = testSerializationToDepthOf(bar, 0);
        System.out.println("depth of ZERO: " + depthOfZero);
        String depthOfOne = testSerializationToDepthOf(bar, 1);
        System.out.println("depth of ONE: " + depthOfOne);
    }

    String testSerializationToDepthOf(BaseClass model, int depth) throws JsonProcessingException {
        ObjectMapper jackson = new ObjectMapper();
        jackson.enable(SerializationFeature.INDENT_OUTPUT);

        SimpleModule module = new SimpleModule("TestModule");
        module.setSerializerModifier(new BeanSerializerModifier() {
            @Override
            public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription beanDesc,
                                                      JsonSerializer<?> serializer) {
                if (BaseClass.class.isAssignableFrom(beanDesc.getType().getRawClass())) {
                    return new DepthLimitedSerializer((BeanSerializerBase) serializer, depth);
                }
                return super.modifySerializer(config, beanDesc, serializer);
            }
        });

        jackson.registerModule(module);
        return jackson.writeValueAsString(model);
    }
}

This gives us exactly what we expect, notice that with a depth of 0, we only get the id field of the nested Foo fooOfBar, which is correct, but at a depth of 1, we get the full output. This works to arbitrary depths just fine, and when it runs, that modifySerializer method as well as those in the DepthLimitedSerializer all run for EACH nested model!

depth of ZERO: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : {
    "id" : "fooID"
  }
}
depth of ONE: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : {
    "id" : "fooID",
    "someBaseProperty" : "someBaseValue",
    "someFooProperty" : "foo property value"
  }
}

HOWEVER! If one of these subclasses of BaseClass requires custom serialization, and I attempt to add a custom serializer for a particular class, like Foo:

public class FooSerializer extends StdSerializer<Foo> {

    public FooSerializer() {
        super(Foo.class);
    }

    public void serialize(Foo foo, JsonGenerator jgen, SerializerProvider serializerProvider)
            throws IOException {
        jgen.writeStartObject();
        jgen.writeStringField("custom", foo.getId() + "!" + foo.getSomeFooProperty());
        jgen.writeEndObject();
    }
}

And I uncomment the above @JsonSerialize(using = FooSerializer.class) line and assign the custom serializer to the class, then my modifySerializer method and all others are NEVER RUN for my custom-annotated nested class, and so the depth is ignored and the custom serializer just ALWAYS runs:

depth of ZERO: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : {
    "custom" : "fooID!foo property value"
  }
}
depth of ONE: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : {
    "custom" : "fooID!foo property value"
  }
}

The behavior I would have expected is for the modifySerializer method to STILL be run, and then for the super.modifySerializer() call to find that the FooSerializer, but instead it's only run for the top-level object (and others, not so custom annotated).

How can I achieve this behavior? I tried to make the custom serializer extends DepthLimitedSerializer, but they are of different types of Jackson Serializer` and I have thus far been unable to reconcile them and get them to work together in the correct order! Clearly I can't use the annotation to assign the serializer, but how can I?

Thank you all.

Answer 1

Here is my final working solution. I almost think it's a bug that serializers which were assigned via annotation are NOT processed by SerializerModifiers attached to the ObjectMapper. Nothing in the documentation says they'll "modify serializers, but oh, not those serializers." My solution is a bit hacky, but it works.

First, a custom BeanSerializerFactory to add running the modifiers if the serializer is found via annotation. This happens right at the beginning of the overridden method anyway.

public class FixedBeanSerializerFactory extends BeanSerializerFactory {
    public FixedBeanSerializerFactory(SerializerFactoryConfig config) {
        super(config);
    }

    @Override
    public JsonSerializer<Object> createSerializer(SerializerProvider prov, JavaType origType)
            throws JsonMappingException {

        // Very first thing, let's check if there is explicit serializer annotation:
        final SerializationConfig config = prov.getConfig();
        BeanDescription beanDesc = config.introspect(origType);
        JsonSerializer<?> ser = findSerializerFromAnnotation(prov, beanDesc.getClassInfo());
        if (ser != null) {
            if (_factoryConfig.hasSerializerModifiers()) {
                for (BeanSerializerModifier mod : _factoryConfig.serializerModifiers()) {
                    ser = mod.modifySerializer(config, beanDesc, ser);
                }
            }
            return (JsonSerializer<Object>) ser;
        }

        return super.createSerializer(prov, origType);
    }

    @Override
    public SerializerFactory withConfig(SerializerFactoryConfig config) {
        return new FixedBeanSerializerFactory(config);
    }
}

Now, this DepthLimitedSerializer is a LOT longer than it needs to be, mostly because I had to duplicate a lot of the code from BeanSerailzier. Why, you ask? Because the serialize method in BeanSerializer is final, so I can't override it. Instead, I'm forced to override BeanSerializerBase and copy a bunch of code from it.

public class DepthLimitedSerializer extends BeanSerializerBase {
    public static final int DEFAULT_DEPTH = 2;

    private static final ObjectMapper jackson = new ObjectMapper();
    private static final ThreadLocal<Integer> maxDepth = ThreadLocal.withInitial(() -> DEFAULT_DEPTH);
    private static final ThreadLocal<Integer> currentDepth = ThreadLocal.withInitial(() -> -1);

    public static JsonSerializer<Object> forSerializer(JsonSerializer<Object> serializer,
                                                       BeanDescription beanDesc,
                                                       int depth) {

        if (serializer instanceof BeanSerializerBase) {
            return new DepthLimitedSerializer((BeanSerializerBase) serializer, depth);

        } else {
            BeanSerializerBuilder builder = new BeanSerializerBuilder(beanDesc);
            JavaType type = jackson.constructType(serializer.handledType());
            BeanPropertyWriter[] properties = {};
            BeanPropertyWriter[] filteredProperties = {};
            maxDepth.set(depth);
            return new DepthLimitedSerializer(serializer, type, builder, properties, filteredProperties);
        }
    }

    protected JsonSerializer<Object> src;

    public DepthLimitedSerializer(BeanSerializerBase src, int depth) {
        super(src);
        this.src = src;
        maxDepth.set(depth);
    }

    protected DepthLimitedSerializer(DepthLimitedSerializer depthLimitedSerializer,
                                     BeanPropertyWriter[] properties, BeanPropertyWriter[] filteredProperties) {
        super(depthLimitedSerializer, properties, filteredProperties);
        this.src = depthLimitedSerializer;
    }

    protected DepthLimitedSerializer(JsonSerializer<Object> src, JavaType type, BeanSerializerBuilder builder,
                                     BeanPropertyWriter[] properties, BeanPropertyWriter[] filteredProperties) {
        super(type, builder, properties, filteredProperties);
        this.src = src;
    }

    protected DepthLimitedSerializer(BeanSerializerBase src, ObjectIdWriter objectIdWriter, Object filterId) {
        super(src, objectIdWriter, filterId);
        this.src = src;
    }

    protected DepthLimitedSerializer(BeanSerializerBase src,
                                     BeanPropertyWriter[] properties, BeanPropertyWriter[] filteredProperties) {
        super(src, properties, filteredProperties);
        this.src = src;
    }

    @Override
    public BeanSerializerBase withObjectIdWriter(ObjectIdWriter objectIdWriter) {
        return new DepthLimitedSerializer(this, objectIdWriter, _propertyFilterId);
    }

    @Override
    protected BeanSerializerBase withByNameInclusion(Set<String> toIgnore, Set<String> toInclude) {
        return null;
    }

    @Override // @since 2.11.1
    protected BeanSerializerBase withProperties(BeanPropertyWriter[] properties,
                                                BeanPropertyWriter[] filteredProperties) {
        return new DepthLimitedSerializer(this, properties, filteredProperties);
    }

    @Override
    protected void serializeFields(Object bean, JsonGenerator gen, SerializerProvider provider) throws IOException {
        if (maxDepth.get() < 0 || currentDepth.get() < maxDepth.get()) {
            currentDepth.set(currentDepth.get() + 1);

            super.serializeFields(bean, gen, provider);

            currentDepth.set(currentDepth.get() - 1);
        } else {
            try {
                JsonDepthLimited depthAnnotation = bean.getClass().getAnnotation(JsonDepthLimited.class);
                String defaultFieldName = depthAnnotation.defaultField();
                boolean includeNulls = depthAnnotation.includeNulls();
                if (StringUtils.isNotEmpty(defaultFieldName)) {
                    if (!includeNulls) {
                        Arrays.stream(_props).
                                filter(p -> p.getName().equals(defaultFieldName))
                                .findFirst().orElseThrow()
                                .serializeAsField(bean, gen, provider);
                    } else {
                        Arrays.stream(_props).forEach(p -> {
                            try {
                                if (p.getName().equals(defaultFieldName)) {
                                    p.serializeAsField(bean, gen, provider);
                                } else {
                                    gen.writeNullField(p.getName());
                                }
                            } catch (Exception e) {
                                throw new RuntimeException(e);
                            }
                        });
                    }

                }
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
    }

    @Override
    protected BeanSerializerBase asArraySerializer() {
        if ((_objectIdWriter == null)
                && (_anyGetterWriter == null)
                && (_propertyFilterId == null)
        ) {
            return new BeanAsArraySerializer(this);
        }
        // already is one, so:
        return this;
    }

    @Override
    public BeanSerializerBase withFilterId(Object filterId) {
        return new DepthLimitedSerializer(this, _objectIdWriter, filterId);
    }

    @Override
    public void serialize(Object bean, JsonGenerator gen, SerializerProvider provider)
            throws IOException {

        if (src == null || src instanceof BeanSerializerBase) {
            if (_objectIdWriter != null) {
                gen.setCurrentValue(bean); // [databind#631]
                _serializeWithObjectId(bean, gen, provider, true);
                return;
            }
            gen.writeStartObject(bean);
            if (_propertyFilterId != null) {
                serializeFieldsFiltered(bean, gen, provider);
            } else {
                serializeFields(bean, gen, provider);
            }
            gen.writeEndObject();

        } else {
            if (maxDepth.get() < 0 || currentDepth.get() < maxDepth.get()) {
                Class<?> t = src.handledType();
                src.serialize(t.cast(bean), gen, provider);
            } else {
                gen.writeNull();
            }
        }

    }

    @Override public String toString() {
        return "DepthAwareSerializer for " + handledType().getName();
    }
}

This annotation marks the classes included and lets us pass a few arguments.

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Inherited
public @interface JsonDepthLimited {
    String defaultField() default StringUtils.EMPTY;
    boolean includeNulls() default false;
}

And here is our POJO hierarchy to demonstrate:

@JsonDepthLimited(defaultField = "id")
@Getter @Setter
public class BaseClass {
    private String id;
    private String someBaseProperty;
}

@Getter @Setter
@JsonSerialize(using = FooSerializer.class)
public class Foo extends BaseClass {
    private String someFooProperty;
}

@Getter @Setter
public class Bar extends BaseClass {
    String someBarProperty;
    Foo fooOfBar;
}

A custom serializer for Foo:

public class FooSerializer extends StdSerializer<Foo> {
    protected FooSerializer() {
        super(Foo.class);
    }

    public void serialize(Foo foo, JsonGenerator jgen, SerializerProvider serializerProvider)
            throws IOException {
        jgen.writeStartObject();
        jgen.writeStringField("custom", foo.getId() + "!" + foo.getSomeFooProperty());
        jgen.writeEndObject();
    }
}

And finally, the demonstration:

class SerializeTest {
    @Test
    void testSerialization() throws JsonProcessingException {
        Foo foo = new Foo();
        foo.setId("fooID");
        foo.setSomeBaseProperty("someBaseValue");
        foo.setSomeFooProperty("foo property value");

        Bar bar = new Bar();
        bar.setId("barId");
        bar.setSomeBaseProperty("base of Bar");
        bar.setFooOfBar(foo);
        bar.setSomeBarProperty("bar property value");

        String depthOfZero = testSerializationToDepthOf(bar, 0);
        System.out.println("depth of ZERO: " + depthOfZero);
        String depthOfOne = testSerializationToDepthOf(bar, 1);
        System.out.println("depth of ONE: " + depthOfOne);
    }

    String testSerializationToDepthOf(BaseClass model, int depth) throws JsonProcessingException {
        ObjectMapper jackson = new ObjectMapper();
        jackson.enable(SerializationFeature.INDENT_OUTPUT);
        SerializerFactoryConfig factoryConfig = new SerializerFactoryConfig();
        jackson.setSerializerFactory(new FixedBeanSerializerFactory(factoryConfig));

        if (depth >= 0) {
            SimpleModule fragmentModule = new SimpleModule("FragmentModule");
            BeanSerializerModifier modifier = new BeanSerializerModifier() {
                @Override
                public JsonSerializer<?> modifySerializer(SerializationConfig config, BeanDescription beanDesc,
                                                          JsonSerializer<?> serializer) {
                    if (beanDesc.getClassAnnotations().has(JsonDepthLimited.class)) {

                        return DepthLimitedSerializer
                                .forSerializer((JsonSerializer<Object>) serializer, beanDesc, depth);
                    }
                    return serializer;
                }
            };

            fragmentModule.setSerializerModifier(modifier);
            jackson.registerModule(fragmentModule);
        }

        return jackson.writeValueAsString(model);
    }
}

And here in the final output you can see that the custom serializer runs when the depth is set to 1, but doesn't run (null is used) when the depth is set to 0, exactly as it should.

depth of ZERO: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : null
}
depth of ONE: {
  "id" : "barId",
  "someBaseProperty" : "base of Bar",
  "someBarProperty" : "bar property value",
  "fooOfBar" : {
    "custom" : "fooID!foo property value"
  }
}