Colander: How do I handle null values for nested schema

Question

Using colander 1.5.1, if I pass null to an attribute defined by a nested schema:

class ChildSchema(colander.Schema):
    a = colander.SchemaNode(colander.Integer(), missing=None)
    b = colander.SchemaNode(colander.Integer(), missing=None)

class ParentSchema(colander.Schema):
    c = colander.SchemaNode(colander.Integer(), missing=None)
    d = ChildSchema(missing=None)

example json:
{
    "c": 1,
    "d": null
}

Then I get this error when deserialising:

"\"None\" is not a mapping type: Does not implement dict-like functionality."

Not passing the attribute d functions as expected, and deserialises to None. How do I correctly handle deserialising a null value passed to a nested schema? I would expect the behaviour to return None, based on the documentation. Deserialization Combinations

Answer 1

Please consider the following:

You need the following imports.

import colander
from colander import SchemaType, Invalid, null

Below is the same as your Child Schema.

class ChildSchema(colander.Schema):
    a = colander.Schema(colander.Integer(), missing=None)
    b = colander.Schema(colander.Integer(), missing=None)

Below is where all the magic occurs. We create our own type. Note that it may lack some of the basic functionalities that the built-ins in colander SchemaTypes may offer (like serialize). If the recieved object is null or None then it is returned with not changes. If it is not null or None and not a dictionary it will raise an error, and if it is a dictionary it will be serialized with your ParentSchema, even if the attributes of the Parent are null or None ({"d": null}).

class MyType(SchemaType):
    def deserialize(self, node, cstruct):
        if cstruct is null:
            return null
        if cstruct is None:
            return None
        if not isinstance(cstruct, dict):
            raise Invalid(node, '%r is not an object' % cstruct)
        else:
            return ChildSchema().deserialize(cstruct)

We create the Parent Schema using the magic type:

class ParentSchema(colander.Schema):
    c = colander.SchemaNode(colander.Integer(), missing=None)
    d = colander.SchemaNode(MyType(), missing=None)

"d" will be deserialize as u wish. Now lets see some examples of its usage:

schema = ParentSchema()

Example 1. "d" is null (The main question)

schema.deserialize({"c": 1, "d": null})

Output 1

{"c": 1, "d": None}

Example 2. "d" is None

schema.deserialize({"c": 1, "d": None})

Output 2

{"c": 1, "d": None}

Example 3. Normal behaviour

schema.deserialize({'c': 1, 'd': {'a': 1}})

Output 3.

{'c': 1, 'd': {'a': 1, 'b': None}}

Example 5. Error "d" is not dict

 schema.deserialize({'c': 1, 'd': [] })

Output 5

 # Invalid: {'d': '[] is not an object'}

Example 6. Error validator not number

 schema.deserialize({'c': 1, 'd': {'a': "foobar"}})

Output 6

# Invalid: {'a': u'"foobar" is not a number'}

To write this answer I used https://docs.pylonsproject.org/projects/colander/en/latest/extending.html as a source.