API JSON Schema Validation with Optional Element using Pydantic

Question

I am using fastapi and BaseModel from pydantic to validate and document the JSON schema for an API return.

This works well for a fixed return but I have optional parameters that change the return so I would like to include it in the validation but for it not to fail when the parameter is missing and the field is not returned in the API.

For example: I have an optional boolean parameter called transparency when this is set to true I return a block called search_transparency with the elastic query returned.

{
  "info": {
    "totalrecords": 52
  },
  "records": [],
  "search_transparency": {"full_query": "blah blah"}
}

If the parameter transparency=true is not set I want the return to be:

{
  "info": {
    "totalrecords": 52
  },
  "records": []
}

However, when I set that element to be Optional in pydantic, I get this returned instead:

{
  "info": {
    "totalrecords": 52
  },
  "records": [],
  "search_transparency": None
}

I have something similar for the fields under records. The default is a minimal return of fields but if you set the parameter full=true then you get many more fields returned. I would like to handle this in a similar way with the fields just being absent rather than shown with a value of None.

This is how I am handling it with pydantic:

class Info(BaseModel):
    totalrecords: int

class Transparency(BaseModel):
    full_query: str

class V1Place(BaseModel):
    name: str

class V1PlaceAPI(BaseModel):
    info: Info
    records: List[V1Place] = []
    search_transparency: Optional[Transparency]

and this is how I am enforcing the validation with fastapi:

@app.get("/api/v1/place/search", response_model=V1PlaceAPI, tags=["v1_api"])

I have a suspicion that maybe what I am trying to achieve is poor API practice, maybe I am not supposed to have variable returns.

Should I instead be creating multiple separate endpoints to handle this?

eg. api/v1/place/search?q=test vs api/v1/place/full/transparent/search?q=test

EDIT

More detail of my API function:

@app.get("/api/v1/place/search", response_model=V1PlaceAPI, tags=["v1_api"])

def v1_place_search(q: str = Query(None, min_length=3, max_length=500, title="search through all place fields"),
                    transparency: Optional[bool] = None,
                    offset: Optional[int] = Query(0),
                    limit: Optional[int] = Query(15)):

    search_limit = offset + limit

    results, transparency_query = ESQuery(client=es_client,
                                          index='places',
                                          transparency=transparency,
                                          track_hits=True,
                                          offset=offset,
                                          limit=search_limit)

    return v1_place_parse(results.to_dict(), 
    show_transparency=transparency_query)

where ESQuery just returns an elasticsearch response. And this is my parse function:

def v1_place_parse(resp, show_transparency=None):
    """This takes a response from elasticsearch and parses it for our legacy V1 elasticapi

    Args:
        resp (dict): This is the response from Search.execute after passing to_dict()

    Returns:
        dict: A dictionary that is passed to API
    """

    new_resp = {}
    total_records = resp['hits']['total']['value']
    query_records = len(resp.get('hits', {}).get('hits', []))

    new_resp['info'] = {'totalrecords': total_records,
                        'totalrecords_relation': resp['hits']['total']['relation'],
                        'totalrecordsperquery': query_records,
                        }
    if show_transparency is not None:
        search_string = show_transparency.get('query', '')
        new_resp['search_transparency'] = {'full_query': str(search_string),
                                           'components': {}}
    new_resp['records'] = []
    for hit in resp.get('hits', {}).get('hits', []):
        new_record = hit['_source']
        new_resp['records'].append(new_record)

    return new_resp

Answer 1

Probably excluding that field if it is None can get the job done.

Just add a response_model_exclude_none = True as a path parameter

@app.get(
    "/api/v1/place/search",
    response_model=V1PlaceAPI,
    tags=["v1_api"],
    response_model_exclude_none=True,
)

You can customize your Response model even more, here is a well explained answer of mine I really suggest you check it out.