Function scan in DynamoDB doesn't bring some of the results

Question

I got a function in AWS Lambda that lists every patient in a table from DynamoDB. I realized that some items from the table were not on the list. This is my function to list:

module.exports.listPatients = async (event) => {
  try {
    const queryString = {
      limit: 5,
      ...event.queryStringParameters,
    };

    const { limit, next, name } = queryString;

    const localParams = {
      ...patientsParams,
      Limit: limit,
      FilterExpression: "contains(full_name, :full_name)",
      ExpressionAttributeValues: { ":full_name": name },
    };

    if (next) {
      localParams.ExclusiveStartKey = {
        id: next,
      };
    }

    const data = await dynamoDb.scan(localParams).promise();
    const nextToken = data.LastEvaluatedKey ? data.LastEvaluatedKey.id : "";

    const result = {
      items: data.Items,
      next_token: nextToken,
    };

    return {
      statusCode: 200,
      body: JSON.stringify(result),
    };
  } catch (error) {
    console.log("Error: ", error);
    return {
      statusCode: error.statusCode ? error.statusCode : 500,
      body: JSON.stringify({
        error: error.name ? error.name : "Exception",
        message: error.message ? error.message : "Unknown error",
      }),
    };
  }
};

Am I missing something?

I tried with and without a limit, removed the filters, and yet nothing. I tested one of the ids with get() to test with the server can find one of those who are missing, and it worked. I am using Serverless to deploy the code, and when I try offline, it's working. Stackoverflow recommended this post when writing my question, but I am using DynamoDB.DocumentClient without specifying the full attribute type in the filter expression: How to scan in DynamoDB without primary sort key with Nodejs

Answer 1

Looks like you are paginating using scan(). Using query() with some Global Secondary Indexes and ScanIndexForward would give you a much better performance. scan() doesn't scale well when your data grows.