Is it possible that aws-sdk/dynamoose causes an SQLite syntax error on DynamoDB local?

Question

Context:

Problem found while upgrading from Nodejs 6 to 12 and with that the project's dependencies. Using dynamoose 2.3 Containerized application using docker-compose: backend and dynamodb instance only

Docker file for dynamodb:

FROM openjdk:latest

\# Bundle dynamodb
COPY . .

EXPOSE 8000
CMD [ "java", "-jar", "DynamoDBLocal.jar" ]

Problem: when lifting up the containers, after the backend initializes the dynamodb instance throws the errors below, causing any subsequent query or call to stall and return on timeout on the backend's side.

Error:

dynamodb_1  | Sep 03, 2020 8:14:36 AM com.almworks.sqlite4java.Internal log
dynamodb_1  | WARNING: [sqlite] SQLiteDBAccess$10@b6f156c: job exception
dynamodb_1  | com.almworks.sqlite4java.SQLiteException: [1] DB[1] prepare() DROP INDEX Foobar*HVI; [near "*": syntax error]
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteConnection.throwResult(SQLiteConnection.java:1436)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteConnection.prepare(SQLiteConnection.java:580)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteConnection.prepare(SQLiteConnection.java:635)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteConnection.prepare(SQLiteConnection.java:622)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.AmazonDynamoDBOfflineSQLiteJob.getPreparedStatement(AmazonDynamoDBOfflineSQLiteJob.java:138)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$10.dropGSISQLiteIndex(SQLiteDBAccess.java:1221)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$10.dropIndices(SQLiteDBAccess.java:1169)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$10.doWork(SQLiteDBAccess.java:1155)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$10.doWork(SQLiteDBAccess.java:1152)
dynamodb_1  |   at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.AmazonDynamoDBOfflineSQLiteJob.job(AmazonDynamoDBOfflineSQLiteJob.java:97)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteJob.execute(SQLiteJob.java:372)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteQueue.executeJob(SQLiteQueue.java:534)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:667)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77)
dynamodb_1  |   at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205)
dynamodb_1  |   at java.base/java.lang.Thread.run(Thread.java:832)

I suspect this is happening when creating the tables through Dynamoose's model() which under the hood calls aws' DynamoDB createTable method.

I'm currently just analysing the upgrade to nodejs 12 and dynamoose 2.3. In local I would prefer to have it run to test other parts of the project, so I don't mind updating indexes and recreating tables, but wish to know where this syntax error is coming from to fix it and carry on.

Question: Is it possible that the aws-sdk or dynamoose cause a DynamoDB local instance to attempt to drop an index with an SQLite syntax error?

Answer 1

The problem is that Dynamoose 0.8.7 used to support Schemas with attributes that were both hashkey and had an index marked as global. When jumping to the latest version, some breaking changes made it add characters like '*' to their queries and thus making SQLite complain.

Example:

const FoobarSchema = new dynamoose.Schema({
    foo: {
        type: String,
        hashkey: true,
        index: {
             global:true
        }
    }
});

const Foobar = dynamoose.model('Foobar', FoobarSchema, Options); // Blow up

I'm new to the project and DynamoDb. I don't understand what was the reasoning behind having those two conditions together and whether it should be supported or not. I'll dig deeper into this when I have the chance and update this.

Answer 2

Deleting the "shared-local-instance.db" file in my "docker/dynamodb" folder resolved the issue for me (the file is generated again on docker-compose up and everything runs fine)