I am using pg-promise for performing a multi row insert of around 800k records into a table in postgres database and facing the following error:
Error: Connection terminated unexpectedly at Connection. (/usr/src/app/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:509:28) at Connection.emit (node:events:390:28) at Socket. (/usr/src/app/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:390:28) at TCP. (node:net:687:12)
I have already tried to configure the connection with parameters like idleTimeoutMillis, connectionTimeoutMillis (also tried adding and removing parameters like keepAlive, keepalives_idle, statement_timeout, query_timeout) and still facing the issue.
const db = pgp({
host: host,
port: port,
database: database,
user: user,
password: pass,
idleTimeoutMillis: 0, // tried with multiple values
connectionTimeoutMillis: 0, // tried with multiple values
keepAlive: true,
keepalives_idle: 300,
statement_timeout: false,
query_timeout: false,
});
I am able to insert around 300k records without defining any of the extra parameters mentioned above.
Edited:
Following is the postgres logs:
LOG: server process (PID 680025) was
terminated by signal 9: Killed
2024-03-31 17:22:51.693 UTC [1] DETAIL:
Failed process was running: insert into
table
Looks like the query was killed. I am not sure if this is related to memory as I'am able to bulk insert with less records.
Any help is highly appreciated.