Postgres constraint list filled with numbers and booleans

192 views Asked by At

I am migrating a database from MySQL to Postgres with a lot of data (128GB). I therefore, load the database without constraints and foreign keys to optimize performance, but it ultimately gives me weird constraints, that I do not understand.

I load the new database without constraints, for performance purposes and populate with data from old database (All code can be found below). However after completing all of this I get some weird constraints that I do not understand. The list is full of numbers and booleans values, whereas I would expect only the foreign keys, primary keys and unique constraint to be present. Can anyone tell me what it is? Moreover, when "only" loading 128GB is it then okay to build the database with unique constraint? The first code snippet shows the constraints. Thank you very much in advance

** Constraints **

(33129, 'forecasts_pkey', 2200, 'p', False, False, True, 33124, 0, 33128, 0, 0, ' ', ' ', ' ', True, 0, True, [1], None, None, None, None, None, None)
(33176, 'fk_models', 2200, 'f', False, False, True, 33124, 0, 33115, 0, 33112, 'a', 'a', 's', True, 0, True, [2], [1], [96], [96], [96], None, None)
(33182, 'unique_seriesid_modelrundate_modelrun_valuetimeutc_publishedatu', 2200, 'u', False, False, True, 33124, 0, 33181, 0, 0, ' ', ' ', ' ', True, 0, True, [2, 3, 7, 5, 8], None, None, None, None, None, None)

** BUILD SCRIPT **

    create table if not exists ModelType (
        Id int primary key,
        ModelType varchar (50)
    );

    create table if not exists Models (
        SeriesId int primary key,
        model varchar(50),
        ModelGroup varchar(50),
        Type varchar(20),
        Country varchar(50),
        Provider varchar(50),
        WeatherSystem varchar(10),
        Unit varchar(50),
        Area Varchar(10),
        ModelTypeId int,
        constraint fk_modeltype
            Foreign key (ModelTypeId) 
            references ModelType (Id)
    );

    CREATE TABLE IF NOT EXISTS Forecasts (
        id Serial PRIMARY KEY,
        SeriesId int,
        ModelRunDate date,
        InsertedAtUTC timestamp,
        ValueTimeUTC timestamp,
        Value REAL,
        ModelRun int,
        PublishedAtUTC timestamp,
    );

** After database has been fully loaded ** And run this code after I set constraints by first removing all duplicate code and set the constraints using below code:

def delete_duplicate_rows_from_forecast():
    cur = conn.cursor()
    sql = '''DELETE FROM forecasts f1
                    USING forecasts f2
                    WHERE f1.ctid < f2.ctid
                    AND f1.seriesid = f2.seriesid
                    AND f1.modelrundate = f2.modelrundate
                    AND f1.modelrun = f2.modelrun
                    AND f1.valuetimeutc = f2.valuetimeutc
                    AND f1.insertedatutc = f2.insertedatutc;'''
    cur.execute(sql)
    cur.close()
    conn.commit()

def set_constraints_after_load():
    delete_duplicate_rows_from_forecast()
    cur = conn.cursor()
    try:
        cur.execute('''ALTER TABLE Forecasts
            ADD CONSTRAINT fk_models
                FOREIGN KEY (SeriesId) 
                REFERENCES Models (SeriesId);''')
        cur.execute('''ALTER TABLE Forecasts
            ADD CONSTRAINT unique_seriesid_modelrundate_modelrun_valuetimeutc_publishedatutc
            UNIQUE (seriesid,modelrundate, modelrun,valuetimeutc, publishedatutc);''')
        conn.commit()

    except Exception as e:
        print(f'Set constraints error: {e}')
    finally:
        cur.close
        conn.close()

** Script to get all constraints **

def get_constraints(tablename):
    cur = conn.cursor()
    sql = f'''SELECT con.*
        FROM pg_catalog.pg_constraint con
        INNER JOIN pg_catalog.pg_class rel ON rel.oid = con.conrelid
        INNER JOIN pg_catalog.pg_namespace nsp ON nsp.oid = connamespace
        WHERE  rel.relname = '{tablename}';'''
    cur.execute(sql)
    for record in cur:
        print(record)

1

There are 1 answers

0
Umut TEKİN On

As @Richard Huxton said you don' t need to worry about pg_constraint entries. Yet, briefly

(33129, 'forecasts_pkey', 2200, 'p', ...

means;

33129 -> unique object identifier for the constraint
forecasts_pkey-> the constraint name
2200 -> public schema' s oid
p -> tells you about the type of the constraint, so for this one, it is primary

and it goes on. For details check pg_contraint.

To have a quick overview on the database object it might be better and easier to use management tools like psql, pgadmin or heidisql. Apart from these to get definition with the functions you can use pg_get_constraintdef for constraints and pg_get_indexdef for indexes.