sqlalchemy, read a bytea string from postrgresql into a pockle.loads() function

32 views Asked by At

I am seeking to store ML fit() models into a postgresql database and then retrieve the models using an api to run a predict() function. I am inserting the data using a param SQL statement from the pickle dump serialize to string function; then later use pickle serialized to model using the pickle load function.

Dump model to table

model_as_byte = pickle.dumps(model)
params = {'year': RUN_YEAR, ..., 'model': model_as_byte}
sql = (f"INSERT INTO table values (:year, ..., :model); COMMIT;")
query = text(sql)
engine.connect().execute(query, params)

Data is stored successfully.

Retrieve model from table

sql = f"SELECT * FROM table where year = {RUN_YEAR}"
query = text(sql)
model_results = engine.connect().execute(query)
row = model_results.fetchone()
while row:
    large_data = bytes(row[15])  # Access LOB using large_object
    fit_model = pickle.loads(large_data)
    ....  other stuff here...
    row = model_results.fetchone()

The problem is: the value in column 15 contains "<memory at 0x12b08f280>". When moved to large_data it also contains "<memory at 0x12b08f280>". What am looking for is the bytea data in that location so that can pass it to pickle.loads() to restore the in-memory fit model and run a predict() on it. Google search and GPT indicates that row[15] should return the value, but it is returning a memory address. How can I retrieve the data at that memory location?

0

There are 0 answers