Python Pandas to_sql removes all table indices when writing to table

1.1k views Asked by At

I have the following code which reads a MYSQL select command formed from left joining many tables together. I then want to write the result to another table. When I do that however, (with Pandas), it works properly and the data is added to the table, but it somehow destroys all indices from the table, including the primary key.

Here is the code:

q = "SELECT util.peer_id as peer_id, util.date as ts, weekly_total_page_loads as page_loads FROM %s.%s as util LEFT JOIN \
(SELECT peer_id, date, score FROM %s.%s  WHERE date = '%s') as scores \
ON util.peer_id = scores.peer_id AND util.date = scores.date  WHERE util.date = '%s';"\
    % (config.database_peer_groups, config.table_medians, \
   config.database_peer_groups, config.db_score, date, date)

group_export = pd.read_sql(q, con = db)

q = 'USE %s;' % (config.database_export) 
cursor.execute(q) 
group_export.to_sql(con = db, name = config.table_group_export, if_exists = 'replace', flavor = 'mysql', index = False)
db.commit()

Any Ideas?

Edit:

It seems that,by using if_exists='replace', Pandas drops the table and recreates it, and when it recreates it, it doesn't rebuild the indices.

Furthermore, this question: to_sql pandas method changes the scheme of sqlite tables suggests that by using a sqlalchemy engine it might potentially solve the problem.

Edit:

When I use if_exists="append" the problem doesn't appear, it is only with if_exists="replace" that the problem occures.

0

There are 0 answers