Can we append a dataframe to snowflake table having some data, when some columns are same and some columns are different?

Question

I have a dataframe which contains some columns and snowflake table is having some columns. Some columns are same and some columns are different between them. As of now, I am extracting the snowflake table to python code and concatenating both and again replacing the table. But table is having huge data, it's very hectic. Is it possible to append the dataframe directly to the snowflake table when some columns are different and some are same. If yes, please tell me how can I do this.No solution is working for me. How can I do it effectively, with less time?

Answer 1

Yes It's possible to append the data to an existing table in a snowflake. Setup your connection.

You can use sqlalchemy and create an engine later you can push df to snowflake using:

from snowflake.connector.pandas_tools import pd_writer

df.to_sql('<snowflaketablename>', engine, index=False, method=pd_writer, if_exists='append')

remember to give option if_exists="append" to append the data frame to existing table.