Panda's DataFrame dump to CSV file is not decoding values correctly. It has Bytea data as columns

140 views Asked by At

I have a complex table structure in the Database, which I am reading Panda's DataFrame. While printing DataFrame everything is printing correctly but when I dump in CSV or convert it to a list (each of DataFrame row as list) I see the following data at few columns: <memory at 0x11a2c4640>

After debugging a little more I came to know they are BYTEA columns of Postgres and perhaps the conversation is falling. But actual confusion is if the print is fine, why write to CSV is not working. Is there any way to raw dump dataFrames?

2

There are 2 answers

0
Ajay A On

Try changing the type of the column, which holds BYTEA data

df[col_name] = df[col_name].astype(bytes)
df.to_csv(filename)
0
Pablo Vilas On

Try to save the csv with an special encoding:

df.to_csv(r"C:\your path\ file.csv, index =True, encoding='utf-8-sig')