Unable to insert clean unicode text back into DataFrame in pandas

167 views Asked by At

I am doing 2 things. 1) filter a dataframe in pandas 2) clean unicode text in a specific column in the filtered dataframe.

import pandas as pd
import probablepeople
from unidecode import unidecode
import re

#read data
df1 = pd.read_csv("H:\\data.csv")
#filter
df1=df1[(df1.gender=="female")]
#reset index because otherwise indexes will be as per original dataframe
df1=df1.reset_index()

Now i am trying to clean unicode text in the address column

#clean unicode text
for i in range(10):
    df1.loc[i][16] = re.sub(r"[^a-zA-Z.,' ]",r' ',df1.address[i])

However, i am unable to do so and below is the error i am getting.

c:\python27\lib\site-packages\ipykernel\__main__.py:4: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
1

There are 1 answers

7
jezrael On BEST ANSWER

I think you can use str.replace:

df1=df1[df1.gender=="female"]
#reset index with parameter drop if need new monotonic index (0,1,2,...)
df1=df1.reset_index(drop=True)

df1.address = df1.address.str.replace(r"[^a-zA-Z.,' ]",r' ')