Columns with missing values not dropping

29 views Asked by At

The code below doesn't drop all (it does drop some) columns where null values exist, how come? Does it make distinctions between None, np.nan, etc?

columns_to_drop = [x for x in X.columns if X[x].isnull().sum()>0]
print(columns_to_drop)

dropped_X = X.drop(columns_to_drop,axis = 1).copy()

Neither does the simpler:

X.dropna(axis=1, how='any')

Imputing values isn't an option.

Finally, I had to drop rows where missing values existed, after having dropped all columns where missing values existed (clearly I hadn't though)

columns_to_drop = [col for col in X2.columns if X[col].isnull().any()]
print(columns_to_drop)

dropped_X = X.drop(columns_to_drop,axis = 1).copy()
dropped_X2 = X2.drop(columns_to_drop,axis=1).copy()

dropped_X2.dropna(axis = 0, how = 'any',inplace = True)

Having ran that last line I was able to proceed. Why did I have to, though?

0

There are 0 answers