R: Identify and remove columns with invalid column names

1.7k views Asked by At

Is there a way to identify invalid column names in R? perhaps using a regular expression or another technique.

I am generating a DocumentTermMatrix (DTM) from text column and then convert this DTM to a data frame. I end up with columns with invalid names. e.g.

"node" "CLASS" "️️️️" "️️️" " de" " des" " je devais" " the night" "her eyes " " cpas chaud" " lush cosmétiques" " ive see"

when I pass this dataset to mlr::makeClassificationTask, I get the following error message

Error in makeClassifTask(data = dat, target = "CLASS") : Assertion on 'data' failed: Columns must be named according to R's variable naming rules.

Hence, I would like to identify and remove all columns that have invalid names. something like

invalidColumnNames <- identify indexes of columns with invalid names
dat <- dat[,-invalidColumnNames]

data for REPRODUCIBLE EXAMPLE:

cols <- c("node", "CLASS", "️️️️", "️️️", " de", " des", 
" kmh", " points", " zéro", "\u2615️\u2615️", "\u2615️", 
"\u2693️\u2693️", "\u26f5️\u2693️", "\u2728\u2728\u2728\u2728\u2728", 
"aaliassime", "aaron", "abaixoassinado", "abandono", "abat", 
"abattu", "abiertamente", "abierto", "abit", "able", "abomination", 
"abonnements", "abonnés", "abonnez", "abraham", "absolutely", 
"abstract", "abused", "acaba", "acabar", "acabo", "acadiebathurst", 
"acaï", "acc", "accept", "accèsloisirs", "access", "accessible", 
"accessories", "accident", "accidentally", "acción", "acciones", 
"accommodationsreligious", "accompli", "accomplie", "accomplir", 
"accorde", "accordent", "account", "accounts", "accro", "accueil", 
"accueille", "accueillir", "accurate", "accusé", "accusent", 
"acérées", "acériculteur", "acha", "achat", "achei", "acheté", 
"acheter", "acho", "acidités", "acknowledge", "acontecem", "acordei", 
"acquis", "across", "action", "activité", "activités", "actresses", 
"actualité", "actuel", "adam", "adaptation", "adapter", "added", 
"addicive", "addicted", "addition", "additives", "addressed", 
"adds", "adeus", "adjoint", "adjointeadministrative", "adjust", 
"administratives", "adopción", "adopté", "adorable")

DESIRED OUTCOME:

"node", "CLASS", " de", " des", 
" kmh", " points", " zéro", "aaliassime", "aaron", 
"abaixoassinado", "abandono", "abat", 
"abattu", "abiertamente", "abierto", "abit", "able", "abomination", 
"abonnements", "abonnés", "abonnez", "abraham", "absolutely", 
"abstract", "abused", "acaba", "acabar", "acabo", "acadiebathurst", 
"acaï", "acc", "accept", "accèsloisirs", "access", "accessible", 
"accessories", "accident", "accidentally", "acción", "acciones", 
"accommodationsreligious", "accompli", "accomplie", "accomplir", 
"accorde", "accordent", "account", "accounts", "accro", "accueil", 
"accueille", "accueillir", "accurate", "accusé", "accusent", 
"acérées", "acériculteur", "acha", "achat", "achei", "acheté", 
"acheter", "acho", "acidités", "acknowledge", "acontecem", "acordei", 
"acquis", "across", "action", "activité", "activités", "actresses", 
"actualité", "actuel", "adam", "adaptation", "adapter", "added", 
"addicive", "addicted", "addition", "additives", "addressed", 
"adds", "adeus", "adjoint", "adjointeadministrative", "adjust", 
"administratives", "adopción", "adopté", "adorable"

Any help is greatly appreciated.

2

There are 2 answers

1
myincas On

maybe you can try this new package:

library(janitor) newdataobject <- read.csv("yourcsvfilewithpath.csv", header=T) %>% clean_names()

2
thelatemail On

See ?make.names for this sort of thing. I'd also remove the spaces at the starts and ends of variables, so:

cols <- trimws(cols)
cols[make.names(cols)==cols]

# [1] "node"  "CLASS"   "de"    "des"                    
# [5] "kmh"   "points"  "zéro"  "aaliassime" ...