SparkR functions failing in databricks

20 views Asked by At

I'm not an R user, but i've had a need to use SparkR for some analysis and i am trying to run the code below. Are pipes the issue here?

table %>%
SparkR::select('station') %>%
SparkR::count_distinct()

But got the error message:

Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘count_distinct’ for signature ‘"SparkDataFrame"’

also tried these to no avail.

SparkR::select(table, table$STATION) %>% 
SparkR::countDistinct()
SparkR::select(table, table$STATION) %>% 
SparkR::count_distinct()
1

There are 1 answers

1
dst On
select(table, count_distinct(table$STATION))