Why eclipse thinks df.as[CaseClass] as an error in Scala Spark program?

Question

I'm trying to convert a dataframe to dataset using the syntax

case class Schema(...)
val ds = df.as[Schema]

So my code looks like

case class Rule(rule_on: String, rule_operator: String, rule_value: Int, rule_name: String)
val rules_ds = rules_df
   .select("rule_on", "rule_operator", "rule_value", "rule_name")
   .as[Rule]

But eclipse is highlighting .as[Rule] as error. Screen shots of the same as below.

How to resolve this issue? I know its not a Scala issue, as it works on command line.

Environment (as in Eclipse):

• Scala - 2.11.11
• Spark - 2.4.0
• JRE - 1.8

Answer 1

As suggested by Raphael Roth (in comments) I defined case class outside main method and it works like charm.

Also other solution (without using case class) is to typecast the dataframe to dataset as below

import org.apache.spark.sql._
val ds: Dataset[Row] = df

The above solution was taken from here