spark streaming and kafka integration dependency problem

Question

am trying to write a script that reads data from kafka with spark streaming but when i run "sbt compile" i get this error:

sbt.librarymanagement.ResolveException: Error downloading org.apache.spark:spark-sql-kafka-0-10_3:3.5.0

• spark version: 3.5.0
• kafka version: kafka_2.13-3.6.1

any help would be appreciated

if this has anything to do with version compatibility what are the possibe solutions for this problem.

Answer 1

_3 mentioned in the error isn't a valid version for that package

This is correct for sbt

libraryDependencies += "org.apache.spark" %% "spark-sql-kafka-0-10" % "3.5.0"

But make sure you have Scala 2.13 installed, not Scala 3, as Spark doesn't support that