I am trying to access some json data using sqlContext.jsonFile in zeppelin...
following code execute without any error:
import sys.process._
val sqlCon = new org.apache.spark.sql.SQLContext(sc)
val jfile = sqlCon.jsonFile(s"file:///usr/local/src/knoldus/projects/scaladay_data/scalaDays2015Amsterdam_tweets.json")
import sqlContext.implicits._
jfile.registerTempTable("jTable01")
output :
import sys.process._ sqlCon: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@7034473 jfile: org.apache.spark.sql.DataFrame = [id: struct, content: string, hashtags: array, score: struct, session: string, timestamp: bigint, tweetId: bigint, username: string] import sqlContext.implicits.
next I verify the table name which I just registered
sqlCon.tableNames().foreach(println)
output :
jTable01
but when I try to run following I get an error:
%sql
select * from jTable01
output :
no such table jTable01; line 1 pos 14
at the same time when I run the tutorial example for "bank" it works.... the only difference I could make out is that in bank tutorial we are using sc.textFile but in my case I wanted to use sqlContext.jsonFile.
could you please provide any guidance as how I can fix this issue?
found the solution removed
val sqlCon = new org.apache.spark.sql.SQLContext(sc)
from my code and used zeppelin default sqlContext and it works!!!