I am trying to load/write the data from Redis cache in my spark-sql java based application. Here is my code:
SparkSession sprk = SparkSession.builder().appName("Bulk processing").master("local").config("spark.redis.host", "127.0.0.1")
.config("spark.redis.port", "57855").getOrCreate();
Dataset<Row> employeeDF = sprk.read().format("org.apache.spark.sql.redis").
option("table","employees").load();
employeeDF.show();
This is the exception I am getting:
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:205)
at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
at redis.clients.jedis.Protocol.process(Protocol.java:155)
at redis.clients.jedis.Protocol.read(Protocol.java:220)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:318)
at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:255)
at redis.clients.jedis.Connection.getBulkReply(Connection.java:245)
at redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2972)
at com.redislabs.provider.redis.RedisConfig.clusterEnabled(RedisConfig.scala:194)
at com.redislabs.provider.redis.RedisConfig.getNodes(RedisConfig.scala:317)
at com.redislabs.provider.redis.RedisConfig.getHosts(RedisConfig.scala:233)
at com.redislabs.provider.redis.RedisConfig.<init>(RedisConfig.scala:132)
at org.apache.spark.sql.redis.RedisSourceRelation.<init>(RedisSourceRelation.scala:34)
at org.apache.spark.sql.redis.DefaultSource.createRelation(DefaultSource.scala:13)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at com.apps.employees.spark.Main.main(Main.java:41)
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:126)
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:199)
... 18 more
I have started the redis using docker and kubectl in Windows machine. enter image description here
I am not sure what is causing the issue. Could someone help me to resolve this issue