I am trying to load/write the data from Redis cache in my spark-sql java based application. Here is my code:

SparkSession sprk = SparkSession.builder().appName("Bulk processing").master("local").config("spark.redis.host", "127.0.0.1")
                .config("spark.redis.port", "57855").getOrCreate();

Dataset<Row> employeeDF = sprk.read().format("org.apache.spark.sql.redis").
                option("table","employees").load();

employeeDF.show();

This is the exception I am getting:

Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset
    at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:205)
    at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
    at redis.clients.jedis.Protocol.process(Protocol.java:155)
    at redis.clients.jedis.Protocol.read(Protocol.java:220)
    at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:318)
    at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:255)
    at redis.clients.jedis.Connection.getBulkReply(Connection.java:245)
    at redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2972)
    at com.redislabs.provider.redis.RedisConfig.clusterEnabled(RedisConfig.scala:194)
    at com.redislabs.provider.redis.RedisConfig.getNodes(RedisConfig.scala:317)
    at com.redislabs.provider.redis.RedisConfig.getHosts(RedisConfig.scala:233)
    at com.redislabs.provider.redis.RedisConfig.<init>(RedisConfig.scala:132)
    at org.apache.spark.sql.redis.RedisSourceRelation.<init>(RedisSourceRelation.scala:34)
    at org.apache.spark.sql.redis.DefaultSource.createRelation(DefaultSource.scala:13)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
    at com.apps.employees.spark.Main.main(Main.java:41)

Caused by: java.net.SocketException: Connection reset
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:126)
    at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:199)
    ... 18 more

I have started the redis using docker and kubectl in Windows machine. enter image description here

I am not sure what is causing the issue. Could someone help me to resolve this issue

0

There are 0 answers