TechQA.

Failed to find data source: kafka. Please deploy the application as per the deployment section of Structured Streaming + Kafka Integration Guide Spark

22 views Asked by Nícolas Farfán Cheneaux At 17 December 2023 at 00:20

Structured streaming with spark-kafka 3.3.2 failing

76 views Asked by Alex Oscherov At 16 December 2023 at 20:07

why pyspark task not stop？

15 views Asked by 小天哥 At 24 November 2023 at 08:13

Spark-Kafka Integration not working: Kafka broker with producer and consumer script are getting stuck as soon as we run spark script(consumer.py)

156 views Asked by Rushi At 04 October 2023 at 07:26

How to add Kafka dependencies for PySpark on a Jupyter notebook

297 views Asked by rarpal At 16 August 2023 at 18:05

Spark Kafka: understanding offset management with enable.auto.commit

75 views Asked by AzUser1 At 25 July 2023 at 09:07

Spark Streaming: How to handle failure in Spark when connecting to multiple kafka cluster via Union of Dstream?

25 views Asked by Deepank Porwal At 10 July 2023 at 22:53

Running Kafka and Spark with docker-compose

1k views Asked by yaviens At 12 May 2023 at 16:02

How to map a message to a object with `schema` and `payload` in Spark structured streaming correctly?

276 views Asked by Hongbo Miao At 27 April 2023 at 22:06

Difference between spark-streaming-kafka-0-10 vs spark-sql-kafka-0-10

590 views Asked by Hongbo Miao At 25 April 2023 at 02:38

Getting Error for org.apache.spark.sql.Encoder and missing or invalid dependency find while loading class file SQLImplicits, LowPrioritySQLImplicits

294 views Asked by Chandan Gawri At 19 April 2023 at 09:36

What is the best way to set up a kafka connection with apache spark

94 views Asked by Trodenn At 25 March 2023 at 02:28

I have more data in a kafka topic but when i extract data using my pyspark application, I am getting only 1 row extracted, how to fix?

121 views Asked by rakk At 01 March 2023 at 07:37

How to read DF Column of struct type and add they key value pairs to kafka headers?

78 views Asked by abhimanyu At 09 February 2023 at 21:20

spark streaming from kafka on spark operator(Kubernetes)

257 views Asked by haripriya rajendran At 09 February 2023 at 15:19

Pyspark : Kafka consumer for multiple topics

60 views Asked by erك At 21 January 2023 at 09:08

I am facing "java.lang.NoSuchMethodError: org.apache.spark.kafka010.KafkaTokenUtil$.needTokenUpdate" error while working with pyspark

753 views Asked by Muhammad Affan At 16 January 2023 at 08:27

java.lang.ClassNotFoundException: org.apache.spark.sql.kafka010.KafkaSourceRDDPartition

792 views Asked by Utpal Dutta At 05 January 2023 at 13:06

Error while pulling kafka jks certificates from hdfs (trying with s3 as well) in spark

409 views Asked by richa bharwal At 04 January 2023 at 15:13

How to process the dataframe which was read from Kafka Topic using Spark Streaming

1k views Asked by Kulasangar At 15 December 2022 at 11:15