TechQA.

org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus can't be cast to org.apache.spark.sql.execution.datasources.FileStatusWithMetadat

47 views Asked by Sarthak Sharma At 14 March 2024 at 08:15

How to setup and run scala-spark in intellij?

129 views Asked by PRATIK CHAPADGAONKAR At 10 January 2024 at 10:00

Scala Spark distributed run on Google Cloud Platform but worker not working

111 views Asked by Luca Genova At 10 December 2023 at 10:31

How to remove duplicate value from parquet file

433 views Asked by kislay kashyap At 20 November 2023 at 15:02

Spark write working with memory errors in Shell but not with spark-submit

56 views Asked by Andy_101 At 17 November 2023 at 07:29

Explode nested list of objects into DataFrame in Spark

84 views Asked by vZ10 At 10 November 2023 at 23:58

Access newly created column in withinColumn

105 views Asked by Kewitschka At 25 October 2023 at 07:17

Spark Array column - Find max interval between two values

92 views Asked by Abishek At 15 September 2023 at 11:52

Divide a column value into multiple rows by number of months based on start date & end date columns

94 views Asked by isrikanthd At 02 September 2023 at 13:43

Scala - Create Dataframe with only 1 row from a List using for comprehension

109 views Asked by Jiaming Pei At 11 July 2023 at 21:29

How to improve spark filter() performance on an array of struct?

331 views Asked by Yue Wang At 05 July 2023 at 05:29

AWS Glue scala spark job failing - org.apache.spark.util.collection.CompactBuffer[] not registered in Kryo

79 views Asked by Aki008 At 30 June 2023 at 04:24

Spark extract values from Json struct

126 views Asked by marc At 30 June 2023 at 03:23

Spark broadcasts right dataset from left join, which causes org.apache.spark.sql.execution.OutOfMemorySparkException

126 views Asked by alsetr At 02 June 2023 at 15:31

How to apply custom logics on spark dataframe using scala

46 views Asked by Vivek Gowda At 25 May 2023 at 07:27

Performance Implications of spark read

345 views Asked by Shiv Konar At 04 May 2023 at 12:54

MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-27c9b887c306cb9c-conf-map" not found

1.7k views Asked by My coding Way At 27 April 2023 at 14:29

Exporting file from Google Dataproc Job

35 views Asked by AT181903 At 21 April 2023 at 12:55

Convert Vector[String] to Dataframe in Scala Spark

73 views Asked by AT181903 At 29 March 2023 at 14:38

spark scala exploding struct array throwing error ambigous reference to fields

84 views Asked by instancedeveloper At 29 March 2023 at 06:16