List Question
20 TechQA 2024-02-02T15:18:28.853000Reading a parquet file using parquet-tool converting string values to scientific notation
95 views
Asked by SwapnilM
Is there a painless way to migrate existing Parquet files to a new schema? I wish to update an AWS Glue table column data type
46 views
Asked by Andrew Parsons
pyspark structural streaming one column dynamic json save to elasticsearch as json object
13 views
Asked by Baitanik
pyspark error while writing data to existing hive table
124 views
Asked by dheeraj saini
ValueError: Unable to parse datatype from schema. Could not parse datatype: interval year
197 views
Asked by ZygD
pyspark - how to add a new element to ArrayType column
187 views
Asked by visu
Need to add headers in existing data frame
91 views
Asked by Giri Sankar
o1566.showString error persists even with os env variables defined with PySpark in jupyter notebook
63 views
Asked by user14255281
can we merge two parquet files into one which has different datatypes to the columns using the pyspark
56 views
Asked by prasanna kumar
check if rows are already present in pyspark dataset
169 views
Asked by x89
PySpark DataFrame creation is throwing PySparkTypeError
343 views
Asked by Mojdeh Ebrahimi
PySpark: Get Number of Columns from DataSchema
55 views
Asked by LordRofticus
How to get a timestamp data type column without the seconds in Pyspark?
81 views
Asked by bigdataadd
Pyspark calculate new rows based on previous rows from current and other multiple columns
124 views
Asked by prince13i
How do I grab all data from multiple tables found in a certain table in PySpark?
270 views
Asked by user3461502
Azure Synapse PySpark - Load Schema from a Schema Definition File
161 views
Asked by LordRofticus
Mulitple Row in spark dataframe using schema
46 views
Asked by Blue Clouds
Selecting a column with backtick in its name - AnalysisException: cannot resolve Column
186 views
Asked by Jim Macaulay
How to replace null value with some value using coalesce in pyspark
270 views
Asked by Vivek Mishra