Read the smb file using pyspark

37 views Asked by At

Read the smb file using pyspark:

for p in conn.listPath(share, path, pattern=pattern):
    p: SharedFile = p
    src = f'{share}{path}/{p.filename}'
    filepath = f'{path}/{p.filename}'
    src_temppath = tempfile.mktemp()
    enter code here
    with open(src_temppath, 'wb') as temp:
        conn.retrieveFile(share, filepath, temp)
  
    df = spark.read.format("binaryFile").load(f'file:///{src_temppath}')
    df = df.withColumn('values', F.explode(load_xls('content')).alias('values'))
    df = df.select('values.*')
    rows_count = df.count()

error: File file:///tmp/tmpzq27b61e does not exist

Is this a correct way to read the excel file from the SMB?

0

There are 0 answers