How can I pass a parameter in spark-submit and throw it into a dataframe with Python 2.4?

437 views Asked by At

I need to pass a parameter in spark-submit call and assign it in a dataframe with Python 2.4, how can I do it?

import pyspark
from pyspark.sql import SparkContext
from pyspark.sql import SQLContext
from pyspark.sql import HiveContext
from pyspark.sql.functions import*
from pyspark.sql.types import*
import os, sys
import datetime
from datetime import datetime, timedelta

df_parm = <<parm-spark-submit>>

df_parm.show()
```
0

There are 0 answers