I have a
df with one column
type and I have two lists
women = ['0980981', '0987098'] men = ['1234567', '4567854']
now I want to add another column based on the value of the
type column like this:
from pyspark.sql import functions as psf df_ = df.withColumn('new_col', psf.when(psf.col('type') == 'men', men).when(psf.col('type') == 'women', women))
But I guess we cannot insert list directly as we can insert
Array('1234567', '4567854') in Scala.
I have tried
psf.lit(men) as well but no luck.
Any idea on how to do it?