Linked Questions

Popular Questions

Pyspark method call within a class value error

Asked by At

I am calling function B within C in a class. See below.

class A(object):

   def __init__(
    self,
    intvl = 60 ):

   def B(self, ip):
   if ':' not in ip:
        numbers = list(map(int, ip.split('.')))
        return '{:02x}{:02x}'.format(*numbers)
    else:
        return ip

   def C(self, df):
     df = df.withColumn('ip_new', self.C(df['ip'])) 
     return df

I get the error

ValueError: Cannot convert column into bool: please use '&' for 'and', '|' for 'or', '~' for 'not' when building DataFrame boolean expressions.

Tried to put into udf and get pickling error. Can someone help?

Related Questions