Mask the data coming from Kafka stream

379 views Asked by At

I am using spark Structured streaming to stream data from kafka, which gives me dataframe with below schema

Column     Type
key        binary
value      binary
topic      string
partition  int
offset     long
timestamp  long
timestampType   int

Value Colum is coming here in binary format buts its actually a json string with a struct type and the requirement is to read the json struct and mask few fields coming in it and write the data.

0

There are 0 answers