Is there a way to remove a sub-attribute from a json column through spark sql

Question

I have a table in which there is column with nested json data. Need to remove a attribute from that json through spark sql

Checked on basic spark json function but not getting a way to do it

Answer 1

Assuming you read in a JSON file and print the schema you are showing us like this:

val df = sqlContext.read().json("/path/to/file").toDF();
    df.registerTempTable("df");
    df.printSchema();

Then you can select nested objects inside a struct type like so...

val app = df.select("app");
        app.registerTempTable("app");
        app.printSchema();
        app.show();

val appName = app.select("element.appName");
        appName.registerTempTable("appName");
        appName.printSchema();
        appName.show();

val trimmedDF = appName.drop("firstname")