Given the following dataset:
| title | start | end | bla | 10 | 30
I would like to find the difference (start - end) between the two numbers and set them into a new column, so that it looks like:
| title | time_spent | | bla | 20 |
The data is of type
dataset = dataset.withColumn("millis spent: ", col("end") - col("start")).as("Time spent");
I expected this to work, as i saw in this question, but it does ot, maybe because that thread is about DataFrames and not DataSets, or maybe because Scala allows it where as in Java is illegal?