I'm trying to run a notebook in Microsoft Fabric using a spark pool but keep getting the below error:
I have tried to go the notebook in the stacktrace but it doesn't look like it's running. I can't seem to find where the actively running jobs are listed in Fabric so I can cancel them. I've tried to add a delay to a new cell at the end of the notebook like so:
import time
time.sleep(3)
However, this doesn't seem to make a difference (though I'm probably doing this wrong). The only solution I've found is to wait for the spark pool to stop completely and start it up again from scratch. There must be a faster/more efficient way to deal with this though!
Any help or advice that anyone can provide would be very much appreciated! I am very very new to Fabric though I'm hoping a Spark solution might work as well...
Many thanks!