Background Worker: Make sure that ProgressChanged method has finished before executing RunWorkerCompleted

Question

Let's assume I'm using a Background Worker and I've the following methods:

private void bw_DoWork(object sender, DoWorkEventArgs e)
{
    finalData = MyWork(sender as BackgroundWorker, e);
}

private void bw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
    int i = e.ProgressPercentage; // Missused for i
    Debug.Print("BW Progress Changed Begin, i: " + i + ", ThreadId: " + Thread.CurrentThread.ManagedThreadId);
    // I use this to update a table and an XY-Plot, so that the user can see the progess.
    UpdateGUI(e.UserState as MyData);
    Debug.Print("BW Progress Changed End,   i: " + i + ", ThreadId: " + Thread.CurrentThread.ManagedThreadId);
}

private void bw_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
    if ((e.Cancelled == true))
    {
        // Cancelled
    }
    else if (!(e.Error == null))
    {
        MessageBox.Show(e.Error.Message);
    }
    else
    {        
        Debug.Print("BW Run Worker Completed Begin, ThreadId: " + Thread.CurrentThread.ManagedThreadId);
        // I use this to update a table and an XY-Plot, 
        // so that the user can see the final data.
        UpdateGUI(finalData);
        Debug.Print("BW Run Worker Completed End,   ThreadId: " + Thread.CurrentThread.ManagedThreadId);
    }
}

Now I would assume that the bw_ProgressChanged method has finished before the bw_RunWorkerCompleted method is called. But that's not the case and I don't understand why?

I get the following output:

Worker, i: 0, ThreadId: 27
BW Progress Changed Begin, i: 0, ThreadId: 8
BW Progress Changed End,   i: 0, ThreadId: 8
Worker, i: 1, ThreadId: 27
BW Progress Changed Begin, i: 1, ThreadId: 8
BW Progress Changed End,   i: 1, ThreadId: 8
Worker, i: 2, ThreadId: 27
BW Progress Changed Begin, i: 2, ThreadId: 8
BW Run Worker Completed Begin, ThreadId: 8
BW Run Worker Completed End,   ThreadId: 8
A first chance exception of type 'System.InvalidOperationException' occurred in mscorlib.dll
ERROR <-- Collection was modified; enumeration operation may not execute.
ERROR <-- NationalInstruments.UI.WindowsForms.Graph.ClearData()

The MagagedID 8 is the Main Thread and 27 is a Worker Thread. I can see this in the Debug / Windows / Threads.

If I don't call UpdateGUI int the bw_ProgressChanged method then no error occurs. But then the user doesn't see any progress in the table and the XY-Plot.

EDIT

The MyWork method looks like that:

public MyData[] MyWork(BackgroundWorker worker, DoWorkEventArgs e)
{
     MyData[] d = new MyData[n];
     for (int i = 0; i < n; i++) 
         d[i] = null;
     for (int i = 0; i < n; i++)
     {
         if (worker.CancellationPending == true)
         {
             e.Cancel = true;
             break;
         }
         else
         {
             d[i] = MyCollectDataPoint(); // takes about 1 to 10 seconds
             Debug.Print("Worker, i: " + i + ", ThreadId: " + Thread.CurrentThread.ManagedThreadId)
             worker.ReportProgress(i, d);
         }
     }
     return d;
}

and the UpdateGUI method looks like that:

private void UpdateGUI(MyData d)
{
   UpdateTable(d); // updates a DataGridView
   UpdateGraph(d); // updates a ScatterGraph (NI Measurement Studio 2015)
}

If I don't call UpdateGraph method it works as aspected. So the ProgressChanged method has finished before executing RunWorkerCompleted.

So I guess the problem is the combination of the ScatterGraph from NI Measurement Studio 2015 and the BackgroundWorker. But I don't understand why?

The UpdateGraph method looks like that:

private void UpdateGraph(MyData d)
{
    plot.ClearData();
    plot.Plots.Clear(); // The error happens here (Collection was modified; enumeration operation may not execute).
    int n = MyGetNFromData(d);        
    for (int i = 0; i < n; i++)
    {
        ScatterPlot s = new ScatterPlot();
        double[] xi = MyGetXiFromData(d, i);
        double[] yi = MyGetYiFromData(d, i);
        s.XAxis = plot.XAxes[0];
        s.YAxis = plot.YAxes[0];
        s.LineWidth = 2;
        s.LineColor = Colors[i % Colors.Length];
        s.ProcessSpecialValues = true;
        s.PlotXY(xi, yi);
        plot.Plots.Add(s);
    }
}

Edit 2

If I set a breakpoint in the bw_RunWorkerCompleted method then the call stack looks like that:

bw_RunWorkerCompleted
[External Code]
UpdateGraph // Line: plot.ClearData()
UpdateGUI
bw_ProgressChanged
[External Code]
Program.Main

and the first [External Code] block:

System.dll!System.ComponentModel.BackgroundWorker.OnRunWorkerCompleted(System.ComponentModel.RunWorkerCompletedEventArgs e) Unknown
[Native to Managed Transition]  
mscorlib.dll!System.Delegate.DynamicInvokeImpl(object[] args)   Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.InvokeMarshaledCallbackDo(System.Windows.Forms.Control.ThreadMethodEntry tme) Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(object obj) Unknown
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.InvokeMarshaledCallback(System.Windows.Forms.Control.ThreadMethodEntry tme)   Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.InvokeMarshaledCallbacks()    Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control caller, System.Delegate method, object[] args, bool synchronous) Unknown
System.Windows.Forms.dll!System.Windows.Forms.Control.Invoke(System.Delegate method, object[] args) Unknown
System.Windows.Forms.dll!System.Windows.Forms.WindowsFormsSynchronizationContext.Send(System.Threading.SendOrPostCallback d, object state)  Unknown
NationalInstruments.Common.dll!NationalInstruments.Restricted.CallbackManager.CallbackDispatcher.SynchronousCallbackDispatcher.InvokeWithContext(System.Delegate handler, object sender, System.EventArgs e, System.Threading.SynchronizationContext context, object state) Unknown
NationalInstruments.Common.dll!NationalInstruments.Restricted.CallbackManager.a(NationalInstruments.Restricted.CallbackManager.CallbackDispatcher A_0, object A_1, object A_2, System.EventArgs A_3)    Unknown
NationalInstruments.Common.dll!NationalInstruments.Restricted.CallbackManager.RaiseEvent(object eventKey, object sender, System.EventArgs e)    Unknown
NationalInstruments.Common.dll!NationalInstruments.ComponentBase.RaiseEvent(object eventKey, System.EventArgs e)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.XYCursor.OnAfterMove(NationalInstruments.UI.AfterMoveXYCursorEventArgs e) Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.XYCursor.a(object A_0, NationalInstruments.Restricted.ControlElementCursorMoveEventArgs A_1)  Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.OnAfterMove(NationalInstruments.Restricted.ControlElementCursorMoveEventArgs e)  Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.a(NationalInstruments.UI.Internal.CartesianPlotElement A_0, double A_1, double A_2, int A_3, bool A_4)   Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.MoveCursorFreely(double xValue, double yValue, bool isInteractive, NationalInstruments.UI.Internal.XYCursorElement.Movement movement)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.MoveCursorXY(double xValue, double yValue, bool isInteractive)   Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.ResetCursor()    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYCursorElement.a(object A_0, NationalInstruments.Restricted.ControlElementEventArgs A_1)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.PlotElement.OnDataChanged(NationalInstruments.Restricted.ControlElementEventArgs e)  Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.PlotElement.OnDataChanged()  Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.CartesianPlotElement.a(object A_0, NationalInstruments.UI.Internal.PlotDataChangedEventArgs A_1) Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYDataManager.a(NationalInstruments.UI.Internal.PlotDataChangedEventArgs A_0)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYDataManager.a(NationalInstruments.UI.Internal.PlotDataChangeCause A_0, int A_1)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.XYDataManager.ClearData(bool raiseDataChanged)   Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.CartesianPlotElement.ClearData(bool raiseDataChanged)    Unknown
NationalInstruments.UI.dll!NationalInstruments.UI.Internal.PlotElement.ClearData()  Unknown
NationalInstruments.UI.dll!NationalInstruments.Restricted.XYGraphManager.ClearData()    Unknown
NationalInstruments.UI.WindowsForms.dll!NationalInstruments.UI.WindowsForms.Graph.ClearData()   Unknown

Answer 1

Well, you have hard evidence that the RunWorkerCompleted event runs while the ProgressChanged event runs. That is not normally possible of course, they are supposed to run on the same thread.

There are two possible ways that this can happen anyway. The more obvious one is that the event handlers don't actually run on the UI thread. Which is fairly common mishap, although you tend to notice from the InvalidOperationException that causes. That exception is however not always reliably raised, it uses a heuristic. Beware that your UpdateGraph() method is not so likely to trip it since it doesn't appear to use a standard .NET control.

Diagnosing this mishap is otherwise easy, just set a breakpoint on the event handler and use the Debug > Windows > Threads debugging window to verify it runs on the main thread. Using Debug.Print to display the value of Thread.CurrentThread.ManagedId can help ensure that all invocations run on the UI thread. You fix it by ensuring that the RunWorkerAsync() call is executed on the main thread.

And then there is the rat trap of a re-entrancy bug, it occurs when ProgressChanged does something that gets the UI dispatcher running again. Tends to be about as hard to debug as a threading race. Three basic ways that can happen:

• using the infamous Application.DoEvents()

• its evil step-sister, ShowDialog(). ShowDialog is DoEvents in disguise, it pretends to be less lethal by disabling the windows of the UI. Which tends to work okay, except when you run code that isn't activated by the UI. Like this code. Beware that you do appear to use MesssageBox.Show() for debugging, never a good idea. Always favor breakpoints and Debug.Print() to avoid this trap.

• doing something that blocks the UI thread, like lock, Thread.Join(), WaitOne(). Blocking an STA thread is formally illegal, high odds for deadlock, so the CLR does something about it. It pumps its own message loop to ensure deadlock is avoided. Yes, like DoEvents does, it does some filtering to avoid the nasty cases. But not otherwise enough for this code. Beware that this might be done by code you did not write, like that Graph control.

Diagnose a re-entrancy bug by setting a breakpoint on the RunWorkerCompleted event. You should see the ProgressChanged event handler back, buried deep in the call stack. And the statement that causes the re-entrancy. If the trace doesn't help you figure it out then post it in your question.

Answer 2

The biggest flaw is your assumption below is wrong.

Now I would assume that the bw_ProgressChanged method has finished before the bw_RunWorkerCompleted method is called. But that's not the case and I don't understand why?

Do not get caught up into mentally serializing the flow of logic. With WinForms/WPF, you have two completely independent and asynchronous events occurring. You have the BGW sending a request (via worker.ReportProgress) to the UI to perform a progress update. The UI thread must receive that request and schedule when the bw_ProgressChanged event runs.

Independent of that the BGW (via myWork) decides to terminate, perhaps by fully completing the job, or because an untrapped exception was thrown, or perhaps the end-user desired to cancel the work at a given instance. This then sends a request to the UI thread to run the bw_RunWorkerCompleted method. Once again the UI must schedule it on its many list of things to do.