Problems calling IConnectionPointImpl interface from C++ invoked via modal WinForms

Question

We have a native C++ application which supports some VBA macros of various types over COM. One of these types, VBAExtension, registers itself with the core C++ application, resulting in an instance of (a class derived from) IConnectionPointImpl<Extension, &DIID_IExtensionEvents, CComDynamicUnkArray>. This works fine; both core and other VBA macros can access the methods on IExtensionEvents, given an appropriate VBAExtension object.

We also have a .NET assembly (written in C#) which is also loaded in to the core application at run-time. For historical reasons, the assembly is loaded in by an auto-running VBA macro; then, when the user presses a particular button, another VBA macro runs the main entry point of the assembly, which brings up a System.Windows.Forms dialog for further interaction.

That’s the setup. I’m seeing some weird behaviour accessing the VBAExtension methods from within the .NET assembly. Specifically, I am running the following code from various locations in the assembly:

foreach (VBAExtension ve in app.Extensions)
{
    System.Diagnostics.Debug.Print("Ext: " + ve.Name);
}

If I run it from the constructor of the assembly’s main object; or from the assembly’s main entry point (before the dialog is displayed), everything is fine – I get the names of the VBAExtensions printed out.

However, if I run the same code from a command kicked off by a button in the assembly’s (modal - we're calling form.ShowDialog()) WinForm, the ve.Names are all blank. The pDispatch->Invoke call made by the IConnectionPointImpl subclass succeeds (returns S_OK), but does not set any return vars.

If I change the dialog to be non-modal (invoked with form.Show()), then the names work again. The modality (modalness?) of the form appears to affect whether the IConnectionPointImpl calls succeed.

Anyone know what's going on?

Edit: Since first posting, I've demonstrated that it's not the invoking call stack that matters; instead, it's whether the call is made from a modal dialog. I have updated the main text.

Edit 2: Per Hans Passant's answer, here are the answers to his diagnostic questions:

• As expected, in the good (modeless) case there is no error if I rename the VBA event handler. The call simply returns no data.
• I've put a MsgBox call into the VBA handler; it displays in the modeless case, but does not in the modal case. Ergo, the handler is not executed in the modal case.
• By use of Err, I can tell that if we hit an exception in the VBA handler we get a VBA error dialog. Once clearing that, the C++ Invoke call has 0x80020009 ("Exception occurred") as return code, and pExcepInfo filled in with generic failure values (VBA has swallowed the actual details)
• The event does not fire on the second display of the modal dialog, either immediately following the first dialog or during a second invocation of the C# add-in.

I'll try to dig into our message loops as a next step.

Answer 1

There are excessively few hard facts in this question to build an answer on. Could be something very simple, could be a nasty memory corruption problem or an obscure dependency inside the VBA interpreter on the thread state. The rough diagnostic is that the VBA event handler simply did not run. That is not an uncommon accident in general, the declarative style used in Basic to declare event handlers leave very few good way to ever diagnose subscription problems. Many a VBA programmer has lost clumps of hair trying to troubleshoot a "why did the event handler not run" problem like this one.

Gather some hard facts first and add them to your question:

• First verify that your C++ code can actually see that there is no event handler at all. Use the good version, rename the event handler. Expectation is that you don't, raising an event that the sink doesn't subscribe is not an error.
• Verify that the event handler actually executed in the bad version. Have it do something else than assign the BSTR argument, some you can easily see like a file on disk.
• Verify that you can properly diagnose an exception in the event handler. Assign the Err object and verify that your C++ code generates a proper diagnostic. Note that your IDispatch::Invoke() call passes NULL for pExcepInfo, not a good way to generate a diagnostic.
• Check if the event runs the second time you display a window, if it does then you have an execution order problem.

Focusing a bit on ShowDialog(). This method does have plenty of side-effects. The first thing that no longer works is the message loop in your C++ code. It is now the .NET message loop that dispatches messages. Look at yours for side-effects, doing more work than simply GetMessage/DispatchMessage(). That work no longer gets done. Also search your codebase for PostThreadMessage(), those messages fall on the floor when the .NET code pumps.

And keep in mind that your native C++ code loses control when the C# code calls ShowDialog(). It doesn't regain control until you close the window. That can trigger a simple order-of-execution problem, your C++ code should not do anything important after does whatever it does to get the C# code running.