The program im installing using Wix requires a Database Server for its database, so I would like to give the user the option to run the Server setup before continuing with the setup.
I know how to chain the setup using CustomActions, but how can I run that CustomAction before continuing with the rest of the setup, and showing some dialog in between?
On
Every action and dialog has it's sequence number. List of standard numbers you can get here. In your custom actions it's Sequence attribute. So just plan you dialogs/actions according to that numbers. It's common for all windows installers, so would work for wixsharp too.
Also I'll recommend you to use Orca while working with sequence numbers. It's powerfull tool that will help you to inspect/debug you msi.
Speaking from plenty of experience here from needing to do the same for multiple different products. My recommendation is to put your database configuration into the first startup of your application.
Software development and management being what it is sometimes this is not possible. So, what I would do is run your custom action as part of the InstallUISequence that way your database is installed and configured before the rest of your install is processing. This DOES mean that you will not be able to get your application installed silently unless the database portion is already configured since the InstallUISequence does not run in a silent install.