Step execution of release code / post-mortem debugging (VS/C++)

Question

Is there any sense to step-execute release code? I noticed that some lines of code are omitted, i.e. some method calls. Also variable preview doesn't show some variables and shows invalid (not real) values for some others, so it's all quite misleading.

I'm asking this question, because loading WinDbg crashdump file into Visual Studio brings the same stack and variables partial view as step-execution. Are there any way to improve crashdump analyze experience, except recompiling application without optimalizations?

Windows, Visual Studio 2005, unmanaged C++

Answer 1

Recompile just the file of interest without optimisations :)

In general:

• Switch to interleaved disassembly mode. Single-stepping through the disassembly will enable you to step into function calls that would otherwise be skipped over, and make inlined code more evident.
• Look for alternative ways of getting at values in variables the debugger is not able to directly show you. If they were passed in as arguments, look up the callstack - you will often find they are visible in the caller. If they were retrieved via getters from some object, examine that object; glance over the assembly generated by the code that calculates them to work out where they were stored; etc. If all else fails and disabling optimisations / adding a printf() distorts timings sufficiently to affect debugging, add a dummy global variable and set it to the value of interest on entry to the section of interest.

Answer 2

Yes - if you have the .pdb for the build, and the .dmp file from the crash, then you can open the debugger on the exact point of failure, and examine the state of your app at that point.

As several have noted - some variables will be optimized away, but if you're mildly creative / inquisitive, you'll find ways to obtain those values.

You can build in a root crash handler for your code to generate a .dmp file automatically which works on all Windows flavors (assuming you are creating a Windows app) using something like the following:

// capture the unhandled exception hook - we will create a mini dump for ourselves
// NOTE: according to docs, if a debugger is present, this API won't succeed (ie. debug builds ignore this)
MiniDumper::Install(
    true,
    filename,
    "Please send a copy of this file, along with a brief description of the problem, to [insert your email address here] so that we might fix this issue."
);

The above would require the MiniDumper class I wrote, below:

#pragma once
#include <dbghelp.h>
#include "DynamicLinkLibrary.h"
#include "FileName.h"

//////////////////////////////////////////////////////////////////////////
// MiniDumper
//
//  Provides a mechanism whereby an application will generate its own mini dump file anytime
//  it throws an unhandled exception (or at the client's request - see GenerateMiniDump, below).
//
//  Warning: the C-runtime will NOT invoke our unhandled handler if you are running a debugger
//  due to the way that the SetUnhandledExceptionFilter() API works (q.v.)
//
//  To use this facility, simply call MiniDumper::Install - for example, during CWinApp initialization.
//
//  Once this has been installed, all current and future threads in this process will be covered.
//  This is unlike the StructuredException and CRTInvalidParameter classes, which must be installed for
//  for each thread for which you wish to use their services.
//
class MiniDumper
{
public:
    // install the mini dumper (and optionally, hook the unhandled exception filter chain)
    // @param filename is the mini dump filename to use (please include a path)
    // @return success or failure
    // NOTE: we can be called more than once to change our options (unhook unhandled, change the filename)
    static bool Install(bool bHookUnhandledExceptionFilter, const CFilename & filenameMiniDump, const CString & strCustomizedMessage, DWORD dwMiniDumpType = MiniDumpNormal)
    {
        return GetSingleton().Initialize(bHookUnhandledExceptionFilter, filenameMiniDump, strCustomizedMessage, dwMiniDumpType); 
    }

    // returns true if we've been initialized (but doesn't indicate if we have hooked the unhandled exception filter or not)
    static bool IsInitialized() { return g_bInstalled; }

    // returns true if we've been setup to intercept unhandled exceptions
    static bool IsUnhandledExceptionHooked() { return g_bInstalled && GetSingleton().m_bHookedUnhandledExceptionFilter; }

    // returns the filename we've been configured to write to if we're requested to generate a mini dump
    static CFilename GetMiniDumpFilename() { return g_bInstalled ? GetSingleton().m_filenameMiniDump : ""; }

    // you may use this wherever you have a valid EXCEPTION_POINTERS in order to generate a mini dump of whatever exception just occurred
    // use the GetExceptionInformation() intrinsic to obtain the EXCEPTION_POINTERS in an __except(filter) context
    // returns success or failure
    // DO NOT hand the result of GenerateMiniDump to your __except(filter) - instead use a proper disposition value (q.v. __except)
    // NOTE: you *must* have already installed MiniDumper or this will only error
    static bool GenerateMiniDump(EXCEPTION_POINTERS * pExceptionPointers);

private:

    // based on dbghelp.h
    typedef BOOL (WINAPI * MINIDUMPWRITEDUMP_FUNC_PTR)(
        HANDLE hProcess, 
        DWORD dwPid, 
        HANDLE hFile, 
        MINIDUMP_TYPE DumpType,
        CONST PMINIDUMP_EXCEPTION_INFORMATION ExceptionParam,
        CONST PMINIDUMP_USER_STREAM_INFORMATION UserStreamParam,
        CONST PMINIDUMP_CALLBACK_INFORMATION CallbackParam
        );

    // data we need to pass to our mini dump thread
    struct ExceptionThreadData
    {
        ExceptionThreadData(EXCEPTION_POINTERS * exceptionPointers, bool bUnhandled, DWORD threadID = ::GetCurrentThreadId())
            : pExceptionPointers(exceptionPointers)
            , dwThreadID(threadID)
            , bUnhandledException(bUnhandled)
        {
        }

        EXCEPTION_POINTERS *    pExceptionPointers;
        DWORD                   dwThreadID;
        bool                    bUnhandledException;
    };

    // our unhandled exception filter (called automatically by the run time if we've been installed to do so)
    static LONG CALLBACK UnhandledExceptionFilter(EXCEPTION_POINTERS * pExceptionPointers);

    // creates a new thread in which to generate our mini dump (so we don't run out of stack)
    static bool ExecuteMiniDumpThread(EXCEPTION_POINTERS * pExceptionPointers, bool bUnhandledException);

    // thread entry point for generating a mini dump file
    static DWORD WINAPI MiniDumpThreadProc(LPVOID lpParam);

    // obtains the one and only instance
    static MiniDumper & GetSingleton();

    // flag to indicate if we're installed or not
    static bool g_bInstalled;

    // create us
    MiniDumper() 
        : m_pPreviousFilter(NULL)
        , m_pWriteMiniDumpFunction(NULL)
        , m_bHookedUnhandledExceptionFilter(false)
    {
    }

    // install our unhandled exception filter
    bool Initialize(bool bHookUnhandledExceptionFilter, const CFilename & filenameMiniDump, const CString & strCustomizedMessage, DWORD dwMiniDumpType);

    // generates a mini dump file
    bool GenerateMiniDumpFile(ExceptionThreadData * pData);

    // handle an unhandled exception
    bool HandleUnhandledException(ExceptionThreadData * pData);

    bool                            m_bHookedUnhandledExceptionFilter;
    CFilename                       m_filenameMiniDump;
    CString                         m_strCustomizedMessage;
    DWORD                           m_dwMiniDumpType;
    MINIDUMPWRITEDUMP_FUNC_PTR      m_pWriteMiniDumpFunction;
    LPTOP_LEVEL_EXCEPTION_FILTER    m_pPreviousFilter;
};

And its implementation:

#include "StdAfx.h"
#include "MiniDumper.h"

using namespace Toolbox;

//////////////////////////////////////////////////////////////////////////
// Static Members

bool MiniDumper::g_bInstalled = false;

// returns true if we were able to create a mini dump for this exception
bool MiniDumper::GenerateMiniDump(EXCEPTION_POINTERS * pExceptionPointers)
{
    // obtain the mini dump in a new thread context (which will have its own stack)
    return ExecuteMiniDumpThread(pExceptionPointers, false);
}

// this is called from the run time if we were installed to hook the unhandled exception filter
LONG CALLBACK MiniDumper::UnhandledExceptionFilter(EXCEPTION_POINTERS * pExceptionPointers)
{
    // attempt to generate the mini dump (use a separate thread to ensure this one is frozen & we have a fresh stack to work with)
    ExecuteMiniDumpThread(pExceptionPointers, true);

    // terminate this process, now
    ::TerminateProcess(GetCurrentProcess(), 0xFFFFFFFF);

    // carry on as normal (we should never get here due to TerminateProcess, above)
    return EXCEPTION_CONTINUE_SEARCH;
}

bool MiniDumper::ExecuteMiniDumpThread(EXCEPTION_POINTERS * pExceptionPointers, bool bUnhandledException)
{
    // because this may have been created by a stack overflow
    // we may be very very low on stack space
    // so we'll create a new, temporary stack to work with until we fix this situation
    ExceptionThreadData data(pExceptionPointers, bUnhandledException);
    DWORD dwScratch;
    HANDLE hMiniDumpThread = ::CreateThread(NULL, 0, MiniDumpThreadProc, &data, 0, &dwScratch);
    if (hMiniDumpThread)
    {
        VERIFY(::WaitForSingleObject(hMiniDumpThread, INFINITE) == WAIT_OBJECT_0);
        VERIFY(::GetExitCodeThread(hMiniDumpThread, &dwScratch));
        VERIFY(::CloseHandle(hMiniDumpThread));
        return AsBool(dwScratch);
    }

    return false;
}

DWORD WINAPI MiniDumper::MiniDumpThreadProc(LPVOID lpParam) 
{
    // retrieve our exception context from our creator
    ExceptionThreadData * pData = (ExceptionThreadData *)lpParam;

    // generate the actual mini dump file in this thread context - with our own stack
    if (pData->bUnhandledException)
        return GetSingleton().HandleUnhandledException(pData);
    else
        return GetSingleton().GenerateMiniDumpFile(pData);
}

bool MiniDumper::HandleUnhandledException(ExceptionThreadData * pData)
{
    // generate the actual mini dump file first - hopefully we get this even if the following errors
    const bool bMiniDumpSucceeded = GenerateMiniDumpFile(pData);

    // try to inform the user of what's happened
    CString strMessage = FString("An Unhandled Exception occurred in %s\n\nUnfortunately, this requires that the application be terminated.", CFilename::GetModuleFilename());

    // create the mini dump file
    if (bMiniDumpSucceeded)
    {
        // let user know about the mini dump
        strMessage.AppendFormat("\n\nOn a higher note, we have saved some diagnostic information in %s", m_filenameMiniDump.c_str());
    }

    // append any custom message(s)
    if (!IsEmpty(m_strCustomizedMessage))
        strMessage.AppendFormat("\n\n%s", m_strCustomizedMessage);

    // cap it off with an apology
    strMessage.Append("\n\nThis application must be terminated now.  All unsaved data will be lost.  We are deeply sorry for the inconvenience.");

    // let the user know that things have gone terribly wrong
    ::MessageBox(GetAppWindow(), strMessage, "Internal Error - Unhandled Exception", MB_ICONERROR);

    // indicate success or not
    return bMiniDumpSucceeded;
}

//////////////////////////////////////////////////////////////////////////
// Instance Members

MiniDumper & MiniDumper::GetSingleton() 
{
    static std::auto_ptr<MiniDumper> g_pSingleton(new MiniDumper);
    return *g_pSingleton.get(); 
}

bool MiniDumper::Initialize(bool bHookUnhandledExceptionFilter, const CFilename & filenameMiniDump, const CString & strCustomizedMessage, DWORD dwMiniDumpType)
{
    // check if we need to link to the the mini dump function
    if (!m_pWriteMiniDumpFunction)
    {
        try
        {
            // attempt to load the debug helper DLL
            DynamicLinkLibrary dll("DBGHelp.dll", true);

            // get the function address we need
            m_pWriteMiniDumpFunction = (MINIDUMPWRITEDUMP_FUNC_PTR)dll.GetProcAddress("MiniDumpWriteDump", false);
        }
        catch (CCustomException &)
        {
            // we failed to load the dll, or the function didn't exist
            // either way, m_pWriteMiniDumpFunction will be NULL
            ASSERT(m_pWriteMiniDumpFunction == NULL);

            // there is nothing functional about the mini dumper if we have no mini dump function pointer
            return false;
        }
    }

    // record the filename to write our mini dumps to (NOTE: we don't do error checking on the filename provided!)
    if (!IsEmpty(filenameMiniDump))
        m_filenameMiniDump = filenameMiniDump;

    // record the custom message to tell the user on an unhandled exception
    m_strCustomizedMessage = strCustomizedMessage;

    // check if they're updating the unhandled filter chain
    if (bHookUnhandledExceptionFilter && !m_bHookedUnhandledExceptionFilter)
    {
        // we need to hook the unhandled exception filter chain
        m_pPreviousFilter = ::SetUnhandledExceptionFilter(&MiniDumper::UnhandledExceptionFilter);
    }
    else if (!bHookUnhandledExceptionFilter && m_bHookedUnhandledExceptionFilter)
    {
        // we need to un-hook the unhandled exception filter chain
        VERIFY(&MiniDumper::UnhandledExceptionFilter == ::SetUnhandledExceptionFilter(m_pPreviousFilter));
    }

    // set type of mini dump to generate
    m_dwMiniDumpType = dwMiniDumpType;

    // record that we've been installed
    g_bInstalled = true;

    // if we got here, we must have been successful
    return true;
}

bool MiniDumper::GenerateMiniDumpFile(ExceptionThreadData * pData)
{
    // NOTE: we don't check this before now because this allows us to generate an exception in a different thread context (rather than an exception while processing an exception in the main thread)
    ASSERT(g_bInstalled);
    if (!g_bInstalled)
        return false;

    HANDLE hFile = ::CreateFile(m_filenameMiniDump.c_str(), GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hFile == INVALID_HANDLE_VALUE)
    {
        // indicate failure
        return false;
    }
    else
    {
        // NOTE: don't use exception_info - its a #define!!!
        Initialized<_MINIDUMP_EXCEPTION_INFORMATION> ex_info;
        ex_info.ThreadId = pData->dwThreadID;
        ex_info.ExceptionPointers = pData->pExceptionPointers;

        // generate our mini dump
        bool bStatus = FALSE != ((*m_pWriteMiniDumpFunction)(GetCurrentProcess(), GetCurrentProcessId(), hFile, (MINIDUMP_TYPE)m_dwMiniDumpType, &ex_info, NULL, NULL));

        // close the mini dump file
        ::CloseHandle(hFile);

        return bStatus;
    }
}

I apologize for the fact that this is not a drop-in solution. There are dependencies on other parts of my Toolbox library. But I think it would go a long way towards giving you the right idea as to how to build-in "capture a crash mini-dump" automatically from your code, which you can then combine with your .dsp files that you can make a normal part of your development cycle - so that when a .dmp comes in - you can fire up the debugger on it with your saved .pdb from your release build (which you don't distribute!) and you can debug the crash conditions quite easily.

The above code is an amalgam of many different sources - code snippets from debugging books, from MSDN documentation, etc., etc. If I have left out attribution I mean no harm. However, I do no believe that any of the above code is significantly created by anyone but myself.

Answer 3

The most important thing is to have the symbol files (*.pdb). You can generate them for release builds, by default they are not active.

Then you have to know that because of optimizations, code might get re-ordered, so debugging could look a bit jerky. Also some intermediate variables might have got optimized away. Generally speaking the behaviour and visibility of data might have some restrictions.

With Visual Studio C++ 2008 you can automatically debug the *.dmp files. I believe it also works for VS 2005. For older compilers I am afraid you´ll have to use WinDbg... (Also specify of course the *.pdb files for WinDbg, otherwise the info will be quite limited)

Answer 4

At least is not a IA64 dump...

There really isn't much you can do beyond having full dump and private symbols. Modern compilers have a field day with your code and is barely recognisable, specially if you add something like LTCG.

There are two things I found usefull:

1. Walk up the stack until you get a good anchor on what 'this' really points to. Most times when you are in an object method frame 'this' is unreliable because of registry optmizations. Usually several calls up the stack you get an object that has the correct address and you can navigate, member reference by member reference, until your crash point and have a correct value for 'this'

2. uf (Windbg's unassembly function command). This little helper can list a function dissasembly in a more manageable form than the normal dissasembly view. Because it follows jumps and code re-arranges, is easier to follow the logic of uf output.