how to use assembly to get the result of a __stdcall function that returns float

Question

I have an assembly routine that calls, in a generic way, a function known to use the stdcall convention and return a float. This function is being used by a marshalling framework to expose stdcall functions to a scripting language.

Background

Here's the function using GNU inline assembly that compiles on MinGW 4.3, Win32:

inline uint64_t stdcall_invoke_return_float(int args_size_bytes,
                                            const char * args_ptr,
                                            void * func_ptr)
{
    uint64_t result;
    assert(
        0 == args_size_bytes % 4
            || !"argument size must be a multiple of 4 bytes");
#if defined(__GNUC__)
    asm
    (
        /* INPUT PARAMS:  %0 is the address where top of FP stack to be stored
         *                %1 is the number of BYTES to push onto the stack, */
        /*                   and during the copy loop it is the address of */
        /*                   the next word to push */
        /*                %2 is the base address of the array */
        /*                %3 is the address of the function to call */
            "testl %1, %1    # If zero argument bytes given, skip \n\t"
            "je    2f        # right to the function call.        \n\t"
            "addl  %2, %1\n"
        "1:\n\t"
            "subl  $4, %1    # Push arguments onto the stack in   \n\t"
            "pushl (%1)      # reverse order. Keep looping while  \n\t"
            "cmp   %2, %1    # addr to push (%1) > base addr (%2) \n\t"
            "jg    1b        # Callee cleans up b/c __stdcall.    \n"
        "2:\n\t"
            "call  * %3      # Callee will leave result in ST0    \n\t"
            "fsts  %0        # Copy 32-bit float from ST0->result"
        : "=m" (result)
        : "r" (args_size_bytes), "r" (args_ptr), "mr" (func_ptr)
        : "%eax", "%edx", "%ecx" /* eax, ecx, edx are caller-save */, "cc"
    );
#else
#pragma error "Replacement for inline assembler required"
#endif
    return result;
}

This is just a little glue to make it easier to write test cases:

template<typename FuncPtr, typename ArgType>
float float_invoke(FuncPtr f, int nargs, ArgType * args)
{
    uint64_t result = stdcall_invoke_return_float(
        nargs * sizeof(ArgType),
        reinterpret_cast<const char *>(args),
        reinterpret_cast<void *>(f)
    );
    return *reinterpret_cast<float *>(&result);
}

Now I have some test cases that invoke this function:

__stdcall float TestReturn1_0Float()
{ return 1.0f; }

__stdcall float TestFloat(float a)
{ return a; }

__stdcall float TestSum2Floats(float a, float b)
{ return a + b; }

static const float args[2] = { 10.0f, -1.0f };

assert_equals(1.0f, float_invoke(TestReturn1_0Float, 0, args)); // test 1
assert_equals(10.0f, float_invoke(TestFloat, 1, args));         // test 2
assert_equals(-1.0f, float_invoke(TestFloat, 1, args + 1));     // test 3
assert_equals(9.0f, float_invoke(TestSumTwoFloats, 2, args));   // test 4

Problem

Randomly, test 3 is giving me garbage output instead of returning -1.0.

I'm wondering if I'm

• failing to preserve some state before the call instruction?
• messing up some state with the fsts instruction?
• fundamentally misunderstanding how to get a float value from a stdcall function that returns float????

All help greatly appreciated.

Answer 1

Lacking a windows machine, I can't fully test this; on Linux, the following gets me the return code of a float function:

extern float something(int);

#include 
#include 

int main(int argc, char **argv)
{
    int val = atoi(argv[1]);
    float ret;

    asm("pushl %1\n\t"
        "call * %2\n\t"
        "addl $4, %%esp"
       : "=t"(ret)
       : "r"(val), "r"(something)
       : "%eax", "%ecx", "%edx", "memory", "cc");

    printf("something(%d) == %f\n", val, ret);
    return 0;
}

The key is the use of the "=t"(ret) constraint - that gets the top of the floating point stack, see Machine Constraints (from the gcc manual). If Windows stdcall returns float results in ST(0) as well, that should work, no need for fld/fst as the compiler can do those for you if necessary.

You also need to specify the memory and cc clobbers when you call functions from within inline assembly.

Answer 2

You are permitting a memory reference for the function pointer, GCC is liable to construct a reference relative the stack pointer on the incorrect assumption that the inline assembly does not change it.