Lua crashes when doing operations of userdata metatables in C

141 views Asked by At

I've stumbled upon a problem in Lua. I am using a library to bind C++ classes to Lua but the problem seems to be in Lua and not related to the library.

Steps to reproduce the crash:

  • Expose a C++ class to Lua as userdata and bind a function to a metamethod like __tostring.
  • In Lua create an instance of the class.
  • Call getmetatable with the instance as parameter.
  • Now pass the returned metatable to another C function that interacts with metafields. Like the function print calls luaL_tolstring which calls the metafield __tostring.
  • Lua crashes trying to call the class metamethod.

What causes the crash:

  • A table and not a userdata is passed to for example luaL_tolstring
  • luaL_tolstring calls __tostring metamethod.
  • The proxy function bound to __tostring is invoked in LuaBridge
  • LuaBridge crashes because it received a table and not the valid userdata holding a pointer to the class.

Example using the library LuaBridge

// Some example class with a tostring method.
class ExampleClass
{
public:
    ExampleClass() :
        a(0), b(0), c(0)
    {

    }

    string tostring() const
    {
        return "whatever";
    }

    int a, b, c;
};

lua_State* L = luaL_newstate();

luaL_openlibs(L);

// Expose example class and add a __tostring metamethod
luabridge::getGlobalNamespace(L)
    .beginClass<ExampleClass>("ExampleClass")
        .addConstructor<void(*) ()>()
        .addFunction("__tostring", &ExampleClass::tostring)
    .endClass()
    ;

// Create an instance of the example class.
// Call getmetatable on the instance and pass it to print.
luaL_dostring(L, "local t = ExampleClass(); print(getmetatable(t));");

lua_close(L);

Callstack

main.exe!luabridge::detail::Userdata::getPointer() Line 46  C++
main.exe!luabridge::detail::Userdata::get<ExampleClass>(lua_State * L, int index, bool canBeConst) Line 252 C++
main.exe!luabridge::detail::CFunc::CallConstMember<std::string (__cdecl ExampleClass::*)(void)const>::f(lua_State * L) Line 290 C++
main.exe!precallC(lua_State * L, StackValue * func, int nresults, int(*)(lua_State *) f) Line 506   C
main.exe!luaD_precall(lua_State * L, StackValue * func, int nresults) Line 570  C
main.exe!ccall(lua_State * L, StackValue * func, int nResults, int inc) Line 607    C
main.exe!luaD_callnoyield(lua_State * L, StackValue * func, int nResults) Line 628  C
main.exe!lua_callk(lua_State * L, int nargs, int nresults, __int64 ctx, int(*)(lua_State *, int, __int64) k) Line 1024  C
main.exe!luaL_callmeta(lua_State * L, int obj, const char * event) Line 867 C
main.exe!luaL_tolstring(lua_State * L, int idx, unsigned __int64 * len) Line 885    C
main.exe!luaB_print(lua_State * L) Line 29  C

My question is why does Lua invoke the bound C function of a userdata without passing a valid userdata but rather the metatable.

0

There are 0 answers