Unprotected error when calling nested Lua C library function with lua_pcall()

Question

I'm writing a C++ library function that takes in two tables and takes the values from the tables and puts them into a vector. To reduce clutter I've also made a helper function that will read the values in one of the tables and performs some type checking. However when calling luaL_error()in one of these nested functions I get the following in the terminal output: PANIC: unprotected error in call to Lua API () [1] 18449 IOT instruction (core dumped) build/bin/run Where build/bin/run is the executable. This function is called via lua_pcall()and so any errors should be caught, but that seems to not be the case.

Here is the main function that is added to the library:

    static int newMesh(lua_State* L){
        if(!lua_checkstack(L, 20))
            luaL_error(L, "Not enough stack space");
        if( !lua_istable(L, 1) )
            luaL_error(L, "Expected table of vertecies");
        if( !lua_istable(L, 2) )
            luaL_error(L, "Expected table of vertecies");
        
        std::vector<float> verticies;
        std::vector<unsigned int> indicies;
        
        /* Getting vao */
        unsigned int i = 0;

        lua_pushnil(L);
        lua_next(L, 1); 
        indicies.push_back(i++);
        verticies = getVerticies(L); /*Here is one of the helper functions*/
        std::pair<unsigned int, GLenum> format = {verticies.size(), GL_FLOAT};

        while(lua_next(L, 1) != 0){
            indicies.push_back(i++);
            auto vertex = getVerticies(L); /* Its called again here*/
            
            if(vertex.size() != std::get<0>(format))
                luaL_error(L, "Vertex does not follow format %d floats", std::get<0>(format));
            
            verticies.insert(verticies.end(), vertex.begin(), vertex.end());    
        }
    
        /* Getting materials */
        vector<Modulus::Material> materials;

        lua_pushnil(L);
        while(lua_next(L, 2) !=0){
            materials.emplace_back( getMaterial(L));        
            
            lua_pop(L, 1);
        }

        /* Rough size estimate */
        size_t size =     sizeof(GL_FLOAT) * verticies.size() 
                        + sizeof(unsigned int) * indicies.size() 
                        + sizeof(Modulus::Material) * materials.size()
                        + sizeof(unsigned int) + sizeof(GLenum) 
                        + sizeof(GLenum);

        Modulus::Mesh* newMesh = (Modulus::Mesh*)lua_newuserdata(L, size);
        newMesh= new Mesh(verticies, indicies, materials, vector<pair<unsigned int, GLenum>>{format}, GL_TRIANGLES);
        gLuaMeshes.push_back(newMesh);
        
        luaL_setmetatable(L, "Modulus.mesh");

        return 1;
    }

And here's the helper function in question:

    static std::vector<float> getVerticies(lua_State* L, int tableIndex = 4){
        std::vector<float> verticies;
        
        lua_pushnil(L);
        while(lua_next(L, tableIndex) != 0){
            int success;
            verticies.push_back( lua_tonumberx(L, -1, &success));
            if(!success){
                                /* Here is where the error is thrown*/
                luaL_error(L, "Expected number for vertex attribute"); 
                return std::vector<float>();
            }
            
            lua_remove(L, -1);
        }
        
        lua_remove(L, -1);

        return verticies;
    }

And here is where the script that calls the function is run:

bool loadFile(string path){
        int result =  luaL_loadfile(mLuaContext, path.c_str()) | lua_pcall(mLuaContext, 0, LUA_MULTRET, 0);
    if(result != LUA_OK){
        error("%s", lua_tostring(mLuaContext, -1));
            return false;
        }
    return true;
}

I've tried putting the helper function call in a try-catch block where it throws the error string and luaL_error is called in the main function, but the issue still occurs.

Answer 1

Okay so i found the issue.

In the function caller theres an error() function that's called if the lua_pcall() returns an error state.

Here's the definition for error():

void error (const char *fmt, ...){
    va_list argp;
    va_start(argp, fmt);
    luaL_error(mLuaContext, fmt, argp); /* Here's the issue.. */
    va_end(argp);
}

The issue is that i call luaL_error() again with the error string. so i tried printing the error string at the top of the stack instead of calling my error() function, ant it works fine.

    bool loadFile(string path){
        int result =  luaL_loadfile(mLuaContext, path.c_str())
                    | lua_pcall(mLuaContext, 0, LUA_MULTRET, 0);
        if(result != LUA_OK){
            std::cerr << lua_tostring(mLuaContext, -1) << std::endl;
            //error("%s", lua_tostring(mLuaContext, -1));
            return false;
        }
    return true;
}

So having nested function calls was not the issue, it was me not understanding the functions i wrote.. Oh well