Dynamic casts returns null when library with C++ python extensions is used as a plugin on RHEL5

Question

I have a library with C++ python extensions (C++ calls python which in turn calls C++) using boost::python and python libraries (this is messy, but a lot of it is legacy) which when tested standalone works correctly. In particular, a certain dynamic_cast works correctly.

But when the library is packaged for use as a plugin on RHEL5 using gcc 4.1.2 with an external application, the dynamic_cast returns NULL resulting in the application not working as expected. On Windows (tested Vista 64 bit using Visual Studio 2005 and 2008) it works fine. When I debugged using ddd for instance, I am able to see that the pointer before casting has the right type_name (slightly mangled by compiler as is usual, I suppose?). Any specific debugging tips here will also be of help.

A reinterpret_cast solved the problem. While this will be certainly baulked at, I am at a loss about how to proceed, esp. since this could be due to issues with the external app. It is a convoluted mess and almost seems futile, but if it can help here is some sample code. The following C++ snippet creates a "smart_handle" to queue certain python commands stored in string "input". The string IMPORT imports locations and definitions of some functions that are called by boost::python::exec(..) in the function call py_api::execute_py_command:

boost::shared_ptr<my_base_class> 
         processor(new my_derived_class());
std::map<std::string, smart_handle> context;
context.insert(std::make_pair<std::string, smart_handle>("default_queue", 
               make_smart_handle(processor)));
const std::string py_command =
       IMPORT + 
       "namesp.dialects.cpython.set_command_queue('default', default_queue)\n" + 
       input;
if( !py_api::execute_py_command(py_command, context) ) {
  return false; 
}

The make_smart_handle is defined as:

template <typename type_t> 
const smart_handle make_smart_handle(const boost::shared_ptr<type_t>& ptr) {
   if( !ptr ) {
      return smart_handle();
   }
   return smart_handle(new detail::smart_handle_weak_impl<type_t>(ptr));
}

The function set_command_queue is defined in a python __init__.py as:

import func1 
import func2
import func3
import func4

COMMAND_QUEUE_MAP = {}
def set_command_queue(queue_name, object):
COMMAND_QUEUE_MAP[queue_name] = object
def get_command_queue(queue_name = 'default'):
return COMMAND_QUEUE_MAP[queue_name]

Now the actual python functions func1, func2, etc. defined in separate python files calls C++ functions defined under namespace "namesp". The very first line of these C++ functions is to recover the "smart_handle" to the "queue" by:

boost::shared_ptr<my_base_class> queue = smart_handle_cast<my_base_class>(handle).lock();

It is in the above function smart_handle_cast that the dynamic_cast is used which returns NULL, when the library is used as a plugin in an external app. Using reinterpret_cast allows it to work correctly. The smart_handle_cast returns a const boost::weak_ptr. For those interested, here is the defintion of the smart_handle_cast<..>() function:

template <typename type_t> 
const boost::weak_ptr<type_t> smart_handle_cast(const smart_handle& handle, bool 
      throw_if_failure) {
   if( !handle.is_valid() ) {
       if( throw_if_failure ) {
          throw smart_handle::bad_handle("Bad handle, attempting to access an invalid 
                                          handle");
       }
       //-No throw version returns a non-initialized weak pointer
       return boost::weak_ptr<type_t>();
   }
   //-This line fails at run time and returns null.
   const detail::smart_handle_weak_impl<type_t>* casted = dynamic_cast<const 
                   detail::smart_handle_weak_impl<type_t>* >(handle.impl());
   if( !casted ) {
       if( throw_if_failure ) {
           throw smart_handle::bad_handle_cast("Bad handle cast, attempting to \
              convert to incorrect pointee type");
       }
       //-No throw version returns a non-initialized weak pointer
       return boost::weak_ptr<type_t>();
   }
   return casted->pointee;
}

Answer 1

Take a look at similar question and GCC FAQ

If you use dlopen to explicitly load code from a shared library, you must do several things. First, export global symbols from the executable by linking it with the "-E" flag (you will have to specify this as "-Wl,-E" if you are invoking the linker in the usual manner from the compiler driver, g++). You must also make the external symbols in the loaded library available for subsequent libraries by providing the RTLD_GLOBAL flag to dlopen. The symbol resolution can be immediate or lazy.