Exception handling in cython

Question

I have two .pyx files - bar.pyx and baz.pyx. I want to combine them into a single .so file.

In baz.pyx I have a function baz that should do some checks and raise an exception if something goes wrong. In bar.pyx I want to call baz() and expect exception to be raised with traceback printed.

Unfortunately, whatever I try, I get some other runtime errors.

Extensions in setup.py

[
    Extension(
        'testlib.baz', ['src/testlib/baz.pyx'],
    ),
    Extension(
        'testlib.foo', ['src/testlib/bar.pyx', 'src/testlib/baz.c'],
    ),
]

How tested

import testlib.foo
testlib.foo.foo_baz()

Variant 1

# baz.pyx

cdef public baz():
    raise ValueError

# bar.pyx

cdef extern baz()

def foo_baz():
    baz()   # Segmentation fault

Variant 2

# baz.pyx

cdef public int baz() except -1:
    PyErr_SetNone(ValueError)
    return -1

# bar.pyx

cdef extern int baz() except -1

def foo_baz():
    baz()   # SystemError: <built-in function foo_baz> returned NULL without setting an error

I can return some value from baz and raise an exception in foo_baz depending on return value, but I want as minimum logic to be present in bar.pyx.

# baz.pyx

cdef public int baz():
    return -1

# bar.pyx

cdef extern int baz()

def foo_baz():
    if baz() == -1:
        raise ValueError    # OK

Answer 1

To expand my comment a bit more: Cython does quite a bit of work when the module is imported setting up C global variables to give it quite access to thinks like C builtin types (including exceptions), string constants, and a few other things. In this case the line is translated to

__Pyx_Raise(__pyx_builtin_ValueError, 0, 0, 0);

and if __pyx_builtin_ValueError isn't initialized then it all goes wrong.

The upshot is that Cython code (even public Cython code) isn't actually standalone but is part of a module and that does rely on the module init function being called for it to work. In your case it failed when raising an exception, but there's a range of similar bugs waiting to happen if you got past that.

---

As a general rule I'd advise against linking multiple modules together into one .so file. However it can be made to work; see https://stackoverflow.com/a/52714500/4657412 for a recipe. As you can see it's quite involved and requires some understanding of the C API import process.

---

Other options (that don't involve writing the code in C yourself) would be to use multiple modules and the cimport mechanism for sharing implementation between them, or possibly the older "include" mechanism which is usually not recommended but is sometimes convenient.

Answer 2

Your except -1 variant should raise an exception the normal way instead of trying to manually set an exception and return an error value:

cdef public int baz() except -1:
    raise ValueError

Cython will handle the exception setting and the return value for you.