Mex compiled function, used to compute set intersection and difference, keeps crashing

Question

[This question has been solved]

(This question has already been posted on the Matlab forum, here : http://www.mathworks.com/matlabcentral/answers/223415-mex-compiled-function-used-to-compute-set-intersection-and-difference-keeps-crashing)

Hi Guys,

I'm trying to build a very simple function that is supposed to computed the "intersection" and "difference" of two sets, returning the corresponding indices.

For instance, if we have

in1 = [1 2 4 5 9]
in2 = [2 3 4 8]

it should return

common1 = [2 3] % since all(in1(common1) == in2(common2))
common2 = [1 3]
only1 = [1 4 5] % the remaining indices, not in common1
only2 = [2 4]   % the ones not in common2

I could do that using intersect and setdiff, but because I have small sets and since I call this function thousands of times, I think doing it using a compiled C-mex file should be the fastest way. It really is the bottleneck of my algorithm at the moment.

I coded this function

#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[],
        int nrhs, const mxArray *prhs[])
{
    mexPrintf("Starting ...\n") ;
    /* Check nbr of inputs and outputs */
    if (nrhs != 2 || nlhs != 4)
        mexErrMsgTxt("intersectFast needs 4 outputs and 2 inputs") ;
    const mxArray* pin1 = prhs[0] ;
    const mxArray* pin2 = prhs[1] ;
    /* Inputs should be column vectors */
    if (mxGetN(pin1) != 1 || mxGetN(pin2) != 1)
        mexErrMsgTxt("inputs arguments should be column vectors") ;
    mwSize dims1 = mxGetM(pin1) ;
    mwSize dims2 = mxGetM(pin2) ;
    double* in1 = mxGetPr(pin1) ;
    double* in2 = mxGetPr(pin2) ;
    mexPrintf("Checks passed\n") ;
    mwIndex* idCommon1 = mxCalloc(dims1, sizeof(mwIndex)) ; // At most dims1 elements
    mwIndex* idCommon2 = mxCalloc(dims2, sizeof(mwIndex)) ; /* AT MOST dims2 and NOT dims1 ... this was the error. Damn I feel so stupid right now. */
    mwIndex* idOnly1   = mxCalloc(dims1, sizeof(mwIndex)) ; /* Same error here */
    mwIndex* idOnly2   = mxCalloc(dims2, sizeof(mwIndex)) ;  
    mwSize sizeCommon1, sizeCommon2, sizeOnly1, sizeOnly2 ;
    mwIndex i, j ;
    mwIndex k, l ;
    int match ;
    /* Intersect fast */
    /* in1 */
    k = 0 ; l = 0 ;
    for(i = 0 ; i < dims1 ; i++) {
        match = 0 ;
        for(j = 0 ; j < dims2 ; j++) {
            if (in1[i] == in2[j]) {
                idCommon1[k++] = (i+1) ; /* Matlab <-> C convention */
                match = 1 ;
                break ;
            }
        }
        if (! match) {
            idOnly1[l++] = (i+1) ;
        }
    }
    sizeCommon1 = k ;
    sizeOnly1 = l ;
    /* in2 */
    k = 0 ; l = 0 ;
    for(i = 0 ; i < dims2 ; i++) {
        match = 0 ;
        for(j = 0 ; j < dims1 ; j++) {
            if (in2[i] == in1[j]) {
                idCommon2[k++] = (i+1) ;
                match = 1 ;
                break ;
            }
        }
        if (! match)
            idOnly2[l++] = (i+1) ;
    }
    sizeCommon2 = k ;
    sizeOnly2 = l ;
    /* Return results */
    mexPrintf("Sizes = %d, %d, %d, %d\n", sizeCommon1, sizeCommon2, sizeOnly1, sizeOnly2) ;
    plhs[0] = mxCreateNumericMatrix(sizeCommon1, 1, mxUINT32_CLASS, mxREAL);
    plhs[1] = mxCreateNumericMatrix(sizeCommon2, 1, mxUINT32_CLASS, mxREAL);
    plhs[2] = mxCreateNumericMatrix(sizeOnly1,   1, mxUINT32_CLASS, mxREAL);
    plhs[3] = mxCreateNumericMatrix(sizeOnly2,   1, mxUINT32_CLASS, mxREAL);
    if (plhs[0] == NULL || plhs[1] == NULL || plhs[2] == NULL || plhs[3] == NULL)
        mexErrMsgTxt("Could not create mxArray.\n");
    mxSetData(plhs[0], idCommon1);
    mxSetData(plhs[1], idCommon2);
    mxSetData(plhs[2], idOnly1);
    mxSetData(plhs[3], idOnly2);
    mexPrintf("Done.\n") ;
}

When I test it, it often works, but it always eventually crashes ... For instance, using

% Test intersect fast
clc ; close all ; clear all ;
while true    
    clc ;    
    id1 = unique(randi(10, 8, 1)) ;
    id2 = unique(randi(12, 6, 1)) ;       
    [idCommon1, idCommon2, idOnly1, idOnly2] = intersectFast(id1, id2) ;     
    pause(0.1)    
end

it always crashes at some point, after the mex function is done. What I mean is that I get an error like "Matlab has encountered an internal problem and needs to close". So I guess there is some problem with either the mxCreateNumericMatrix or the mxSetData, but I can't figure out what is the problem exactly. I tried changing the index types (uint32, uint64, int, ...) but it did not really change anything.

I am using R2015a on OSX 10.10.3 and the compiled is the default one (Clang).

Thanks a lot for your help !

=================

EDIT : let me be more specific about how it crashes. Sometimes, MATLAB just start to freeze (and I get the rotating colored mouse pointer ...), and eventually crashes. I need to force MATLAB to quit in this case. Some other times, I get an error message from MATLAB, saying it encountered an internal error and needs to quit. In this case I can find a Matlab crash file. I uploaded one crash report here : http://pastebin.com/ry7MN7yw

Answer 1

It would be better if you can supply the error message from OS X, which can be obtained by clicking the Crashed notification on the upper-right corner of the screen. For questions involving some low-level (compared to Matlab itself) stuffs like C, the crash report from the Operating System is often useful, since you can see what crashed the program. You can paste it in full to a pastebin or whatever.

If you really find that it crashed inside your code, add the -g flag to clang so you can get some line number in the crash report.

Sorry for not writing this as a comment — I don't have 50 rep yet.