cuda-memcheck fails to detect memory leak in an R package

Question

I'm building CUDA-accelerated R packages, and I want to debug with cuda-memcheck. So in this minimal example (in the deliberate_memory_leak GitHub branch), I create a memory leak in someCUDAcode.c by commenting out a necessary call to cudaFree. Then, I see if cuda-memcheck can find the leak.

$ cuda-memcheck --leak-check full  Rscript -e 'library(rcppcuda); hello()'
========= CUDA-MEMCHECK
An object of class "MyClass"
Slot "x":
 [1]  1  2  3  4  5  6  7  8  9 10

Slot "y":
 [1]  1  2  3  4  5  6  7  8  9 10

[1] "Object changed."
An object of class "MyClass"
Slot "x":
 [1] 500   2   3   4   5   6   7   8   9  10

Slot "y":
 [1]    1 1000    3    4    5    6    7    8    9   10

========= LEAK SUMMARY: 0 bytes leaked in 0 allocations
========= ERROR SUMMARY: 0 errors
$

No luck. Then I saw in the R extensions manual that R -d "valgrind --tool=memcheck --leak-check=full" --vanilla < mypkg-Ex.R is the right way to use valgrind. So I created a test.R file with library(rcppcuda); hello() and tried this.

R -d "cuda-memcheck --leak-check full" --vanilla < test.R
*** Further command line arguments ('--vanilla ') disregarded
*** (maybe use 'run --vanilla ' from *inside* cuda-memcheck --leak-check full)

========= CUDA-MEMCHECK
Fatal error: you must specify '--save', '--no-save' or '--vanilla'
========= LEAK SUMMARY: 0 bytes leaked in 0 allocations
========= ERROR SUMMARY: 0 errors

And finally,

$ cuda-memcheck --leak-check full R --vanilla < test.R
========= CUDA-MEMCHECK

R version 3.2.0 (2015-04-16) -- "Full of Ingredients"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(rcppcuda); hello()
An object of class "MyClass"
Slot "x":
 [1]  1  2  3  4  5  6  7  8  9 10

Slot "y":
 [1]  1  2  3  4  5  6  7  8  9 10

[1] "Object changed."
An object of class "MyClass"
Slot "x":
 [1] 500   2   3   4   5   6   7   8   9  10

Slot "y":
 [1]    1 1000    3    4    5    6    7    8    9   10

> 
========= LEAK SUMMARY: 0 bytes leaked in 0 allocations
========= ERROR SUMMARY: 0 errors
$

Is it possible to make cuda-memcheck work for R packages? I know I thought I had figured it out here, but back then, I didn't actually verify the answers that cuda-memcheck was giving me.

Answer 1

This is not valid CUDA code:

extern "C" void someCUDAcode() {
  int a;
  CUDA_CALL(cudaMalloc((void**) &a, sizeof(int)));
  mykernel<<<1, 1>>>(1);
//  CUDA_CALL(cudaFree(&a));
}

1. When we want to do a cudaMalloc operation, we use pointers in C, not ordinary variables, like this:

     int *a;
     CUDA_CALL(cudaMalloc((void**) &a, sizeof(int)));
   

2. When we want to free a previously allocated pointer, we pass just the pointer, not the address of it:

   CUDA_CALL(cudaFree(a));
   

3. Finally, the command line help for cuda-memcheck (cuda-memcheck --help) indicates:

   --leak-check <full|no> [Default : no]

   Print leak information for CUDA allocations.

   NOTE: Program must end with cudaDeviceReset() for this to work.

(A similar note is in the documentation as well.)

I think item 3 is the key missing ingredient in your code. The following modification to your code produces a proper report for me:

$ cat t786.cu
#include <stdio.h>

#define CUDA_CALL(x) {if((x) != cudaSuccess){ \
  printf("CUDA error at %s:%d\n",__FILE__,__LINE__); \
  printf("  %s\n", cudaGetErrorString(cudaGetLastError()));}}

__global__ void mykernel(int a){
  int id = threadIdx.x;
  int b = a;
  b++;
  id++;
}

int main() {
  int *a;
  CUDA_CALL(cudaMalloc((void**) &a, sizeof(int)));
  mykernel<<<1, 1>>>(1);
//  CUDA_CALL(cudaFree(a));
  cudaDeviceReset();
}
$ nvcc -o t786 t786.cu
$ cuda-memcheck --leak-check full ./t786
========= CUDA-MEMCHECK
========= Leaked 4 bytes at 0x402500000
=========     Saved host backtrace up to driver entry point at cudaMalloc time
=========     Host Frame:/lib64/libcuda.so.1 (cuMemAlloc_v2 + 0x17f) [0x13629f]
=========     Host Frame:./t786 [0x2dbb3]
=========     Host Frame:./t786 [0x610b]
=========     Host Frame:./t786 [0x3deaf]
=========     Host Frame:./t786 [0x2646]
=========     Host Frame:/lib64/libc.so.6 (__libc_start_main + 0xf5) [0x21d65]
=========     Host Frame:./t786 [0x2539]
=========
========= LEAK SUMMARY: 4 bytes leaked in 1 allocations
========= ERROR SUMMARY: 0 errors
$