C# Pinning pixel data of custom object

Question

I have two IntPtr's to two different float images (source.IP and destination.IP).

I copy a block of pixels from one image to the other.

int j = 0;
for (int y0 = y; y0 < h + y; y0++)
{
    float* s = ((float*)source.IP) + (y0 * sourcewidth) + x;
    float* d = ((float*)destination.IP) + ((ydestination + j) * destinationwidth) + xdestination;

    for (int i = 0; i < w; i++)
    {
        *d = *s;
        s++;
        d++;
    }

    j++;
}

The above code is running in an unsafe block.

The question is: How do I pin the image data while copying so that the GC doesn't move it around?

If I use GCHandle.Alloc(source) I get: ArgumentException: Object contains non-primitive or non-blittable data error.

EDIT

The reason for using unsafe code is because the only access to the image data is through the IntPtr source.IP, if there is another way to copy the data I am open minded :)