UB due to allocaArray automatic cleanup or not?

73 views Asked by At

i have this function in my code that seems to work just fine:

-- type declaration just for reference, i don't have it in my actual code
retrieveVulkanArray :: Storable a => (Ptr Word32 -> Ptr a -> IO b) -> IO (Ptr a, Int)
retrieveVulkanArray' f =
    alloca $ \arrCount -> do
        f arrCount vkNullPtr
        arrCount' <- fromIntegral <$> peek arrCount
        allocaArray arrCount' $ \resArray -> do
            f arrCount resArray
            pure (resArray, arrCount')

(for context this is a helper function to get FFI arrays from Vulkan API, f might be for example vkEnumeratePhysicalDevices)

As i was reviewing my code, i noticed that it returns resArray(which from description of allocaArray seems to only be valid within the inner lambda) to its caller. In C, a code like this would be undefined behaviour. Is my intuition correct here or is there something more going on? I haven't noticed any crashes yet after all :)

1

There are 1 answers

2
lehins On BEST ANSWER

The fact that it works certainly doesn't prove that it is correct, in fact this function is indeed very wrong.

alloca, as well as allocaArray, will allocate a Haskell MutableByteArray# convert it to a pointer. Operate on that pointer and then ensure that the array is still alive with a special touch# function. Problem is that once you loose reference to the actual MutableByteArray#, which is what happens when you exit alloca, GC will clean it up and the Ptr a that was pointing to that array will no longer be valid. So if you continue reading or writing into that pointer Ptr a after you return it from retrieveVulkanArray you are reading/writing into memory that can be used by something else and are now in real danger of a segfault and all sorts of other security vulnerabilities that come with it.

The proper way would be:

retrieveVulkanArray
  :: Storable a => (Ptr Word32 -> Ptr a -> IO b) -> IO (ForeignPtr a, Int)
retrieveVulkanArray' f =
    alloca $ \arrCount -> do
        f arrCount vkNullPtr
        arrCount' <- fromIntegral <$> peek arrCount
        resArray <- mallocForeignPtrArray arrCount'
        _ <- withForeignPtr resArray (f arrCount)
        pure (resArray, arrCount')

ForeignPtr a ensures that you can operate on the raw Ptr a when needed, without worrying that memory it points to is freed until ForeignPtr is no longer used.