UnmanagedMemoryStream does not work with ASP.NET Core 5.0

Question

UnmanagedMemoryStream does not work with ASP.NET Core 5.0. I have used to use UnmanagedMemoryStream but it does not work with "1.0.0-beta2 CoreCLR amd64" runtime. I have used temporary the code:

unsafe
        {
            int cnt = 0;
            for (int i = 0; i < count && pos < size; i++)
            {
                buffer[i + offset] = ((byte*)this.buffer.ToPointer())[pos++];
                ++cnt;
            }
            return cnt;
        }

It copies data into manage buffer, but it is slower that use it directly. Is there going to be a fix for this or this is conceptual issue?