Proper way to display PNG using libpng onto a graphics buffer

Question

I am trying to load png image using LibPNG c library onto a graphics buffer like so

//m_pBuffer is the graphics buffer
//row_pointers is the data from png
//width is width of image
//height height of image
//y = 100
//x = 100
//ScreenWidth = 1024
//channel_num = 4  

int pDst = (y + height - 1) * ScreenWidth + x;
for (y = height - 1; y >= 0 ; y--)
{
    png_bytep row = row_pointers[y];
    u32 *pBuffer = m_pBuffer + pDst;
    for(x = 0; x < width; x++) {
      png_bytep px = &(row[x * channel_num]);
      *pBuffer++ = RGBA(px[3], px[2], px[1], px[0]);
    }
    pDst -= ScreenWidth;
} 

but what I get is a slightly distorted image.

Yet this is the original pic

How do I get rid of the slightly yellow background and white borders?

This is how row_pointers is populated

    png_bytep row_pointers[height];

   int row = 0;
   for (row = 0; row < height; row++)
      row_pointers[row] = NULL;

   for (row = 0; row < height; row++)
      row_pointers[row] = (png_byte*)png_malloc(png_ptr, png_get_rowbytes(png_ptr, info_ptr));

   int pass = 0;   
   int number_passes = 1;
   for (pass = 0; pass < number_passes; pass++)
   {
      for (int j = 0; j < height; j++)
      {
          png_read_rows(png_ptr, NULL, &row_pointers[j], 1);
      }
   }
   png_read_end(png_ptr, info_ptr);