How to present numpy array into pygame surface?

Question

I'm writing a code that part of it is reading an image source and displaying it on the screen for the user to interact with. I also need the sharpened image data. I use the following to read the data and display it in pyGame

def image_and_sharpen_array(file_name):
    #read the image data and return it, with the sharpened image
    image = misc.imread(file_name)

    blurred = ndimage.gaussian_filter(image,3)
    edge = ndimage.gaussian_filter(blurred,1)
    alpha = 20
    out = blurred + alpha*(blurred - edge)
    return image,out

#get image data
scan,sharpen = image_and_sharpen_array('foo.jpg')
w,h,c = scan.shape


#setting up pygame
pygame.init()
screen = pygame.display.set_mode((w,h))

pygame.surfarray.blit_array(screen,scan)
pygame.display.update()

And the image is displayed on the screen only rotated and inverted. Is this due to differences between misc.imread and pyGame? Or is this due to something wrong in my code?

Is there other way to do this? The majority of solution I read involved saving the figure and then reading it with ``pyGame''.

Answer 1

Every lib has its own way of interpreting image arrays. By 'rotated' I suppose you mean transposed. That's the way PyGame shows up numpy arrays. There are many ways to make it look 'correct'. Actually there are many ways even to show up the array, which gives you full control over channel representation and so on. In pygame version 1.9.2, this is the fastest array rendering that I could ever achieve. (Note for earlier version this will not work!). This function will fill the surface with array:

def put_array(surface, myarr):          # put array into surface
    bv = surface.get_view("0")
    bv.write(myarr.tostring())

If that is not working, use this, should work everywhere:

# put array data into a pygame surface
def put_arr(surface, myarr):
    bv = surface.get_buffer()
    bv.write(myarr.tostring(), 0)

You probably still get not what you want, so it is transposed or have swapped color channels. The idea is, manage your arrays in that form, which suites this surface buffer. To find out what is correct channel order and axes order, use openCV library (cv2.imread(filename)). With openCV you open images in BGR order as standard, and it has a lot of conversion functions. If I remember correctly, when writing directly to surface buffer, BGR is the correct order for 24 bit and BGRA for a 32 bit surface. So you can try to put the image array which you get out of file with this function and blit to the screen.

There are other ways to draw arrays e.g. here is whole set of helper functions http://www.pygame.org/docs/ref/surfarray.html
But I would not recommend using it, since surfaces are not for direct pixel manipulating, you will probably get lost in references. Small tip: To do 'signalling test' use a picture, like this. So you will immediately see if something is wrong, just load as array and try to render.

Answer 2

My suggestion is to use the pygame.transform module. There are the flip and rotate methods, which you can use to however your transformation is. Look up the docs on this.

My recommendation is to save the output image to a new Surface, and then apply the transformations, and blit to the display.

temp_surf = pygame.Surface((w,h))
pygame.surfarray.blit(temp_surf, scan)

'''transform temp_surf'''

screen.blit(temp_surf, (0,0))

I have no idea why this is. It is probably something to do with the order in which the axes are transferred from a 2d array to a pygame Surface.

Answer 3

I thought technico provided a good solution - just a little lean on info. Assuming get_arr() is a function that returns the pixel array:

pixl_arr = get_arr()
pixl_arr = numpy.swapaxes(pixl_arr, 0, 1)
new_surf = pygame.pixelcopy.make_surface(pixl_arr)
screen.blit(new_surf, (dest_x, dest_y))

Alternatively, if you know that the image will always be of the same dimensions (as in iterating through frames of a video or gif file), it would be more efficient to reuse the same surface:

pixl_arr = get_arr()
pixl_arr = numpy.swapaxes(pixl_arr, 0, 1)
pygame.pixelcopy.array_to_surface(old_surf, pixl_arr)
screen.blit(old_surf, (dest_x, dest_y))

YMMV, but so far this is working well for me.

Answer 4

I often use the numpy swapaxes() method: In this case we only need to invert x and y axis (axis number 0 and 1) before displaying our array :

return image.swapaxes(0,1),out