Is there a way to merge palettes for indexed pygame.Surfaces?

Question

I have a Surface BaseSurf with an indexed colour palette, and I want to blit another indexed-colour Surface to it (let's call it NewSurf). BaseSurf's palette is different from NewSurf's, though, and pygame does not automatically add the missing colours to BaseSurf's palette. I therefore need some way to append NewSurf's colours to BaseSurf's palette without mangling either (overwriting palette indices which are actually used by the Surface's pixels).

When I run this code:

screen = pygame.display.set_mode((222,222))
screen.fill((255,255,255))

BaseSurf = pygame.load.image("4samps.png").convert(8)  # indexed with four colours; larger dimensions
NewSurf  = pygame.load.image("another4samps.png").convert(8)  # indexed with four more colours

screen.blit(BaseSurf, (10,10))
screen.blit(NewSurf, (160,43))

BaseSurf.blit(NewSurf, (33,33))

screen.blit(BaseSurf, (50,122))

...this is what I see...

--EDIT: Notice that the combined graphic approximates the colours in NewSurf to whatever BaseSurf is using, even if the sequence is off. The bottom right blob actually takes a null value (0,0,0,255) to be the closest matching colour!

Obviously, NewSurf (on the right) has had its indices transferred to BaseSurf, which is not incorrect exactly, but is also not what I want. How can I blit NewSurf to BaseSurf, preserving its exact colour data in the revised image (but retaining an indexed colour image)?

Here is another (unanswered) question in the same vein.

Answer 1

I actually figured out an answer in the course of asking it.

Using pygame.Surface.set_palette_at(), I was able to extract the palette data from NewSurf (using NewSurf.get_palette(..) and an iterator to clear out that palette's blanks) and paste it onto the 'end' (that is, after the last non-blank RGBA index value) of BaseSurf's palette using BaseSurf.set_palette_at(first_null_RGBA_index, new_RGBA_value).

You can find first_null_RGBA_value in a number of ways; I've used next() to find the first null value (defined by blankRGBA, which is usually (0,0,0,255) in my experience) in the 'train' of null (unused) values that follows the in-use palette in get_palette(). You could also use get_palette().index(blankRGBA) to get the index of the first blank value (which is super-risky if you actually have blank-value pixels in the graphic!). I'll describe the methods and their issues (as I see them) in a moment.

As it happens, I do not seem to have to reindex NewSurf that it's indices align to their new locations in BaseSurf's palette. I do not know what consequence this will have if you remap a duplicated RGB value! I imagine that pygame approximates RGB values from the blit'd image to match the closest ones in the receiving one-- an exact match, if you purposefully paste the whole palette from NewSurf into BaseSurf beforehand.

Here's how I did that.

   ... # initialized and blitted samples as before...
blankRGBA = (0,0,0,255)  # the RGBA value for null values in the index.

destpal   = list(BaseSurf.get_palette())
pallength = len(destpal) - 1 # This is probably always going to be 255, but just to be sure... (minus one 'cuz indices start at zero.)

nextblank = pallength - next((n for n, RGBA in enumerate(destpal[::-1]) if RGBA != blankRGBA), - 1)   
    #  ^ The palette will have a train of null values matching blankRGBA for every unusued index value. If the palette is complete full, it'll raise an error later.
    # This finds the index of the first such value by following the train until it 'starts', then subtracting that index number from the total length (probably 256). I'll explain why destpal.index(blankRGBA) is chancey later...

copypal = list(NewSurf.get_palette())
while True:
    if copypal[-1] == blankRGBA:  # Get rid of NewSurf's null index train, too.
        copypal.pop()
    else:
        print "Popped all trailing blank values from the incoming palette. %d entries remain." % len(copypal)
        break

    if not copypal:
        raise IndexError, "Depleted incoming palette. It was entirely blank entries?! What??"

    # Now that we have the useful section of NewSurf's palette (copypal) and the indices we can replace with it (nextblank), let's apply the new values and reindex NewSurf ahead of blitting it...

for n, newRGBA in enumerate(copypal):  
    if (nextblank + n) > 255: # It's possible the thing will fill up. For now, I'll have it throw an error.
        raise IndexError, "Ran out of palette space at %s! (colour number %d)" % (newRGBA, n)
    BaseSurf.set_palette_at((nextblank + n), newRGBA)  # Add the palette value to BaseSurf. As it happens, blit will reindex the colours on its own.

baseimage.blit(newimage, (33,33))
screen.blit(baseimage, (50, 122))

pygame.display.flip()

I could probably have also cleaned the blank RGBA values from NewSurf's palette with something like copypal = list(RGBA for RGBA in NewSurf.get_palette() if RGBA != blankRGBA) instead of the while True; ... copypal.pop() loop. I also could have found the first blank in BaseSurf's palette using destpal.index(blankRGBA) instead of the more complicated next() instruction. The reason why I did neither of these is because there's a chance that the palettes used the blankRGBA value for at least one pixel in the images, and that these pixels were intended to be blank-- it's not so unlikely that (0,0,0,255) would be used somewhere in the image.

Presumably, if that were the case, the RGBA indices would be at the start of the palette instead of at the end. If they were anywhere but the last index, they'll be safe. Otherwise, there could be a problem.

---

For situations where you can control the image data very closely, these condensed versions may also work.

Note however that they are very probably less Pythonic as they're harder to read, and that they are more vulnerable to certain problems (accidentally replacing a null-looking index value that's actually in use) and unhandled exceptions (running out of palette space in BaseSurf).

Use at your own risk!

  # init as before...
for newRGBA in tuple(RGBA for RGBA in NewSurf.get_palette() if RGBA != blankRGBA):
    try: swapidx = next(n for n, RGBA in enumerate(BaseSurf.get_palette()) if RGBA == blankRGBA)
    except StopIteration: raise IndexError, "Ran out of palette space!"
    BaseSurf.set_palette_at(swapidx, newRGBA)

BaseSurf.blit(NewSurf, (33,33))
screen.blit(BaseSurf, (50, 122))

This is a little slower, since next() will iterate through the palette for each value in copypal, but hey, it's fewer lines, and that's a good thing.

You could even just use this horrible and extremely unPythonic singleton:

[BaseSurf.set_palette_at(swapidx, newRGBA) for swapidx, newRGBA in zip(tuple(n for n, RGBA in enumerate(BaseSurf.get_palette()) if RGBA == blankRGBA), tuple(RGBA for RGBA in NewSurf.get_palette() if RGBA != blankRGBA))]  # PLEASE GOD NO.

Note that I do not recommend either short version. I've only included them for academic consideration.