How to view real time mosaicing of large image?

620 views Asked by At

I have built a code which will stitch 100X100 images approx. I want to view this stitiching process in real time. I am using pyvips to create large image. I am saving final image in .DZI format as it will take very less memory footprint to display. Below code is copied just for testing purpose https://github.com/jcupitt/pyvips/issues/43.

#!/usr/bin/env python

import sys
import pyvips

# overlap joins by this many pixels
H_OVERLAP = 100
V_OVERLAP = 100

# number of images in mosaic
ACROSS = 40
DOWN = 40

if len(sys.argv) < 2 + ACROSS * DOWN:
    print 'usage: %s output-image input1 input2 ..'
    sys.exit(1)
def join_left_right(filenames):
    images = [pyvips.Image.new_from_file(filename) for filename in filenames]
    row = images[0]
    for image in images[1:]:
        row = row.merge(image, 'horizontal', H_OVERLAP - row.width, 0)
    return row
def join_top_bottom(rows):
    image = rows[0]
    for row in rows[1:]:
        image = image.merge(row, 'vertical', 0, V_OVERLAP - image.height)
    return image
rows = []
for y in range(0, DOWN):
    start = 2 + y * ACROSS
    end = start + ACROSS
    rows.append(join_left_right(sys.argv[start:end]))
image = join_top_bottom(rows)
image.write_to_file(sys.argv[1])

To run this code:

$ export VIPS_DISC_THRESHOLD=100
$ export VIPS_PROGRESS=1
$ export VIPS_CONCURRENCY=1
$ mkdir sample
$ for i in {1..1600}; do cp ~/pics/k2.jpg sample/$i.jpg; done
$ time ./mergeup.py x.dz sample/*.jpg

here cp ~/pics/k2.jpg will copy k2.jpg image 1600 times from pics folder, so change according to your image name and location.

I want to display this process in real time. Right now after creating final mosaiced image I am able to display. Just an idea,I am thinking to make a large image and display it, then insert smaller images. I don't know, how it can be done. I am confused as we also have to make pyramidal structure. So If we create large image first we have to replace each level images with the new images. Creating .DZI image is expensive, so I don't want to create it in every running loop. Replacing images may be a solution. Any suggestion folks??

1

There are 1 answers

4
iangilman On

I suppose you have two challenges: how to keep the pyramid up-to-date on the server, and how to keep it up-to-date on the client. The brute force method would be to constantly rebuild the DZI on the server, and periodically flush the tiles on the client (so they reload). For something like that you'll also need to add a cache bust to the tile URLs each time, or the browser will think it should just use its local copy (not realizing it has updated). Of course this brute force method is probably too slow (though it might be interesting to try!).

For a little more finesse, you'd want to make a pyramid that's exactly aligned with the sub images. That way when you change a single sub image, it's obvious which tiles need to be updated. You can do this with DZI if you have square sub images and you use a tile size that is some even fraction of the sub image size. Also no tile overlap. Of course you'll have to build your own DZI constructor, since the existing ones aren't primed to simply replace individual tiles. If you know which tiles you changed on the server, you can communicate that to the client (either via periodic polling or with something like web sockets) and then flush only those tiles (again with the cache busting).

Another solution you could experiment with would be to not attempt a pyramid, per se, but just a flat set of tiles at a reasonable resolution to allow the user to pan around the scene. This would greatly simplify your pyramid updating on the server, since all you would need to do would be replace a single image for each sub image. This could be loaded and shown in a custom (non-OpenSeadragon) fashion on the client, or you could even use OpenSeadragon's multi-image feature to take advantage of its panning and zooming, like here: http://www.letsfathom.com/ (each album cover is its own independent image object).