TFLearn: How to use image preloader for image in -> image out regression problems

Question

I'm new to tflearn and I'm trying to train a CNN to solve image in -> image out regression problems, where the input images are either MxN grayscale or MxNx3 RGB.

I have a very large training set that will not fit in memory, so I would like to train "on the fly", pulling images from disk as training is ready to consume them.

The image_preloader seems to make the assumption that we will always be solving problems where X is an input image and y is a set of scalar labels corresponding to X.

What is the best way within the tflearn framework to do something similar for regression problems where both X and y are images? I feel like I have to be missing something obvious, but I've looked through the Data Utils, Data Augmentation, and Data Preprocessing sections and don't see anything.

For the full scope of what I'm trying to do: I have a large set of images on disk. On the fly, I need to be able to load my images, then do some modifications to the image to define the input images X and y. The images on disk need to be augmented to properly define X and y. Given a minibatch X read from disk, I can define X and y a batch at a time.

Answer 1

In pure TensorFlow you could use something like:

def load_image(type="feature")
  files = '/path/to/my/images/%s/*.jpg' % type
  filename_queue = tf.train.string_input_producer(tf.gfile.Glob(files))
  reader = tf.WholeFileReader()
  filename, data = reader.read()
  image_tensor = tf.image.decode_jpeg(data, channels=3)
  return image_tensor

Note the channels=3 in decode_jpeg will ensure your images are all tensors of [h, w, 3], even if they were originally in a different colourspace.

At this point you have single image tensors. You can batch them up using background queues like so:

features = load_image("feature")
labels = load_image("labels")
batch_size = 64
input_batch, label_batch = tf.train.shuffle_batch([features, labels],
  batch_size=batch_size, num_threads=4, capacity=1000 + 3 * batch_size,
  min_after_dequeue=1000)

You'll need to start the queue runners too. This (and the magic numbers in the last code snippet) is explained fairly well in the documentation on batching.