How can I publish PIL image binary through ROS without OpenCV?

Question

I'm currently trying to write a ROS Publisher/Subscriber setup that passes image binary opened by PIL. I'd like to not have to use OpenCV due to operating restrictions, and I was wondering if there was a way to do so. This is my current code:

#!/usr/bin/env python
import rospy
from PIL import Image
from sensor_msgs.msg import Image as sensorImage
from rospy.numpy_msg import numpy_msg
import numpy

def talker():
    pub = rospy.Publisher('image_stream', numpy_msg(sensorImage), queue_size=10)
    rospy.init_node('image_publisher', anonymous=False)
    rate = rospy.Rate(0.5)
    while not rospy.is_shutdown():
        im = numpy.array(Image.open('test.jpg'))
        pub.publish(im)
        rate.sleep()

if __name__ == '__main__'
    try:
        talker()
    except ROSInterruptException:
        pass

which on pub.publish(im) attempt throws:

TypeError: Invalid number of arguments, args should be ['header', 'height', 'width', 'encoding', 'is_bigendian', 'step', 'data'] args are (array([[[***array data here***]]], dtype=uint8),)

How would I transform the image into the right form, or is there a conversion method/different message type that supports just sending raw binary over the ROS connection?

Thanks

Answer 1

Indeed Mark Setchell's answer works perfectly (ignoring the alpha channel in this example):

#!/usr/bin/env python
import rospy
import urllib2  # for downloading an example image
from PIL import Image
from sensor_msgs.msg import Image as SensorImage
import numpy as np

if __name__ == '__main__':
    pub = rospy.Publisher('/image', SensorImage, queue_size=10)

    rospy.init_node('image_publisher')

    im = Image.open(urllib2.urlopen('https://cdn.sstatic.net/Sites/stackoverflow/Img/apple-touch-icon.png'))
    im = im.convert('RGB')

    msg = SensorImage()
    msg.header.stamp = rospy.Time.now()
    msg.height = im.height
    msg.width = im.width
    msg.encoding = "rgb8"
    msg.is_bigendian = False
    msg.step = 3 * im.width
    msg.data = np.array(im).tobytes()
    pub.publish(msg)

Answer 2

I don't know anything about ROS, but I use PIL a lot, so if someone else knows better, please ping me and I will delete this "best guess" answer.

So, it seems you need to make something like this from a PIL Image. So you need:

• 'header',
• 'height',
• 'width',
• 'encoding',
• 'is_bigendian',
• 'step',
• 'data'

So, assuming you do this:

im = Image.open('test.jpg')

you should be able to use:

• something you'll need to work out
• im.height from PIL Image
• im.width from PIL Image
• probably const std::string RGB8 = "rgb8"
• probably irrelevant because data is 8-bit
• probably im.width * 3 as it's 3 bytes per pixel RGB
• np.array(im).tobytes()

Before anyone marks this answer down, nobody said answers have to be complete - they can just be "hopefully helpful"!

Note that if your input image is PNG format, you should check im.mode and if it is "P" (i.e. palette mode) immediately run:

im = im.convert('RGB')

to make sure it is 3-channel RGB.

Note that if your input image is PNG format and contains an alpha channel, you should change the encoding to "rgba8" and set step = im.width * 4.