How to send images from ESP32 CAM to IoT Core?

Question

I need the system to be secure.

I tired to encode the image with base64 and sending the string via MQTT to Iot Core. Then decode the string with a cloud function and finally storage the decoded image in Google Cloud Storage. The problem is the limited size of a message in MQTT.

Using a Cloud Function and then storage in Google Cloud Storage is not really secure, anyone could hit that url and I loos control of all the ESP32CAM comunication.

Am I missing something? is there a really secure way to send files to Google Cloud Storage from to IoT Core?

Thanks

Answer 1

IoT Core should not be used to transfer big blobs.

However, you can take advantage of the secure connection between IoT Core and the device to send credentials to the device to access GCS securely.

Create a service account with write only access to your GCS bucket. Pass a key for that service account to the device through IoT Core(via configuration change, for example) The device then can use that key to connect securely to GCS and upload the image.

Depending on your preferences and the particular use case you can rotate the keys to access GCS whenever you want, or be as granular as you want with the permissions (one key for all the devices, one key per device, ...)

Answer 2

The way I've done it is to break the image up into 256K packages (well, 255K-ish with a header of 8 bytes that's an int which represents the order it should be reassembled on the other end since Pub/Sub isn't guaranteed ordering).

@rbarbero's answer is another good one, where you send down creds to talk to GCS directly.

Another way to do it would be to have the device talk to something local that's more powerful which has that service credential directly to GCS and just bypass IoT Core entirely.

Answer 3

No need to base64 encode it and the pubsub MQTT buffer can be resized.

I use:

#include <PubSubClient.h>
...
void setup() {
...
  boolean res = mqttClient.setBufferSize(50*1024); // ok for 640*480
  if (res) Serial.println("Buffer resized."); else Serial.println("Buffer resizing failed");
...
}

void sendPic() {
...
      if (fb->len) // send only images with size >0
        if (mqttClient.beginPublish("test_loc/esp32-cam/pic_ms", fb->len + sizeof(long), false))
        {
          // send image data + millis()
          unsigned long m = millis();  
          int noBytes;
          noBytes = mqttClient.write(fb->buf, fb->len);
          noBytes = mqttClient.write((byte *) &m, sizeof(long));
          if (!mqttClient.endPublish())
          {
            // error!
            Serial.println("\nError sending data.");
          }
        }
...
}

Here I send 640*480 image and append current millis() value at the end for stitching it back to video through ffmpeg.