How to Use images as input for palm api?

Question

I am currently exploring the Palm2 API for image processing, but I am facing challenges due to the lack of comprehensive documentation in the makersuite documentation. I have reviewed the available resources, but couldn't find specific information on how to use Palm2 for image-related tasks.

I would appreciate it if someone with experience in using Palm2 could provide guidance or point me to any unofficial documentation, tutorials, or examples that might be available. Specifically, I am interested in performing specific image processing tasks you want to achieve, e.g., image recognition, feature extraction, etc.

Answer 1

Palm api doesnt offer image processing at this time. It just has

• text prompt
• chat prompt
• and data prompt

if you want to do image processing I think you should have a look at vertex AI

PROJECT_ID = "PROJECT_ID"  # @param {type:"string"}
LOCATION = "LOCATION"  # @param {type:"string"}

import vertexai
from vertexai.vision_models import ImageTextModel, Image

vertexai.init(project=PROJECT_ID, location=LOCATION)
model = ImageTextModel.from_pretrained("imagetext@001")

source_image = Image.load_from_file(location='./gen-img1.png')

answers = model.ask_question(
    image=source_image,
    question="What breed of dog is this a picture of?",
    # Optional:
    number_of_results=2,
)
print(answers)