Google PlaM API responds different on website and API(Python)

Question

I have a problem using the palm API(MakerSuite) It responds differently between the MakerSuite website and the API(using Python)

I just ran the code provided by the MakerSuite website(Yes,I did put the API Key in),and the respond is different from what I tested on the MakerSuite website

I just hope it will work like what it was shown on the website

Answer 1

It'd help to see your code, particularly the prompt you are using with the API, and the call(s) to palm.chat() and palm.reply().

Without the code though, you should be aware that the models have variability in them by default. You can control this with the temperature parameter. If you are still seeing differences when temperature=0.0, then there's something in your code.

Some other ideas for debugging differences in output:

• Check the prompts in both. Even small changes to whitespace will cause different results.
• Check the top_k and top_p parameters on your requests, if you use the text service, then check the max_output_tokens settings on each, and make sure they are the same.
• Since it looks like you're using the chat service, make sure that all parts of the request are identical - i.e. the context/prompt, the examples and the conversation history. If anything is different, you can expect to see different results.

And as a bonus tip, if you need precisely the same output every time, you might need to find a different way to frame your problem. LLMs are good at generating human-like text, so it's a feature that the results can vary a lot. You need to build with this in mind.