Continue chatgpt api function call when is finishes due to length

500 views Asked by At

I am trying to call chatgpt using a function call and it responds with finish reason being the length of the tokens. So my questions is how to continue without messing the function_call results?

I am calling a function recursively when it finishes due to length.

Here is the code:

const chatJSONFunctionCall = async (messages: ChatCompletionMessageParam[],schema: any,lastChatReply: string): Promise<OpenAIApi.Chat.Completions.ChatCompletion> => {
    if(lastChatReply.length > 0){
        messages.push({ role: "assistant", content: lastChatReply })
    }
    
    try {
        const completionStream = await openai.chat.completions.create({
            messages: messages,
            model: "gpt-4",
            max_tokens: 100,
            // n: 3,
            stop: null,
            temperature: 0.5,
            functions: [{ name: "respond_in_json", parameters: schema }],
            function_call: { name:"respond_in_json" }
        });

        return completionStream;
    } catch (error) {
        console.log(error);

        throw new Error("Error in connection")
    }
}

const translate = async (targetLang: string, textObj: any): Promise<any> => {

    const schema = generateFunctionCallSchema(textObj)

    const source_language = "English";

    const prompt = `Translate and rephrase the following ${source_language} JSON to ${targetLang} and try to be sexy. Given JSON: ${JSON.stringify(textObj)}`;

    const messages: ChatCompletionMessageParam[] = [
        { "role": "system", "content": "You are a helpful assistant that translates and rephrases JSON values based on your user's given JSON. Be sure to only change the values of the JSON and not the keys" },
        { "role": "user", "content": prompt }
    ]
    let content: string = ""
    let lastChatReply: string = ""
    const translateRecurr = async (messages: ChatCompletionMessageParam[]): Promise<any> => {
        console.log(messages);
        
        try {
            const completion = await chatJSONFunctionCall(messages,schema,lastChatReply);
            console.log(completion);
            
            const finishReason = completion.choices[0].finish_reason;
            console.log(finishReason);
            
            if (finishReason == "length") {
                content = content + completion.choices[0].message.content
                if(completion.choices[0].message.function_call?.arguments){
                    lastChatReply = completion.choices[0].message.function_call?.arguments
                }
                return await translateRecurr(messages)
            }
            if (finishReason === "stop") {
                content = content + completion.choices[0].message.content
                const result = JSON.parse(content);
                return result

            }
        } catch (error) {
            console.log(error);
        }
  }
    return await translateRecurr(messages);
}

I was expecting to continue the response in json but it just gives back a new json in response that never seems to finish.

This is the conversation

[
  {
    role: "system",
    content: "You are a helpful assistant that translates and rephrases JSON values based on your user's given JSON. Be sure to only change the values of the JSON and not the keys"
  },
  {
    role: "user",
    content: "Translate and rephrase the following English JSON to greek and try to be sexy. Given JSON: {\"backTask\":\"Back to task Yeah\",\"continue\":\"Continue\",\"giveUp\":\"Give up\",\"goBack\":\"Go back\",\"okGotIt\":\"Ok. Got it!\",\"skipQuestion\":\"Skip this question\",\"skipTask\":\"Skip task\",\"pinTask\":\"Pin task\",\"justSkip\":\"Skip\",\"preview\":\"This is a preview of your test, we won't collect any data.\"}"
  },
  {
    role: "assistant",
    content: "{\n  \"backTask\": \"Επιστροφή στην εργασία. Ω ναι!\",\n  \"continue\": \"Συνέχισε\",\n  \"giveUp\": \"Παράτα το\",\n  \"goBack\": \"Πήγαινε πίσω\",\n  \"okGotIt\": \"Εντάξει."
  },
  {
    role: "user",
    content: "continue generating"
  }, {
    role: "assistant",
    content: "{\n  \"okGotIt\": \"Εντάξει. Το έπιασα!\",\n  \"skipQuestion\": \"Παράλειψε αυτή την ερώτηση\",\n  \"skipTask\": \"Παράλειψε την εργασία\",\n  \"pinTask\": \"Καρ"
  },
  {
    role: "user",
    content: "continue generating"
  }, {
    role: "assistant",
    content: "{\n  \"pinTask\": \"Καρφίτσωσε την εργασία\",\n  \"justSkip\": \"Απλά παράλειψε\",\n  \"preview\": \"Αυτή είναι μια προεπισκόπηση του τεστ σου, δεν θα συλλ"
  },
  {
    role: "user",
    content: "continue generating"
  }
]

Does anybody has any idea how to handle the function call response so I get a complete json no matter the length. Thank you

0

There are 0 answers