Why is the body object not reactive when sending POST API request?

Question

I'm using Vercel's AI SDK and its useChat utility hook in my Nuxt.js application to create a chat interface. I have different agents (e.g., General, Exercise, Cardiology) in a select menu, and I want to send the selected agent's value in the useChats()'s body object with the API request to select the appropriate system prompts on the server side.

However, I'm facing an issue where the body object is not updated reactively when the selected agent changes in the UI.

For some reason, this part of the code is not reactive on user select: body: { agent: selectedAgent.value } // <---- for some reason this is not being updated on menu change

This causes the client to always send the same agent as General:

Full client-side details below:

<template>
<div class="text-center mt-6">
      <select class="border border-2" v-model="selectedAgent" @change="changeAgent">
        <option v-for="(agent, index) in agentsList" :key="index" :value="agent">{{ agent }}</option>
      </select>
<div>
      <div v-for="m in messages" key="m.id" >
        {{ m.role === "user" ? "User: " : "AI: " }}
        {{ m.content }}
      </div>
<form @submit="handleSubmit">
        <input v-model="input" />
      </form>
</template>


<script>
import {useChat} from 'ai/vue'
const agentsList = ['General', 'Exercise', 'Cardiology']
const selectedAgent = ref(agentsList[0]); // Default agent

const { messages, input, handleSubmit } = useChat({
  api: '/api/chat',
  headers: { 'Content-Type': 'application/json' },
  body: { agent: selectedAgent.value } // <---- for some reason this is not being updated on menu change
})

const changeAgent = (e) => {
  selectedAgent.value = e.target.value;
  // Clear the chat messages when the agent is changed
  messages.value = []
};
</script>

The full code and demo:

Github repo: https://github.com/dosstx/nuxt-openai-demo

Live URL: https://main--chipper-snickerdoodle-13271f.netlify.app/

Server-side code that expects the agent value from the request body (server/api/chat.ts):

The output is:

import OpenAI from 'openai'
import { OpenAIStream } from 'ai'
import { CreateChatCompletionRequestMessage } from 'openai/resources/chat'

export default defineLazyEventHandler(async () => {
  const apiKey = useRuntimeConfig().OPENAI_API_KEY;
  if (!apiKey) throw new Error('Missing OpenAI API key')
  const openai = new OpenAI({
    apiKey: apiKey
  })

  const agents = {
    Exercise: [
      // ... system prompts for exercise agent
    ],
    Cardiology: [
      // ... system prompts for cardiology agent
    ],
    General: [
      // ... system prompts for General agent
    ]
    // ... other agents
  }

  return defineEventHandler(async event => {

    // Extract the `prompt` and `agent` from the body of the request
    const { messages, agent } = (await readBody(event)) as {
      messages: CreateChatCompletionRequestMessage[],
      agent: keyof typeof agents // Expecting one of the keys from the agents object
    }

    // Select the agent's system prompts
    const agentSystemMessages = agents[agent] || []

    console.log('from server: ', messages, agent)

    try {
      // Create a chat completion request with the system prompts and messages
      const request = {
        model: 'gpt-3.5-turbo',
        stream: true,
        messages: [...agentSystemMessages, ...messages],
        temperature: 0.2,
        // top_p: 1,
        frequency_penalty: 0,
        presence_penalty: 0,
        max_tokens: 400 // set this limit to length of assistants responses
      };

      // Send the request to OpenAI and get a response stream
      const response = await openai.chat.completions.create(request);

      // Return an OpenAIStream object that wraps the response stream
      return OpenAIStream(response)
    } catch (error) {
      console.error(error);
      // Return a 500 Internal Server Error response with a specific message
      return {
        status: 500,
        body: { error: 'An error occurred while processing your request. Please try again later.' },
      };
    }
  })
})

Environment:

• Nuxt version: 3.6.5

What I've tried:

• I've updated the selectedAgent value inside the changeAgent function, but the body object inside the useChat hook does not reflect the updated value.

Question:

How can I make the body object inside the useChat hook reactive to changes in the selected agent from the select menu? This value needs to be sent to the server to select the appropriate system prompts. Any insights or guidance would be greatly appreciated.

Answer 1

useChat composable had design problem and provided no way to provide reactive body in the options. Generally the solution that is idiomatic to Vue is to pass refs for reactive values in composable's arguments, this is the purpose that ref pattern serves.

Once this option has been provided, it's supposed to be used as:

...
const body = computed(() => ({ agent: selectedAgent.value }));

const { messages, input, handleSubmit } = useChat({
  body,
  ...
})