I'm trying to pass message metadata with StreamingTextResponse
and to read it on the client side with useChat
.
This is the sample code I'm using:
import {NextRequest, NextResponse} from "next/server";
import {experimental_StreamData, Message as VercelChatMessage, StreamingTextResponse} from "ai";
import { ChatOpenAI } from "langchain/chat_models/openai";
import {PromptTemplate} from "langchain/prompts";
import {BytesOutputParser} from "langchain/schema/output_parser";
export async function POST(req: NextRequest) {
try {
const body = await req.json();
const messages: VercelChatMessage[] = body.messages ?? [];
const currentMessageContent = messages[messages.length - 1].content;
const model = new ChatOpenAI({
temperature: 0,
streaming: true,
});
const prompt = PromptTemplate.fromTemplate(`
You are a cheerful bot that should respond to all questions in an upbeat manner.
User input: {input}
AI response:
`);
const outputParser = new BytesOutputParser();
const chain = prompt
.pipe(model)
.pipe(outputParser);
const stream = await chain.stream({input: currentMessageContent});
return new StreamingTextResponse(stream);
}
catch (e: any) {
return NextResponse.json({ error: e.message }, { status: 500 });
}
}
I tried using experimental_StreamData()
and passing the data object as the third argument to StreamingTextResponse()
:
const data = new experimental_StreamData();
data.append({text: new Date().toISOString()});
data.close();
return new StreamingTextResponse(stream, {}, data);
However, this just stringifies the data and prepends it to the response message.
What should I change here to pass the data object separately from the response?
Thanks!