Streaming chatbot with nicegui

Question

I am trying to follow the example in this link https://github.com/zauberzeug/nicegui/blob/main/examples/chat_with_ai/main.py but for a streaming chatbot use case.

Every single time, I send a message, its looping through all the messages and even for a third question, it processes the first and second question (already asked) before processing the third question/prompt. I cant quite make it to work like the documented example. I am assuming I am doing something blatantly wrong that will be immediately apparent to an experienced eye. Following are relevant portions of my script:

vertexai.init(project=PROJECT_ID, location=LOCATION)
model = GenerativeModel("gemini-pro")
chat = model.start_chat()

async def get_async_chat_response(chat: ChatSession, prompt: str) -> str:
    response = await chat.send_message_async(prompt, stream=True)
    return response

messages: List[Tuple[str, str]] = []
thinking: bool = False

@ui.refreshable
async def chat_messages() -> None:
    for name, text in messages:
        avatar = human_avatar if name == 'You' else bot_avatar
        if name == 'You':
            ui.chat_message(avatar=avatar, name=name, text=text, sent=name == 'You')
        else:
            task = asyncio.create_task(get_async_chat_response(chat, text))
            response = await task
            async for item in response:
                l = ui.label()
                l.text += item.text
                print(item.text)

    if thinking:
        ui.spinner(size='3rem').classes('self-center')
    if context.get_client().has_socket_connection:
        # ui.run_javascript('window.scrollTo(0, document.body.scrollHeight)')
        ui.run_javascript('history.scrollRestoration = "manual"')

async def send() -> None:
    nonlocal thinking
    message = text.value
    messages.append(('You', text.value))
    thinking = True
    text.value = ''
    # chat_messages.refresh()

    messages.append(('Bot', message))
    thinking = False
    chat_messages.refresh()

with ui.tab_panels(tabs, value=chat_tab).classes('w-full max-w-2xl mx-auto flex-grow items-stretch'):
    with ui.tab_panel(chat_tab).classes('items-stretch'):
        chat_messages()
        
ui.run(title='simple chat app with google gemini')

Answer 1

You're calling get_async_chat_response for every message again and again when refreshing the UI. But chat_messages() should only create the UI for existing messages; requesting AI responses should be done for new messages only. Maybe our "Chat with AI" example can serve as inspiration.