How to stream data chunk by chunk in Django using StreamingHttpResponse and receive it in the front end using axios?

Question

I am trying to stream data from my Django backend to my front-end using the StreamingHttpResponse and axios. However, I am encountering an issue where the data is being buffered and only received in the front-end once all the data is ready or when I stop the server. I am using the code provided above for the backend and front-end. I would like to know how I can prevent this buffering and receive the data chunk by chunk in the onDownloadProgress event of axios.

Backend:

def iterator():
    for i in range(1000):
        yield f'chunk: {i}'
        sleep(0.2)


def generate_post_text(request):
    stream = iterator()
    response = StreamingHttpResponse(stream, status=200, 
               content_type='text/event-stream'')
    response['Cache-Control'] = 'no-cache'
    response['X-Accel-Buffering'] = 'no'
    response.streaming = True
    return response

And here is the front end code:

axios({
            url: '/api/gpt3/generate-post-text',
            method: 'GET',
            onDownloadProgress: progressEvent => {
                const dataChunk = progressEvent.currentTarget.response
                console.log(dataChunk)
            },

        }).then(({data}) => Promise.resolve(data))

Backend middlewares:

MIDDLEWARE = [
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',]

I run the server locally on a Mac

Answer 1

It doesn't look like an issue with your django StreamingHttpResponse, but rather your front end libray axios problem, which does not working in a streaming mode, you should use the responseType option to tell Axios that you want access to the raw response stream:

Axios document reference here.

axios({
    url: '/api/gpt3/generate-post-text',
    method: 'GET',
    responseType: 'stream',  // add this line
    onDownloadProgress: progressEvent => {
        const dataChunk = progressEvent.currentTarget.response
        console.log(dataChunk)
    },

}).then(({data}) => Promise.resolve(data))