How can I process, stream a video and render the results in Django?

Question

I'm currently involved in a real-time machine learning project that seamlessly integrates with Django. In this endeavor, I'm deeply immersed in processing and live-streaming video content. This multifaceted process encompasses the capture of video frames, their real-time processing, and their seamless streaming to a web application. Furthermore, I've integrated the Firebase Realtime Database to facilitate the storage and retrieval of data associated with the video stream.

Here's a brief overview of what the code does:

Here, I have a generator function called 'generate(camera),' which captures frames and yields them as a multipart stream. Within the 'generate' function, I simultaneously process video frames, detect actions, and yield each frame with a content type of 'image/jpeg.' Upon the completion of video processing, indicated by an end marker, the processed data is stored in a database, and the user is redirected to a results page.

def generate(camera):

    action_labels = ['field_goal', 'pass', 'run', 'touchdown']

    actions = {'actions': {}}

    

    while True:
        data = camera.get_frame()

        if len(data) == 4:
            frame = data['frame']

            results = data['results']

            action = data['action_bboxes']

            timestamp = data['timestamp']

            if len(action) > 0:

                try:

                    action_result = action_labels[action[0][0]]

                    actions['actions'][timestamp] = action_result

                    # actions.append(action_data)
                except:
                    pass

            if frame:                
                yield b'--frame\r\nContent-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n'
        else:
            break

    if len(data) == 3:
        # Upload images, generate download URLs, and store them in Realtime Database
        image_urls = {}

        for image_name, image_data in data['images'].items():

            download_url = dataBase.save_image_reference(data['file_name']['V_name'], image_data, image_name[:-4])
        
            image_urls[image_name[:-4]] = download_url

        dataBase.save(data['file_name'], image_urls, actions)

    yield b'--end\r\n\r\n'

Here is process_video which handles video uploads via a form.

def process_video(request):
    if request.method == 'POST':
        form = VideoForm(request.POST, request.FILES)
        if form.is_valid():
            video = form.save()
            return render(request, 'video/video.html', {'video_path': video.file.path})
    else:
        form = VideoForm()
    return render(request, 'video/upload.html', {'form': form})

Here is where I encounter the most significant issue. Within this context, I have a 'live_video' view that utilizes 'StreamingHttpResponse' to provide a streaming response and reads data from the generator generated by the 'generate' function. Once the end marker is detected, I fetch the corresponding data from the database and aim to present it on a results page, subsequently returning the rendered page.

def live_video(request, video_path):
    response = StreamingHttpResponse(generate(Model(video_path)), content_type="multipart/x-mixed-replace; boundary=frame")

    data_path = video_path.split('\\')[-1][:-4]
    
    for chunk in response:
        if chunk == b'--end\r\n\r\n':
            # The end marker was received
            db_ref = db.reference(data_path)
            data = db_ref.get()
            print(f"Team A Data After Streaming \n {data['Team A']}")
            print(f"Team A Score After Streaming \n {data['Score A']}")
            print(f"Team B Data After Streaming \n {data['Team B']}")
            print(f"Team B Score After Streaming \n {data['Score B']}")

            context = {
                'Team_A': data['Team A'],
                'Team_B': data['Team B'],
                'Score_A': int(data['Score A']),
                'Score_B': int(data['Score B'])
            }

            return render(request, 'video/results.html', context)

    # If the end marker was not received, return an appropriate response
    return HttpResponse("The stream ended without receiving the end marker.")

Following are the problems I am facing:

1. If I refrain from iterating over the response, the video streams smoothly, but I'm unable to extract data from the generator or firebase.
2. On the other hand, when I iterate over the response, the streamed video disappears. While I can retrieve data from the database, accessing data from the generator remains a challenge. I'm aiming to display these results on the results page, but I'm currently facing difficulties in rendering them."

I tried many ways to resolve this issue and I would like some suggestions and guidelines to how I can achieve this.