Can't write frame output with background removed to output video

Question

I'm attempting to use opencv and rembg to write an output video that walks through each frame and once it reaches a specific point in the video (start_frame) it runs rembg (remove background) on every frame until it reaches end_frame. It writes the output inline with the other frames. Essentially it will hit a point in the video and the background will disappear, once it reaches another point (end_frame) the background will reappear. I seem to be successful and it writes the video but none of the frames seem affected by the rembg that is run on frames x to x of the video. Am I missing something obvious? Here's my code:

import cv2
from rembg import remove

video_path = "D:\\frontloader.mp4"
video = cv2.VideoCapture("D:\\frontloaderShort.mp4")

w = int(video.get(3))
h = int(video.get(4))

print(w, h)

videoOut = cv2.VideoWriter('FrontLoaderOut.avi', cv2.VideoWriter_fourcc('M','J','P','G'), 24, (w, h))
    
start_frame = 200
end_frame = 230

while(True):
    sucess, frame = video.read()
    if(sucess):
        if video.get(cv2.CAP_PROP_POS_FRAMES) >= start_frame and video.get(cv2.CAP_PROP_POS_FRAMES) <= end_frame:
            print('no background')
            bg_removed = remove(frame)
            cv2.imshow("Frame", bg_removed)
            key = cv2.waitKey(1)
            videoOut.write(bg_removed)
        else:
            print('background')
            cv2.imshow("Frame", frame)
            key = cv2.waitKey(1)
            videoOut.write(frame)
    else:
        break

video.release()
videoOut.release()
cv2.destroyAllWindows() ```
 

Answer 1

The output of format of bg_removed = remove(frame) has 4 color channels and applies BGRA pixel format.

The last channel applies Alpha (transparency channel).
alpha = 0 is fully transparent.
alpha = 255 is fully opaque.
Values between 0 and 255 are scaled linearly.

MJPEG codec does not support transparency.
The best option is to scale the BGR values, by the alpha channel for getting black background (instead of transparent background).

if video.get(cv2.CAP_PROP_POS_FRAMES) >= start_frame and video.get(cv2.CAP_PROP_POS_FRAMES) <= end_frame:
    print('no background')
    bg_removed = remove(frame)
    
    # The format of bg_removed is BGRA, A channel applies Alpha (transparency channel).
    alpha = bg_removed[:, :, 3]
    alpha = cv2.cvtColor(alpha, cv2.COLOR_GRAY2BGR)  # Duplicate alpha to 3 channels before scaling.
    
    # Scale BGR channel by alpha channel (the background is going to be black).
    bg_removed_bgr = cv2.multiply(alpha, bg_removed[:, :, 0:3], scale=1/255)  # bg_removed_bgr = alpha/255 * bg_removed[:, :, 0:3]
    
    cv2.imshow("Frame", bg_removed_bgr)
    key = cv2.waitKey(1)
    videoOut.write(bg_removed_bgr)

---

Sample bg_removed_bgr frame: