multiple images in a single frame in opencv

59 views Asked by At

I extracted a video from a VHS tape. The horizontal sync didn't trigger properly in many of the lines causing that line to be shifted, usually to the right. I wrote some code in an attempt to detect the black left edge and shift frames. That mostly worked, but some frames look like they consist of two images.

frame with multiple images

Notice that the speaker's hair appears to have two exposures; one which has a horizontal shift and another that doesn't. Is this a problem with the video capture device? Do you know of any mitigation? I wouldn't mind paying for software instead of rolling my own.

I don't think important to the question, but this is the code that I'm using per frame in the hope that it will be of use to others. Really just a piecing together of SO code and Bing AI.


def apply_shift(img, column_shifts):
    "Once a left-right shift is established per row, apply and pad with the last pixel"
    rolled = np.empty_like(img)
    for i in range(img.shape[0]):
        rolled[i] = np.roll(img[i], column_shifts[i], axis=0)
    # Replace the rolled values with the end value of each row
    for i in range(img.shape[0]):
        if column_shifts[i] > 0:
            rolled[i, :column_shifts[i]] = rolled[i, column_shifts[i]]
        elif column_shifts[i] < 0:
            rolled[i, column_shifts[i]:] = rolled[i, column_shifts[i]]
    return rolled

def add_counter_to_frame(frame, counter):

    img = Image.fromarray(frame)
    draw = ImageDraw.Draw(img)
    font = ImageFont.load_default()
    draw.text((350, 50), f"{counter}", fill='black', font=font)
    return np.array(img)

G_frame_counter:int = 0
def straighten_frame(frame):
    global G_frame_counter
    G_frame_counter += 1
    frame = add_counter_to_frame(frame, G_frame_counter)
    gray = np.dot(frame[...,:3], [0.2989, 0.5870, 0.1140])
    threshold = 75   # (from 0 to 255)
    wanted_pixel_at_threshold = 4 # Force all scan lines to have their threshold crossing at this pixel
    max_shift = 4 # Shift no more than max_shift pixels
    clean_sigma = 2
    indices = np.argmax(gray > threshold, axis=1)
    shift_amount = wanted_pixel_at_threshold-indices
    shift_amount[shift_amount < -max_shift] = -max_shift
    shift_amount[shift_amount > max_shift] = max_shift
    cleaned_shift_amount = gaussian_filter(shift_amount, sigma=clean_sigma)
    rolled = apply_shift(frame, cleaned_shift_amount)
    return rolled
0

There are 0 answers