I extracted a video from a VHS tape. The horizontal sync didn't trigger properly in many of the lines causing that line to be shifted, usually to the right. I wrote some code in an attempt to detect the black left edge and shift frames. That mostly worked, but some frames look like they consist of two images.
Notice that the speaker's hair appears to have two exposures; one which has a horizontal shift and another that doesn't. Is this a problem with the video capture device? Do you know of any mitigation? I wouldn't mind paying for software instead of rolling my own.
I don't think important to the question, but this is the code that I'm using per frame in the hope that it will be of use to others. Really just a piecing together of SO code and Bing AI.
def apply_shift(img, column_shifts):
"Once a left-right shift is established per row, apply and pad with the last pixel"
rolled = np.empty_like(img)
for i in range(img.shape[0]):
rolled[i] = np.roll(img[i], column_shifts[i], axis=0)
# Replace the rolled values with the end value of each row
for i in range(img.shape[0]):
if column_shifts[i] > 0:
rolled[i, :column_shifts[i]] = rolled[i, column_shifts[i]]
elif column_shifts[i] < 0:
rolled[i, column_shifts[i]:] = rolled[i, column_shifts[i]]
return rolled
def add_counter_to_frame(frame, counter):
img = Image.fromarray(frame)
draw = ImageDraw.Draw(img)
font = ImageFont.load_default()
draw.text((350, 50), f"{counter}", fill='black', font=font)
return np.array(img)
G_frame_counter:int = 0
def straighten_frame(frame):
global G_frame_counter
G_frame_counter += 1
frame = add_counter_to_frame(frame, G_frame_counter)
gray = np.dot(frame[...,:3], [0.2989, 0.5870, 0.1140])
threshold = 75 # (from 0 to 255)
wanted_pixel_at_threshold = 4 # Force all scan lines to have their threshold crossing at this pixel
max_shift = 4 # Shift no more than max_shift pixels
clean_sigma = 2
indices = np.argmax(gray > threshold, axis=1)
shift_amount = wanted_pixel_at_threshold-indices
shift_amount[shift_amount < -max_shift] = -max_shift
shift_amount[shift_amount > max_shift] = max_shift
cleaned_shift_amount = gaussian_filter(shift_amount, sigma=clean_sigma)
rolled = apply_shift(frame, cleaned_shift_amount)
return rolled
