How to remove constant part of a signal in python?

Question

I have a dataset where it is constant sometimes with small noise. It can be on the top, it can be in the middle. I want to remove all constant part of the signal so that it compress into a smaller length.

Lets say if 200 consicutive value is same (or in a threshold limit to be considered as constant) then consider that part as constant .

Any solution code or logical suggestion will be appriciated. Thanks for your time.

I have attached a sample data google drive link in .npy format.

In the below sample image I have indicated (red) which part I want to get rid of as an example:

Answer 1

I came up with the following solution. The answer from @rsenne helped me to start but that was not the solution I have wanted. Please let me know if there anything I should add in my solution to improve the performance.

import numpy as np

def remove_flat_segments(data, threshold, min_length):
       
       """
       Remove flat segments from data within a threshold and with a minimum length.
    
       Parameters:
        data (ndarray): Input data array.
        threshold (float): Threshold for flatness.
        min_length (int): Minimum length of flat segment to be removed.
    
       Returns:
        ndarray: Data with flat segments removed.
       """
    flat_segments = []
    start_idx = None
    
    for i in range(len(data)):
        if start_idx is None:
            start_idx = i
        elif abs(data[i] - data[i-1]) > threshold:
            if i - start_idx >= min_length:
                flat_segments.append((start_idx, i-1))
            start_idx = None
    
    if start_idx is not None and len(data) - start_idx >= min_length:
        flat_segments.append((start_idx, len(data)-1))
    
    if len(flat_segments) > 0:
        flat_indices = np.concatenate([np.arange(start, end+1) for start, end in flat_segments])
        return np.delete(data, flat_indices)
    else:
        return data


threshold = 0.01
min_length = 1000

filtered_data = remove_flat_segments(data, threshold, min_length)

The resulted image looks like:

Answer 2

Not the most elegant solution but this just slides a 200 length window over the signal, looks at the variance, and any variance over a specific magnitude is considered non-constant. There is probably ways this could be modified but with very little effort I was able to remove a lot of the constant.

import numpy as np
import matplotlib.pyplot as plt

x = np.load(r"C:\Users\ryansenne\Downloads\sample.npy")

# Detects constant sections and removes them
def remove_constant_sections(signal, window_size=200, threshold=0.01):
    filtered_signal = []
    i = 0
    while i < len(signal) - window_size:
        window = signal[i:i+window_size]
        if np.var(window) > threshold:
            filtered_signal.extend(window)
            i += window_size
        else:
            i += 1
    # Add the remaining values that may not fit into a full window
    filtered_signal.extend(signal[i:])
    return np.array(filtered_signal)


filtered_signal = remove_constant_sections(x)