IBasicVideoEffect processes only form 3 to 8 first frames

Question

I created a simple test UWP app with one MainPage, that has MediaPlayer:

public sealed partial class MainPage
{
    public MainPage()
    {
        InitializeComponent();
        Loaded += MainPage_Loaded;
    }

    private void MainPage_Loaded(object sender, RoutedEventArgs e)
    {
        var mediaPlayer = new MediaPlayer
        {
            Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/preview.mp4")),
            AutoPlay = true
        };
        mediaPlayer.AddVideoEffect(typeof(VideoEffect).FullName, true, null);
    }
}

and WinRT component with IBasicVideoEffect inherited class that notifies me how many frames were processed:

public sealed class VideoEffect : IBasicVideoEffect
{
    public IReadOnlyList<VideoEncodingProperties> SupportedEncodingProperties => new List<VideoEncodingProperties>();

    public bool IsReadOnly => false;

    public MediaMemoryTypes SupportedMemoryTypes => MediaMemoryTypes.Gpu;

    public void SetProperties(IPropertySet configuration) { }

    public bool TimeIndependent => false;

    public void Close(MediaEffectClosedReason reason) { }

    public void DiscardQueuedFrames() { }

    private int _frameCounter;

    public void ProcessFrame(ProcessVideoFrameContext context)
    {
        _frameCounter++;
        Debug.WriteLine("Frame #" + _frameCounter);
    }

    public void SetEncodingProperties(VideoEncodingProperties encodingProperties, IDirect3DDevice device)
    {
        Debug.WriteLine("SetEncodingProperties");
    }
}

If I run it - only 3 frames will be processed no matter what video file will be.

If I set breakpoint where _frameCounter increments I'll manage to hit F5 for 8 frames.

Why and how can I get all the frames to be processed?

I can solve it using MediaClip and MediaComposition as many examples say, but in this case frames are processed by CPU not GPU video engine which is not my goal.

Answer 1

The problem here is that you are playing a MediaPlayer without displaying it in XAML. So only the first few frames was processed as the media is not rendering on UI, there is no need to process other frames.

To make your VideoEffect work, you can use MediaPlayerElement control with MediaPlayerElement.SetMediaPlayer method to to render the media.

XAML:

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <MediaPlayerElement x:Name="mediaPlayerElement" />
</Grid>

Code-behind:

private void MainPage_Loaded(object sender, RoutedEventArgs e)
{
    var mediaPlayer = new MediaPlayer
    {
        Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/preview.mp4")),
        AutoPlay = true
    };
    mediaPlayer.AddVideoEffect(typeof(VideoEffect).FullName, true, null);

    mediaPlayerElement.SetMediaPlayer(mediaPlayer);
}

After this, you will see _frameCounter increases while the media is playing.

Answer 2

with MediaPlayer you have to add effect before set source.

effect.AddVideoEffect(_player);
_player.Source = MediaSource.CreateFromMediaStreamSource(source);