In VLC-Unity, why is the media player unable to play video feed from USB camera using DirectShow?

Question

I'm experimenting the capabilities of VLC-Unit for various video streaming protocol, the simplest of which should be playing the video feed of a USB camera directly.

For this purpose, I start with a simple example implementation (compiled and executed under Windows):

https://github.com/videolan/vlc-unity/blob/master/Assets/VLCUnity/Demos/Scripts/VLCMinimalPlayback.cs

This demo was written to play a video on GoogleDrive using http protocol:

            if(_mediaPlayer.Media == null)
            {
                // playing remote media
                _mediaPlayer.Media = new Media(new Uri("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"));
            }

I made the following change to repurpose it for the USB camera:

            if (_mediaPlayer.Media == null)
            {
                // playing remote media

                // _mediaPlayer.Media = new Media(new Uri("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"));

                _mediaPlayer.Media =
                    new Media(
                        new Uri("dshow://")
                    );
                _mediaPlayer.Media.AddOption(":dshow-vdev=<My USB Camera Name>);

                Task.Run(async () =>
                    {
                        var success = await _mediaPlayer.PlayAsync();

                        uint height = 0;
                        uint width = 0;
                        _mediaPlayer.Size(0, ref width, ref height);

                        Debug.Log("Media size: " + width + "x" + height);
                    }
                );
            }

Unfortunately, when the example is executed, I got the following debug info:

Media size: 0x0

Obviously, no video could be displayed in this case (audio feed seems to be normal). This phenomenon can be observed even if <My USB Camera Name> was single or double quoted. What may have caused the VLC-Unity to fail on such simple case? How to fix it?

UPDATE 1: The camera is fully functional when being opened from within VLC player, the log snippet (debug-level verbosity) is show here:

main debug: Creating an input for 'dshow://'
main debug: requesting art for new input thread
main debug: using timeshift granularity of 50 MiB
main debug: using timeshift path: C:\Users\pengc\AppData\Local\Temp
main debug: `dshow://' gives access `dshow' demux `any' path `'
main debug: creating demux: access='dshow' demux='any' location='' file='(null)'
main debug: looking for access_demux module matching "dshow": 15 candidates
main debug: looking for meta fetcher module matching "any": 1 candidates
dshow debug: dshow-vdev: <Camera Name>
dshow debug: dshow-adev: none

Update 1: I guess is not a very specific example. So I uploaded 2 minimalistic experiments on Github. Both of the following experiments were conducted on Windows 11, with both VLC player & OBS Studio installed (so OBS Virtual Camera is always the default capture device)

Experiment 1: OBS Virtual Camera

https://github.com/tribbloid/VLCUnitySpike/blob/main/Assets/Scripts/VLCMinimalPlaybackOBS.cs

the media player was created by:

                _mediaPlayer.Media = new Media(
                    new Uri(
                        "dshow://"
                    )
                );
                
                _mediaPlayer.Media.AddOption(":dshow-vdev=\"OBS Virtual Camera\"");
                _mediaPlayer.Media.AddOption(":dshow-adev=None");

It turns out that the Unity application can only play the first frame, displayed video immediately freeze despite continuous buffer updating. Regardless, it has found the correct camera.

Experiment 2: Webcam

https://github.com/tribbloid/VLCUnitySpike/blob/main/Assets/Scripts/VLCMinimalPlaybackWebcam.cs

Similar to the first example, except the following lines:

                _mediaPlayer.Media = new Media(
                    new Uri(
                        "dshow://"
                    )
                );
                
                _mediaPlayer.Media.AddOption(":dshow-vdev=\"USB2.0 HD UVC WebCam\"");
                _mediaPlayer.Media.AddOption(":dshow-adev=None");

The outcome is identical to Experiment 1, the same default OBS camera was used, and only the first frame was displayed. I haven't tested when all arguments are removed but the result should be identical.

This repository can be opened by any Unity Editor 2022 LTS, the VLCUnity asset however has to be manually imported. I imported the proprietary release but the trial release should work equally well.

So far the cause for this problem is still unknown. I am keen to know if others can replicate it.

UPDATE 2: I can upload the trial release of the asset (which should have an open-source license) for easily reproducing my problem.

Answer 1

Found the problem: there is no need to quote the video capture device name in the API, because each string can only contain 1 argument, and there is no need to escape space.

The correct program snippet is:

                _mediaPlayer.Media.AddOption(":dshow-vdev=USB2.0 HD UVC WebCam");
                _mediaPlayer.Media.AddOption(":dshow-adev=None");

The OBS camera issue is a known bug, unfortunately it cannot be used until a later version

Answer 2

1. Thread Safety: When you're using Task.Run(), you're running code on a different thread, which might cause unexpected behaviors in Unity. Especially given that you're accessing _mediaPlayer.Size() within this task. This could potentially be an issue.

2. Check Media Error: It's possible there was an error creating the media. After setting your media, check for errors:

   if (_mediaPlayer.Media != null) { Debug.LogError(_mediaPlayer.Media.ErrorMessage()); }

3. Camera Access: A stupid one, I know, but sometimes it's the small stuff.. If the camera is being used by another application (like a default camera app), it might not be available to VLC. Ensure that no other applications are using the camera.

4. Delaying Size Fetching: Perhaps the video dimensions aren't immediately available after you start the playback. Instead of immediately trying to get the width and height after starting playback, you could delay this:

   await Task.Delay(1000); // wait for 1 second _mediaPlayer.Size(0, ref width, ref height);