Parsing VP8/OPUS frames

Question

I am developing a webrtc simulator in Linux environment using C language. Here I need to send media from one webrtc simulator to other webrtc simulator. I need to work with VP8 and Opus codec. Now I am starting with VP8 codec. As I am new to VP8, kindly help me to understand more as well .

1. Since it is simulator, I do not want to capture / play media real time. I need to read audio / video data from a file and send it to remote. At remote get the media data extracted, save it to a file.

2. Which file format contains encoded VP8 data? Is it webm file format or something else?

3. Assuming webm file contains VP8 and OPUS codec data (which I am not sure), how can I parse frame by frame. For example if I read Extract audio frames from AMR-NB file I can parse amr frames from a file. Similarly is there any help page, where I learn parse logic for VP frame.

4. Or is there any open source using that I can parse VP8/OPUS frames from a webm file.

Answer 1

Q2: I'll start with question number 2, VP8 corresponds with WebM format.

Q3: If you want to parse frame by frame, you need to know the structure of VP8 and OPUS. I don't know what simulation means in your scenario, but in WebRTC environment media data is transfered via RTP. If you are doing simulation without RTP, please start from step 2, otherwise start from No. 1.

1. Depacketize RTP payload from RTP packet. RTP payload contains VP8 packet info like sequence number, timestamp, etc.
2. Note, that VP8 frame may consist of multiple VP8 packets, so basically you need to extract the frame content and concatenate it together. There are multiple ways to find out if frame consists of multiple packets, not all of them being fully standardized, but for now I would recommend you to use sequence number, as packets of one frame share the same SEQ.
3. Write the frame into file. If you want to create a valid WebM file, use function such as av_write_frame() from LibAV (which I personally use).

I can recommend you a piece of code that actually converts raw RTP VP8 packets to VP8 frames: https://github.com/meetecho/janus-gateway/blob/master/postprocessing/pp-webm.c . Packet header reading is used in functions (preprocess and process).

Q4: I use LibAV for such purposes (alternatively FFMPEG).

I'll gladly answer any other question.