Detecting I-frame data in an MPEG-4 transport stream

Question

I am testing a project. I need to break the payload data(making zero some bytes) of the MPEG-4 ts packets by a percentage coming from the user. I am doing it by reading the ".ts" file packet by packet(188 bytes). But the video is changing to really mud after process. (By the way I'm writing the program in C)

So I decided to find the data/packets that belongs to I-frames, then not touching them but scrambling the other datas by percentage. I could find below

(in hex) 00 00 00 01 E0 start of video PES packet .. .. 00 00 01 B8 start of group of pictures header .. .. 00 00 01 00 the picture start code. This is 32 bits. The 10 bits immediately following this is called as the temporal reference. So temporal reference will include the byte following the picture start code and the first two bits of the second byte after the picture start code ie one byte(8 bits) + 2 bits. These we need to skip. Now the three bits present(3, 4 and 5th bits of the second byte from the picture start code) will indicate the Frame type ie I, B or P. So to get this simply logical AND & the second byte from the picture start code with 0x38 and right shift >> with 3.

For example the data is like that;

00 00 01 00 00 0F FF F8 00 00 01 B5........... and so on.

Here the first four bytes 00 00 01 00 is the picture start code. The fifth byte and the first two bits of the sixth byte is the temporal reference. So our concern is in the sixth byte --> 0F

((0F & 38)>>3)

Frame type = 1 ==> I Frame

Frame type 000 forbidden

Frame type 001 intra-coded (I) - iframe

Frame type 010 predictive-coded (P) - p frame

Frame type 011 bidirectionally-predictive-coded (B) - b frame

But this is for MPEG-2. Is there some patterns like that so I recognize and get the frame type with bitwise operations for MPEG-4 transport stream(extension is ".ts")?

And I need to get how many bytes or packets belong to that frame?

Thanks a lot for your help

Answer 1

I would parse the complete TS packet. So first determine what PID your video stream belongs to (by parsing the PAT and PMT). Then find keyframes by looking for the 'Random Access indicator' bit in the Adaptation Field.

uint8_t *pkt = <your 188 byte TS packet>;
assert( 0x47 == pkt[0] );
int16_t pid = ( ( pkt[1] & 0x1F) << 8 ) | pkt[2];
if ( pid == video_pid ) {
    // found video stream
    if( ( pkt[3] & 0x20 ) && ( pkt[4] > 0 ) ) {
        // have AF
        if ( pkt[5] & 0x40 ) {
          // found keyframe
} } }

Answer 2

If you are using H.264 there should be specific byte stream for I and P frame .. Like 0x0000000165 for I frame and 0x00000001XX for P frame .. So just parse and look for continuous such byte stream in such a way you can identify I or P frame.. Again above byte stream is codec implementation dependent .. For more information you can look into FFMPEG..