how increase quality and decrease frame rate in webrtc?

1.2k views Asked by At

we are using android and IOS devices to stream images to a backend for analysis. we opt to use webrtc to decrease delay and latency.

I succeed with changing focus and exposure settings on the fly and high constant resolution 2592x1944 frames while keeping bandwidth hovering around 1mbp.

now, we want to increase the quality (reduce compression artefact)and reduce the frame rate to 2-5 fps while/to keep bandwidth under control (1-2 mbp).

I am trying to do so via recompiling the webrtc library (currently only android) after changing the encoder parameters.

this is the bit of code I am plying with located in "HardwareVideoEncoder.java" in the webrtc project.

MediaFormat format = MediaFormat.createVideoFormat(codecType.mimeType(), width, height);
      format.setInteger(MediaFormat.KEY_BIT_RATE, adjustedBitrate);
      format.setInteger(KEY_BITRATE_MODE, 4);
      format.setInteger(MediaFormat.KEY_COLOR_FORMAT, colorFormat);
      format.setInteger(MediaFormat.KEY_FRAME_RATE, bitrateAdjuster.getCodecConfigFramerate());
      format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, keyFrameIntervalSec);
      if (codecType == VideoCodecMimeType.H264) {
        String profileLevelId = params.get(VideoCodecInfo.H264_FMTP_PROFILE_LEVEL_ID);
        Logging.e(TAG, "profileLevelId:" + profileLevelId);
        if (profileLevelId == null) {
          profileLevelId = VideoCodecInfo.H264_CONSTRAINED_HIGH_3_1;
        }
        switch (profileLevelId) {
          case VideoCodecInfo.H264_CONSTRAINED_HIGH_3_1:
            format.setInteger("profile", VIDEO_AVC_PROFILE_HIGH);
            format.setInteger("level", VIDEO_AVC_LEVEL_3);
            break;
          case VideoCodecInfo.H264_CONSTRAINED_BASELINE_3_1:
            break;
          default:
            Logging.w(TAG, "Unknown profile level id: " + profileLevelId);
        }

I tried to increase MediaFormat.KEY_BIT_RATE, changed bit rate mode, keyframe time and trying vp8,vp9 , H264 codec ( while changing sdp to) but I am not feeling that I am getting a different result. with some settings I am getting encoder init failed but the app still sends stream ( which I do not understand how is it possible)enter image description here

also, I am trying to change Chroma Subsampling to 4:4:4 to get better quality.

so my question is what are a good set of parameters/codec to use to achieve high quality, low compression and low frames rate?

also do we need to change/recopmile the decoder side to to achieve this or the encoder is enough ?

any articles/documentation/tutorials that will help me understand ( I am punching above my weight) and achieve this is appreciated

0

There are 0 answers