I'm working on a project with Preview/Local Recording/Live Stream/AI features.

All those features might be activated/deactivated at any time.

and each stream have their own resolution at a certain frame rate with customized water marks.

So i'm searching for a solution for this,

and it would be wonderful if the solution can balanced in user experience and efficiency.

Potential solutions i learned.

----

[Android Camera API2]

By activate different target output surface into camera repeating request.

Pros:

  1. Easy to split to different resolution.

Cons:

  1. Replacing repeating request will introduce frame gap and noticeable jitters.
  2. Can not control fps for each output surface and may slow the whole request's fps down. (Too many output surface and ImageReader for yuv data extraction)
  3. Limited output surface supported. (Normally 3)
  4. Don't know how do frame post-processing for each stream like adding the water mark.

[Android Camera API2 + libyuv]

Adding one ImageReader to Camera API2 and copy ImageReader's data to several stream and use libyuv to scale/water-mark for each.

Pros:

  1. Fps controllable for each stream.
  2. Add/remove streams on the fly.

Cons:

  1. High CPU loading for using libyuv to sacle/water-mark.
  2. High CPU for too many data buffer coping and transferring.
  3. Thermal issue if 4K resolution involved.

[Android Camera API2 + OpenGLES]

  1. Generate a texture Object.
  2. Create a TextureSurface(texture Object).
  3. Feed TextureSurface to Camera API2.
  4. UpdateTexImage when FrameAvailable.

Adding a stream by:

  1. Create a thread using the shared GlContext.
  2. Create EGLSurface for the output surface.(SurfaceView/MediaCodec/ImageReader..)
  3. Set a periodical timer (fps controlling) to draw frame.
  4. When the bell rings.
  5. BindTexture and draw to current EGLSurface.
  6. Blend a bitmap as water-mark.
  7. if the stream need YUV420 outputs, i can implement rgbaToYuv by fragment shader ,then glReadPixels().

Removing a stream by:

  1. Release related resource.
  2. Terminate this Thread.

Pros:

  1. Dynamic add/remove streams without affect others.
  2. Easy fps controlling
  3. Easy adding water-mark

Cons:

  1. Too many GPU loading maybe?

I really not sure. I'm not very familiar with OpenGLES stuff.

So please comment.

How do you think about above solutions?

Which one is better?

and Any other approaches?

Pardon me if I am lacking in my understanding.

0

There are 0 answers