I'm trying to create two camera previews in the same fragment layout, but I've encountered some strange issues, like:
- the preview freezes and the camera preview isn't changed,
- sometimes camera doesn't start at all.
Here is my custom SurfaceView for the Camera:
@SuppressWarnings("deprecation")
public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback {
private static final String TAG = "CameraPreview";
private SurfaceHolder mHolder;
private Camera mCamera;
private List<Camera.Size> mSupportedPreviewSizes;
private Camera.Size mPreviewSize;
private int currentVolume;
private boolean isVolumeChanged = false;
public CameraPreview(Context context, Camera mCamera) {
super(context);
setCamera(mCamera);
init();
}
public CameraPreview(Context context) {
super(context);
}
public CameraPreview(Context context, AttributeSet attrs) {
super(context, attrs);
}
public CameraPreview(Context context, AttributeSet attrs, int defStyleAttr) {
super(context, attrs, defStyleAttr);
}
public CameraPreview(Context context, AttributeSet attrs, int defStyleAttr, int defStyleRes) {
super(context, attrs, defStyleAttr, defStyleRes);
}
/**
* Initialize the SurfaceView which will be used to display the camera preview.
* <p/>
* Only an general setup for the SurfaceView should be done in this method.
*/
private void init() {
setDrawingCacheEnabled(true);
}
/**
* Set the camera to the SurfaceView.
*
* @param camera
* The camera object which will be set to the SurfaceView.
*/
public void setCamera(Camera camera) {
mCamera = camera;
// supported preview sizes
mSupportedPreviewSizes = mCamera.getParameters().getSupportedPreviewSizes();
// Install a SurfaceHolder.Callback so we get notified when the
// underlying surface is created and destroyed.
mHolder = getHolder();
mHolder.addCallback(this);
// deprecated setting, but required on Android versions prior to 3.0
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
}
public void surfaceCreated(SurfaceHolder holder) {
// empty. surfaceChanged will take care of stuff
// The Surface has been created, now tell the camera where to draw the preview.
try {
mCamera.setPreviewDisplay(holder);
mCamera.startPreview();
} catch (IOException e) {
LogUtils.d(TAG, "Error setting camera preview: " + e.getMessage());
}
}
public void surfaceDestroyed(SurfaceHolder holder) {
// empty. Take care of releasing the Camera preview in your activity.
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
// If your preview can change or rotate, take care of those events here.
// Make sure to stop the preview before resizing or reformatting it.
if (mHolder.getSurface() == null) {
// preview surface does not exist
return;
}
// stop preview before making changes
try {
mCamera.stopPreview();
} catch (Exception e) {
// ignore: tried to stop a non-existent preview
}
// set preview size and make any resize, rotate or reformatting changes here
try {
Camera.Parameters parameters = mCamera.getParameters();
parameters.setPreviewSize(mPreviewSize.width, mPreviewSize.height);
// set the orientation of the pictures taken
//TODO: match this orientation value with the one used for display
parameters.setRotation(90);
// set the focus mode
parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
mCamera.setParameters(parameters);
} catch (RuntimeException e) {
// This could happen when concurrency of the camera is happening (especially in dual shot mode)
//TODO: optimize the camera flow to avoid this issue
LogUtils.d(TAG, "Error setting camera parameters: " + e.getMessage());
}
// Force the orientation in Portrait mode
// TODO: get the orientation from the device
mCamera.setDisplayOrientation(90);
// start preview with new settings
try {
mCamera.setPreviewDisplay(mHolder);
mCamera.startPreview();
} catch (IOException e) {
LogUtils.e(TAG, "Error starting camera preview: " + e.getMessage());
}
}
/**
* Disable the default shutter sound when taking a picture.
*/
private void disableShutterSound() {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
boolean shutterState = mCamera.enableShutterSound(false);
LogUtils.i(TAG, "Shutter sound was" + (!shutterState ? "not " : " ") + "disabled");
} else {
LogUtils.i(TAG, "Trying to disable shutter sound by altering the system audio manager.");
// Backward compatibility method for disabling the shutter sound
AudioManager audio = (AudioManager) getContext().getSystemService(Context.AUDIO_SERVICE);
currentVolume = audio.getStreamVolume(AudioManager.STREAM_SYSTEM);
audio.setStreamVolume(AudioManager.STREAM_SYSTEM, 0, AudioManager.FLAG_REMOVE_SOUND_AND_VIBRATE);
MediaPlayer media = MediaPlayer.create(getContext(), R.raw.camera_shutter_click);
media.setAudioStreamType(AudioManager.STREAM_NOTIFICATION);
isVolumeChanged = true;
}
}
@Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
if (mCamera != null) {
final int width = resolveSize(getSuggestedMinimumWidth(), widthMeasureSpec);
final int height = resolveSize(getSuggestedMinimumHeight(), heightMeasureSpec);
if (mSupportedPreviewSizes != null) {
mPreviewSize = getOptimalPreviewSize(mSupportedPreviewSizes, width, height);
}
float ratio;
if (mPreviewSize.height >= mPreviewSize.width) {
ratio = (float) mPreviewSize.height / (float) mPreviewSize.width;
} else {
ratio = (float) mPreviewSize.width / (float) mPreviewSize.height;
}
// One of these methods should be used, second method squishes preview slightly
// setMeasuredDimension(width, width);
setMeasuredDimension(width, (int) (width * ratio));
// setMeasuredDimension((int) (width * ratio), height);
} else {
setMeasuredDimension(widthMeasureSpec, heightMeasureSpec);
}
}
private Camera.Size getOptimalPreviewSize(List<Camera.Size> sizes, int w, int h) {
final double ASPECT_TOLERANCE = 0.1;
double targetRatio = (double) h / w;
if (sizes == null) {
return null;
}
Camera.Size optimalSize = null;
double minDiff = Double.MAX_VALUE;
int targetHeight = h;
for (Camera.Size size : sizes) {
double ratio = (double) size.height / size.width;
if (Math.abs(ratio - targetRatio) > ASPECT_TOLERANCE) {
continue;
}
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
if (optimalSize == null) {
minDiff = Double.MAX_VALUE;
for (Camera.Size size : sizes) {
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
}
return optimalSize;
}
public void releaseCamera() {
if (mCamera != null) {
mCamera.stopPreview();
mCamera.setPreviewCallback(null);
mCamera.release();
mCamera = null;
}
}
/**
* Check if the Camera object was passed and set to the SurfaceView.
*
* @return {@code true} if the Camera object is valid, {@code false} otherwise
*/
public boolean isCameraSet() {
return null != mCamera;
}
public void takePicture() {
if (null != mCamera) {
// Disable the shutter sound when taking an picture
disableShutterSound();
mCamera.takePicture(shutterCallback, rawCallback, postViewCallback, jpegCallback);
}
}
private void resetCam() {
if (null != mCamera) {
mCamera.startPreview();
setCamera(mCamera);
}
}
/**
* The shutter callback occurs after the image is captured
*/
private Camera.ShutterCallback shutterCallback = new Camera.ShutterCallback() {
public void onShutter() {
MediaPlayer.create(getContext(), R.raw.camera_shutter_click).start();
LogUtils.d(TAG, "onShutter");
}
};
/**
* The raw callback occurs when the raw image data is available.<br/>(<b>NOTE:</b> the data will be null if there is
* no raw image callback buffer available or the raw image callback buffer is not large enough to hold the raw
* image).
*/
private Camera.PictureCallback rawCallback = new Camera.PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
LogUtils.d(TAG, "onPictureTaken - raw");
}
};
/**
* The jpeg callback occurs when the compressed image is available.
*/
private Camera.PictureCallback jpegCallback = new Camera.PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
if (isVolumeChanged) {
// Reset the audio settings which where set before trying to take an picture
AudioManager audio = (AudioManager) getContext().getSystemService(Context.AUDIO_SERVICE);
audio.setStreamVolume(AudioManager.STREAM_SYSTEM, currentVolume,
AudioManager.FLAG_REMOVE_SOUND_AND_VIBRATE);
// reset flag for volume control
isVolumeChanged = false;
}
new CameraSavePicture().execute(data);
resetCam();
LogUtils.d(TAG, "onPictureTaken - jpeg");
}
};
/**
* The postview callback occurs when a scaled, fully processed postview image is available. <br/> (<b>NOTE:</b> not
* all hardware supports this)
*/
private Camera.PictureCallback postViewCallback = new Camera.PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
LogUtils.d(TAG, "onPictureTaken - postview");
}
};
}
This is the relevant part of the layout for the fragment:
<RelativeLayout
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/black">
<FrameLayout
android:id="@+id/full_camera_preview"
android:layout_width="match_parent"
android:layout_height="match_parent"
/>
<FrameLayout
android:id="@+id/small_camera_preview"
android:layout_width="100dp"
android:layout_height="100dp"
android:layout_alignParentTop="true"
android:layout_alignParentStart="true"
android:layout_alignParentLeft="true"
/>
</RelativeLayout>
And this is the Camera fragment:
public class CameraFragment extends Fragment implements View.OnClickListener, EventBusInterface {
private static final String TAG = "Camera Fragment";
private ImageButton flipCameraButton;
private Button takePicture;
private RelativeLayout content;
private RelativeLayout controlPanel;
private CameraPreview fullCameraPreview;
private FrameLayout fullCameraLayout;
private CameraPreview smallCameraPreview;
private FrameLayout smallCameraLayout;
/**
* The mode of the camera which shoul initially be displayed.
*/
private CameraMode currentCameraMode = CameraMode.BACK;
/**
* Do not use to instantiate a fragment. Use newInstance method instead.
*/
public CameraFragment() {
}
/**
* Use this method to get a new instance of the fragment, and give eventually add parameters if you need to pass
* data to it and retrieve it in onCreate using getArguments.
*
* @return a new instance of the current fragment.
*/
public static CameraFragment newInstance() {
final CameraFragment cameraFragment = new CameraFragment();
Bundle arguments = new Bundle();
cameraFragment.setArguments(arguments);
return cameraFragment;
}
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Get the argument passed on the custom initialization of the fragment
Bundle arguments = getArguments();
if (null != savedInstanceState) {
// Get the last camera mode set by the user
CameraMode savedCameraMode = (CameraMode) savedInstanceState.getSerializable("CAMERA_MODE");
if (null != savedCameraMode) {
currentCameraMode = savedCameraMode;
}
}
// setRetainInstance(true); //Will ignore onDestroy Method (Nested Fragments no need this if parent
// have it)
}
@Override
public void onSaveInstanceState(Bundle outState) {
outState.putSerializable("CAMERA_MODE", currentCameraMode);
super.onSaveInstanceState(outState);
}
@Override
public void onViewStateRestored(Bundle savedInstanceState) {
super.onViewStateRestored(savedInstanceState);
}
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
View view = inflater.inflate(R.layout.fragment_timeline_camera, container, false);
// BUG FIX: To avoid the window background overlapping this fragment we set it's background to transparent so
// that when the fragment is moved(scrolled) it will still be visible
this.getActivity().getWindow().setBackgroundDrawable(new ColorDrawable(android.graphics.Color.TRANSPARENT));
initViews(view);
setListeners();
// Setup the camera mode which will initially be displayed.
setUpCameraMode(currentCameraMode);
return view;
}
@Override
public void onResume() {
super.onResume();
}
@SuppressWarnings("deprecation")
private void setUpCameraMode(final CameraMode cameraMode) {
// The front camera SurfaceView will need to be changed, release it first
releaseSmallCamera();
// The full SurfaceView has to be changed because the camera mode will be changed from Back to Front or
// Front to Back
releaseFullCamera();
resizeFullCamera(cameraMode == CameraMode.DUAL);
switch (cameraMode) {
case BACK:
setupFullCamera(CameraInfo.CAMERA_FACING_BACK);
break;
case FRONT:
setupFullCamera(CameraInfo.CAMERA_FACING_FRONT);
break;
case DUAL:
setUpDualCamera();
break;
}
}
@Override
public void onStart() {
super.onStart();
if (!EventBus.getDefault().isRegistered(this)) {
EventBus.getDefault().register(this);
}
// This is required because we need to resume the camera when we get back in the fragment.
if (null != getView()) {
// Setup the camera mode which will initially be displayed.
setUpCameraMode(currentCameraMode);
}
}
@Override
public void onStop() {
EventBus.getDefault().unregister(this);
super.onStop();
// The front camera SurfaceView will need to be changed, release it first
releaseSmallCamera();
// The full SurfaceView has to be changed because the camera mode will be changed from Back to Front or
// Front to Back
releaseFullCamera();
}
/**
* Initialize the view for the current fragment.
*
* @param view
* The view which was inflated for this fragment.
*/
private void initViews(View view) {
controlPanel = (RelativeLayout) view.findViewById(R.id.control_panel);
fullCameraLayout = (FrameLayout) view.findViewById(R.id.full_camera_preview);
smallCameraLayout = (FrameLayout) view.findViewById(R.id.small_camera_preview);
flipCameraButton = (ImageButton) view.findViewById(R.id.flip_camera_button);
takePicture = (Button) view.findViewById(R.id.camera_take_picture);
}
/**
* Setup the listeners required for this fragment.
*/
private void setListeners() {
flipCameraButton.setOnClickListener(this);
takePicture.setOnClickListener(this);
}
/**
* Sets up the SurfaceView where the Full preview will be display. <br/> The full SurfaceView can show the camera
* that faces the same direction as the screen and the camera tjat faces the opposite direction as the screen.
*
* @param facingOfTheCamera
* The facing of the camera that has to be shown in the Full SurfaceView. <br/>Usually {@link
* CameraInfo#CAMERA_FACING_BACK} or {@link CameraInfo#CAMERA_FACING_FRONT}.<br/> If an invalid value is
* set, the preview for the back camera will be setup.
*/
@SuppressWarnings("deprecation")
private void setupFullCamera(int facingOfTheCamera) {
// Full camera can show the back & front cameras
CameraMode cameraMode;
if (facingOfTheCamera == CameraInfo.CAMERA_FACING_BACK) {
cameraMode = CameraMode.BACK;
} else if (facingOfTheCamera == CameraInfo.CAMERA_FACING_FRONT) {
cameraMode = CameraMode.FRONT;
} else {
// Default to the back camera
cameraMode = CameraMode.BACK;
}
if (null == fullCameraPreview) {
fullCameraPreview = new CameraPreview(getActivity(), CameraUtils.getCamera(cameraMode));
}
if (fullCameraLayout.getVisibility() != View.VISIBLE) {
fullCameraLayout.setVisibility(View.VISIBLE);
}
if (fullCameraLayout.getChildCount() == 0) {
// The View for displaying the Camera preview was not added in the layout, we need to add it now.
fullCameraLayout.addView(fullCameraPreview);
}
}
/**
* Sets up the SurfaceView where the Small preview will be display. <br/> The small SurfaceView will always show the
* camera that faces the same direction as the screen.
*/
private void setupSmallCamera() {
// Small camera always shows the camera that faces the same as the screen
if (null == smallCameraPreview) {
smallCameraPreview = new CameraPreview(getActivity(), CameraUtils.getCamera(CameraMode.FRONT));
}
smallCameraPreview.setZOrderOnTop(true);
// smallCameraPreview.setZOrderMediaOverlay(true);
if (smallCameraLayout.getVisibility() != View.VISIBLE) {
smallCameraLayout.setVisibility(View.VISIBLE);
}
if (smallCameraLayout.getChildCount() == 0) {
// The View for displaying the Camera preview was not added in the layout, we need to add it now.
smallCameraLayout.addView(smallCameraPreview);
}
}
/**
* Setup Full SurfaceView preview (display Back camera) and Small SurfaceView preview (display Front camera).
*/
@SuppressWarnings("deprecation")
private void setUpDualCamera() {
// In dual mode camera preview display, the full preview is showing the back camera & the small preview
// displays the front camera
setupFullCamera(CameraInfo.CAMERA_FACING_BACK);
setupSmallCamera();
}
private void resizeFullCamera(boolean shrink) {
// The dual camera doesn't work if two SurfaceViews overlap - use this to resize the full camera so that it
// is not overlapped by the small camera
RelativeLayout.LayoutParams p = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.MATCH_PARENT);
if (shrink) {
p.addRule(RelativeLayout.BELOW, R.id.small_camera_preview);
} else {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
p.removeRule(RelativeLayout.BELOW);
} else {
p.addRule(RelativeLayout.BELOW, 0);
}
}
fullCameraLayout.setLayoutParams(p);
}
/**
* Release the Camera Object used for the full camera mode. <br/> Remove the SurfaceView from the layout and
* invalidate it.
*/
private void releaseFullCamera() {
if (null != fullCameraPreview) {
fullCameraPreview.releaseCamera();
}
if (null != fullCameraLayout) {
// Remove the SurfaceView from the layout
fullCameraLayout.removeAllViews();
fullCameraLayout.setVisibility(View.INVISIBLE);
}
// Invalidate the SurfaceView
fullCameraPreview = null;
}
/**
* Release the Camera Object used for the small camera mode. <br/> Remove the SurfaceView from the layout and
* invalidate it.
*/
private void releaseSmallCamera() {
if (null != smallCameraPreview) {
smallCameraPreview.releaseCamera();
}
if (null != smallCameraLayout) {
// Remove the SurfaceView from the layout
smallCameraLayout.removeAllViews();
smallCameraLayout.setVisibility(View.INVISIBLE);
}
// Invalidate the SurfaceView
smallCameraPreview = null;
}
/**
* Flip between the camera modes supported. <br/> The flip is like an infinite carousel.
*/
private void flipCamera() {
// Set up the next mode for the camera preview/s display
setUpCameraMode(currentCameraMode.getNext());
// Update the current mode of the camera preview/s display
currentCameraMode = currentCameraMode.getNext();
}
@Override
public void onClick(View v) {
final int id = v.getId();
switch (id) {
case R.id.flip_camera_button:
// Flip camera to the next mode available.
flipCamera();
break;
case R.id.camera_take_picture:
takePicture();
break;
}
}
private void takePicture() {
switch (currentCameraMode) {
case BACK:
case FRONT:
if (null != fullCameraPreview) {
fullCameraPreview.takePicture();
} else {
ToastModel toastModel = new ToastModel(ToastType.GENERAL, "Please wait for camera warming up!");
new CustomToast(getActivity()).displayToast(toastModel);
}
break;
case DUAL:
ToastModel toastModel = new ToastModel(ToastType.GENERAL, "Dual picture not supported yet!");
new CustomToast(getActivity()).displayToast(toastModel);
break;
}
}
@Override
public void onEvent(Object event) {
}
@Override
public void onEventMainThread(Object event) {
if (event instanceof String) {
ToastModel toastModel = new ToastModel(ToastType.GENERAL, (String) event);
new CustomToast(getActivity()).displayToast(toastModel);
}
}
@Override
public void onEventBackgroundThread(Object event) {
}
@Override
public void onEventAsync(Object event) {
}
}
What could be the issue for this weird behavior? Currently I'm not trying to show both previews at the same time, but one by one.
LE: I have updated the logic for how the flip between camera modes is done. When only the front or back camera are displayed in the full SurfaceView everything works fine. The problem appears when I use both of them (not necessarily both Camera objects at the same time, but both SurfaceViews visible at the same time & simply change from one to another). I noticed the following errors in the logs:
D/Camera﹕ [seungmin]_open start
D/Camera﹕ [China_security]_before
D/Camera﹕ [China_security]_after
D/Camera﹕ [seungmin]_open End
D/CameraPreview﹕ Error setting camera parameters: getParameters failed (empty parameters)
W/CameraBase﹕ mediaserver's remote binder Camera object died
W/CameraBase﹕ Camera service died!
W/CameraBase﹕ mediaserver's remote binder Camera object died
E/Camera﹕ Error 100
E/Camera﹕ Error 100
LE2: I have updated my code & optimized the flow. I have also figured out why the Camera dies - it was because the SurfaceView for the second Camera was placed on top of the first one. Once the SurfaceViews didn't overlap anymore everything started working fine. My question now is how can I fix this issue? Why is this happening?
PS: Most of the tests where made on LG G2 device which supports dual shot directly.
You may have stumbled upon an Android limitation regarding overlapping
SurfaceViews. You can read more about this limitation here.Don't think of
SurfaceViewsas classic views, they function more like a placeholder for getting the screen area (position and size) you want to surrender control to a rendering component. Rendering is actually done on a separate window.As a solution, you can try calling
setZOrderOnTop(true)on theSurfaceViewyou want on top.This isfalseby default, so the 2SurfaceViewswill not be on the same z anymore - making them work; one drawback is that the one on top will also be on top of any view you have.If they do end up working, this is not guaranteed to happen on all devices, so I recommend extensive testing on what devices you consider important.
Another thing you can research is to see if you can tap into the
Camerafeed and see if you can merge the 2 streams and feeding the result to 1SurfaceView