I've been messing around with MediaCodec and MediaMuxer for a while now to record from a virtual display. I got this working nicely. Now I want to record sound from the microphone and add that to the video. I have found dosens of examples that does exactly what I want, however all these examples are making use of the deprecated parts of the MediaCodec. I am really confused how to refactor to the new api.
To start of I got a thread reading bytes from an AudioRecord like this.
while (running && recording) {
audioPresentationTimeNs = System.nanoTime();
iReadResult = mAudioRecord.read(mTempBuffer, 0, SAMPLES_PER_FRAME);
if(iReadResult == AudioRecord.ERROR_BAD_VALUE || iReadResult == AudioRecord.ERROR_INVALID_OPERATION)
Log.e(TAG, "audio buffer read error");
// send current frame data to encoder
try {
This wont work as dequeueInputBuffer cant be called async... So what will i need to do to get the right inputBufferIndex here?
int inputBufferIndex = audioEncoder.dequeueInputBuffer(-1);
if (inputBufferIndex >= 0) {
inputBuffer = audioEncoder.getInputBuffer(inputBufferIndex);
inputBuffer.clear();
inputBuffer.put(mTempBuffer);
if(VERBOSE)Log.d(TAG, "sending frame to audio encoder");
audioEncoder.queueInputBuffer(inputBufferIndex, 0, mTempBuffer.length, audioPresentationTimeNs / 1000, 0);
}
} catch (Throwable t) {
Log.e(TAG, "sendFrameToAudioEncoder exception");
t.printStackTrace();
}
}
Then I have my callback which is used by my AudioEncoder that should receive data from AudioRecord:
private class audioEncoderCallback extends MediaCodec.Callback{
Here should I probably receive the inputData and send to onOutputBufferAvailable somehow?
@Override
public void onInputBufferAvailable(MediaCodec codec, int index) {
}
This piece are supposed to deliver the data to the muxer
@Override
public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
if(muxerStarted) {
ByteBuffer encodedData = codec.getOutputBuffer(index);
encodedData.position(info.offset);
encodedData.limit(info.offset + info.size);
mMuxer.writeSampleData(audioTrackIndex, encodedData, info);
if (VERBOSE) Log.d(TAG, "AudioEncoder sent " + info.size + " bytes to mMuxer");
codec.releaseOutputBuffer(index, false);
}
}
@Override
public void onError(MediaCodec codec, MediaCodec.CodecException e) {
Log.d(TAG, "Error: " + e);
}
Am I thinking right about having both an audioTrackIndex and videoTrackIndex to deliver data into the muxer?
@Override
public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {
if (VERBOSE) Log.d(TAG, "encoder output format changed: " + format);
audioTrackIndex = mMuxer.addTrack(format);
if(videoTrackIndex != -1) {
muxerStarted = true;
mMuxer.start();
}
}
}