Enhance poor performance of decoding audio with MediaExtractor and MediaCodec to PCM

by saiday

Current situation

It takes 30 seconds for decoding a mp3 (file size: 4 MB, bit rate: 128 kbps, length: 4 minutes) file to PCM using MediaExtractor and MediaCodec, this is unbearable for user experience.

Problem

  1. MediaExtractor can only read samples by advance() and reads 418 bytes in a single step.
  2. MediaCodec's input buffer and output buffer lock the current thread on executing.

Real-world example:

4 MB file leads to 1,048,576 x 2 (input/output buffer) execution by MediaCodec which means 2,097,152 times thread locks. Of course we cannot have a good performance.

Solution

The goal is to reduce MediaCodec execution counts by collecting enough samples by MediaExtractor in memory then feeding it to MediaCodec. We can expect performance enhancement.

96% faster

If I collect 524,288 bytes (0.5 MB) for MediaCodec's input buffer instead of just-in-time's 418 bytes, now it's 96% faster.

HUGE.

Sample code:

Brief description:

1. inputFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, DECODE_INPUT_SIZE); for setting input buffer's size

2. collect samples from MediaExtractor and feed samples to MediaCodec on MediaCodec's onInputBufferAvailable() callback

3. decode to PCM on MediaCodec's onOutputBufferAvailable() callback

4. store decoded samples on mDecodedSamples