Deactivate iOS audio session when not in use?

Question

I have an iOS app that records segments of audio for specified periods of time, performs analysis over that audio, returns the results and then sits basically inactive until the user wants to record again.

The app's audio configuration is similar to that of aurioTouch2 and utilizes the RemoteI/O audio unit and Audio Session Services. The app currently sets AudioSessionSetActive(true) during initial startup and then starts the Remote I/O. The audio session and audio unit remain running throughout the lifetime of the app.

So here's my question—since the app is really only using the microphone for a relatively short (15-60 seconds) amount of time, should I deactivate the audio session via AudioSessionSetActive(false) when it's not being actively used?

I've tried running both the audio unit and then the audio session only during the recording process—starting when recording beings, and stopping when recording if finished—and although it works as expected on the simulator, it crashes when running it on a physical device. Here's the error:

*** Terminating app due to uncaught exception 'NSRangeException', reason: '-[__NSCFArray objectAtIndex:]: index (0) beyond bounds (0)'
*** First throw call stack:
(0x2e4baf4b 0x3884a6af 0x2e4bae8d 0x2e4054b3 0xb2e67 0xb420d 0x2ddd5ee5 0x2ddd5b1f 0x2ddd6283 0x2ddd6163 0x2ddd6045 0x2ddd5fc7 0x2ddd57a5 0x2e4859df 0x2e48597b 0x2e48414f 0x2e3eec27 0x2e3eea0b 0x330cd283 0x30c92049 0xa1d6f 0x38d52ab7)
libc++abi.dylib: terminating with uncaught exception of type NSException

And here's how I'm starting/stopping the recording:

- (void)startRecording
{
    CheckError(AudioSessionSetActive(true), "Couldn't set audio session active\n");

    if (![self unitIsRunning]) {
        CheckError(AudioOutputUnitStart([self audioUnit]), "Couldn't start unit");
        [self setUnitIsRunning:YES];
    }


}

- (void)stopRecording
{
    if ([self unitIsRunning]) {
        CheckError(AudioOutputUnitStop([self audioUnit]), "Couldn't stop unit");
        [self setUnitIsRunning:NO];
    }

    // if I comment out the following line, and keep the audio session running
    // throughout the lifetime of the app, everything works fine
    CheckError(AudioSessionSetActive(false), "Couldn't set audio session inactive\n");
}

Obviously I've left out the audio render callback that occurs when the unit is running as well as the configuration of the audio session/audio unit that is done on initial startup for brevity, but again, that's very close to what's done in aurioTouch2. The error seems to be due to setting the audio session inactive, but I can't seem to figure out why.

I've had a hard time finding whether or not Apple recommends having the same audio session running even if the app isn't actually doing anything. Should the session remain for the lifetime of the application or on a need to use basis? Any thoughts?