Unable to load audio from video asset with AVKit

34 views Asked by At

I have problem with extracting/reading audio from AVCaptureSession.
Step by step:

  • I ended session AVCaptureSession and send my outputFileURL to the next view.
  • On next view I started extracting audio from video:
Task(priority: .background) {
    do {
        try await extractAudioFromVideo()
        withAnimation(.spring) {
            print("Audio length: \(lengthInSeconds), videoDuration: \(avplayer.currentItem?.asset.duration ?? .zero)")
            isExportEnding = true
        }
    } catch {
        print("Error extracting \(error.localizedDescription)")
    }
}

private func extractAudioFromVideo() async throws {
    let asset = AVAsset(url: url)
    
    if FileManager.default.fileExists(atPath: Constants.sampleOutput!.path) {
        try? FileManager.default.removeItem(atPath: Constants.sampleOutput!.path)
    }
    
    guard let exportSession = AVAssetExportSession(asset: asset,
                                                   presetName: AVAssetExportPresetAppleM4A)
    else {
        print("Unable to create AVAssetExportSession.")
        return
    }
    
    exportSession.outputFileType = .m4a
    exportSession.outputURL = Constants.sampleOutput
    
    await exportSession.export()
    switch exportSession.status {
    case .completed:
        print("Audio extraction successful!")
        makeEngineConnections()
    case .failed:
        print("Audio extraction failed: \(exportSession.error ?? .none)")
        if let error = exportSession.error {
            throw error
        }
    default:
        print("Status: \(exportSession.status)")
    }
}

And receive this error:

Audio extraction failed: Optional(Error Domain=AVFoundationErrorDomain Code=-11838 "Operation Stopped" UserInfo={NSLocalizedFailureReason=The operation is not supported for this media., NSLocalizedDescription=Operation Stopped, NSUnderlyingError=0x28119c0f0 {Error Domain=NSOSStatusErrorDomain Code=-16976 "(null)"}}).

Then I tried NextLevelSessionExporter:

private func extractAudioFromVideo() async throws {
    let asset = AVAsset(url: url)
    let exporter = NextLevelSessionExporter(withAsset: asset)
    exporter.outputFileType = .m4a
    
    if FileManager.default.fileExists(atPath: Constants.sampleOutput!.path) {
        try? FileManager.default.removeItem(atPath: Constants.sampleOutput!.path)
    }
    
    exporter.outputURL = Constants.sampleOutput
    
    let compressionDict: [String: Any] = [
        AVVideoAverageBitRateKey: NSNumber(integerLiteral: 6000000),
        AVVideoProfileLevelKey: AVVideoProfileLevelH264HighAutoLevel as String,
    ]
    exporter.videoOutputConfiguration = [
        AVVideoCodecKey: AVVideoCodecType.h264,
        AVVideoWidthKey: NSNumber(integerLiteral: 1280),
        AVVideoHeightKey: NSNumber(integerLiteral: 720),
        AVVideoScalingModeKey: AVVideoScalingModeResizeAspectFill,
        AVVideoCompressionPropertiesKey: compressionDict
    ]
    
    exporter.audioOutputConfiguration = [
        AVFormatIDKey: kAudioFormatMPEG4AAC,
        AVEncoderBitRateKey: NSNumber(integerLiteral: 128000),
        AVNumberOfChannelsKey: NSNumber(integerLiteral: 2),
        AVSampleRateKey: NSNumber(value: Float(44100)),
        AVEncoderAudioQualityKey: AVAudioQuality.min.rawValue
    ]
    
    exporter.export { _, _, _ in
        
    } progressHandler: { progress in
        print("Export progress \(progress)")
    } completionHandler: { result in
        switch result {
            case .success(let status):
                switch status {
                    case .completed:
                        print("NextLevelSessionExporter, export completed, \(exporter.outputURL?.description ?? "")")
                        print("File size \(fileSize(fromPath: exporter.outputURL?.path ?? "nil") ?? "")")
                        print("Audio extraction successful!")
                        makeEngineConnections()
                    default:
                        print("Status \(status)")
                        break
                }
            case .failure(let error):
                print("Audio extraction failed: \(error.localizedDescription)")
        }
    }

I am not confident that I provide the right settings. But my extracting was successful, despite this when I try to open:

try AVAudioFile(forReading: Constants.sampleOutput!)

It throws an error:

2023-12-21 23:21:58.139672+0300 AVideo[45226:15581700] OpenFromDataSource failed

2023-12-21 23:21:58.140417+0300 AVideo[45226:15581700] Open failed

2023-12-21 23:21:58.141723+0300 AVideo[45226:15581700] [default] ExtAudioFile.cpp:193 about to throw 'dta?': open audio file

2023-12-21 23:21:58.143190+0300 AVideo[45226:15581700] [avae] AVAEInternal.h:109 [AVAudioFile.mm:134:AVAudioFileImpl: (ExtAudioFileOpenURL((CFURLRef)fileURL, &_extAudioFile)): error 1685348671 Error reading source file: The operation couldn’t be completed. (com.apple.coreaudio.avfaudio error 1685348671.)

NOTE THERE

It occurs only with AVCaptureSession and when video duration is more than ~10sec. And all code provided above works perfectly with assets from gallery with any duration ...

0

There are 0 answers