Starting up the media2 MediaSessionService causing some random crashes (using MediaController and ExoPlayer)

Question

I've got some strange crashes on Firebase (and I was unable to reproduce them), that occur at the launch of my MediaSessionService :

Fatal Exception: android.app.ForegroundServiceDidNotStartInTimeException
Context.startForegroundService() did not then call Service.startForeground(): ServiceRecord{1667291 u0 MyAudioService}

I only start my service using a media controller, by passing it the media session token :

val sessionToken = MediaSessionManager.getInstance(context).sessionServiceTokens.first { sessionToken -> 
    sessionToken.packageName == context.packageName 
}

val mediaController = MediaController.Builder(context)
            .setSessionToken(sessionToken)
            .setControllerCallback(threadPoolExecutor, mediaControllerCallback)
            .build()

And for the notification, I use the PlayerNotificationManager from exoplayer, saving the notification sent in the callback onNotificationPosted(), only if it's ongoing (otherwise clearing it). I also clear it in the onNotificationCleared() callback. And then passing this saved notification to the onUpdateNotification(session: MediaSession) method of my MediaSessionService.

I've also tried to always send a null MediaNotification to the method onUpdateNotification(session: MediaSession), and then manually handle the foreground state of the service inside the callback of the PlayerNotificationManager, but that didn't resolve this issue either.

The occurence is relatively low (around 0.5% of our users), but still, I've been wrapping my head around this issue for a while without figuring out a fix, so if you got any ideas it will be awesome !