Feature Request: Configurable iOS/Android Audio Session Options

[delphinius81]

Summary

PlatformAudio currently hard-codes the iOS audio session options in LiveKit_ConfigureAudioSessionForVoIP(). Apps that play background audio (music, ambient sound) have no way to opt into ducking or mixing — the session silences all other audio as soon as LiveKit starts.

Proposed API

Add an AudioSessionOptions parameter to the PlatformAudio constructor (or a static configure method) that lets callers supply additional AVAudioSessionCategoryOptions flags before the WebRTC ADM initializes:

// C# API change (example)
public PlatformAudio(AudioSessionOptions options = AudioSessionOptions.Default)

public enum AudioSessionOptions
{
Default = 0,
DuckOthers = 1, // Lower volume of other audio sessions (background audio continues quietly)
MixWithOthers = 2 // Full-volume mix with other audio sessions
}

On the native side, LiveKit_ConfigureAudioSessionForVoIP (or a new LiveKit_ConfigureAudioSessionForVoIPWithOptions(int options)) would pass through the caller's flags:

// Objective-C (LiveKitAudioSession.mm)
void LiveKit_ConfigureAudioSessionForVoIPWithOptions(int additionalOptions) {
AVAudioSessionCategoryOptions opts =
AVAudioSessionCategoryOptionDefaultToSpeaker |
AVAudioSessionCategoryOptionAllowBluetooth |
AVAudioSessionCategoryOptionAllowBluetoothA2DP;

  if (additionalOptions & kDuckOthers)   opts |= AVAudioSessionCategoryOptionDuckOthers;
  if (additionalOptions & kMixWithOthers) opts |= AVAudioSessionCategoryOptionMixWithOthers;

  [session setCategory:AVAudioSessionCategoryPlayAndRecord
                 mode:AVAudioSessionModeVoiceChat
              options:opts                                                                                                              
                error:&error];
  ...

}

Use case

Apps that play background audio alongside a LiveKit conversation need DuckOthers so the background track continues at reduced volume instead of being silenced entirely. The hard-coded session config makes this impossible without forking the package.

Android equivalent

A similar opt-in for AudioManager.setStreamVolume/AudioFocusRequest ducking behavior would be valuable for Android parity.

[MaxHeimbrock]

Thanks for reporting, I am investigating the issue both on Android and iOS.

[MaxHeimbrock]

In your case, is the other audio also coming from the same app? For example, from an audio source within Unity?
Or is this case for example if Spotify or another app is playing audio in the background?

[delphinius81]

It's potentially both, but for my app main concern is already playing audio from my own app that's routed through Unity's audio system. As soon as the livekit starts, it seems to be interrupting all audio that Unity is playing (most likely due to the OS level audio session values changing, which cause an interrupt signal that Unity is processing). What is interesting is that if I mute and then unmute my LocalAudioTrack instance, Unity audio starts back up again. But I haven't been able to get this to work without that manual step.

I'm currently making builds via Unity 2022.3.62. I haven't tried this in Unity 6 yet to see if there are unity audio engine changes that handle this kind of thing better.

[MaxHeimbrock]

Thanks for clarifying, I am currently working on an agent sample that I build with Unity 6 and tried on MacOS, Android and iOS, which also has Unity audio playing in the background.

For me the situation looks as follows:

• On MacOS and iOS everything looks good
• On Android we need additional native calls and the resulting setup is not perfect, the WebRTC Platform Audio is then treated as call audio and the Unity audio as media audio. You can not mute the call audio to zero

I now had a look as well at the Meet sample which currently targets 2022.3.62f1. If I add a Unity audio source playing music, target iOS and build, I can hear the music and the call.

Can you tell me your exact setup and also give the Meet sample a shot?

[delphinius81]

Do you start your unity audio before or after the livekit connection? I'll take a look at the meet sample to see if we are doing something out of sync here.

[MaxHeimbrock]

I have music play on startup and looping. These are some cases I observed on iOS and Unity 2022.62f1 with the Meet sample:

• When I join a room muted and without any remote participant, the music output is the same

• When I unmute, the music now plays out of the earpiece instead of loudspeaker

• When I mute again, the music is gone

• When I join a room muted and there is a remote track playing, the audio is mixed. I think the louder the remote plays, the more quiet the Unity gets. Easy proof is an agent speaking.

• When I dispose Platform Audio, Unity audio is broken.

[delphinius81]

On iOS, if I insert some delays between the various steps and force the mic to mute / unmute as part of the startup process, I am able to keep the background audio going.

I didn't see any real differences between the meet example and what I am doing as far as startup is concerned.

But there's definitely an issue with disposing platform audio. Unity audio stays broken. At one point when testing though, I think I received a phone call. I think after dismissing the phone call and returning to Unity, I was able to get Unity audio again.