Apple Core Audio
Core Audio is Apple's collection of audio services for formats, conversion, processing, device access, and timed playback.
Core Audio is a collection of Apple audio services, rather than one codec or one fixed output mode. It covers reading audio, describing formats, converting data, processing samples, and communicating with output devices.
The name therefore does not establish whether a player mixes audio, changes its sample rate, uses spatial processing or preserves PCM values. Those results depend on the services and configuration used by the application.
The responsibilities within the framework
| Part | Responsibility |
|---|---|
| Audio file and stream services | Interpret stored or incoming audio data |
| Audio Converter services | Convert supported representations and sample formats |
| Audio Units | Process and route signal blocks, including mixing and output |
| Audio Queue services | Manage queued playback or recording buffers |
| Audio hardware abstraction layer, HAL | Provide device-oriented access on macOS |
| AVAudioSession on supported Apple platforms | Express the app's audio use and preferences, and observe the selected route |
Higher-level playback facilities can build on these services. An Emby app using a platform player does not have to call every lower-level API itself.
The macOS device model and the iOS-family audio-session model also differ. A device access technique on macOS should not be assumed to be an equivalent selectable mode on a phone.
A sample frame can span several buffers
An AudioStreamBasicDescription, commonly ASBD, describes the stream's rate, format, channels, sample representation and packet/frame dimensions. Buffer organization then determines where the samples are stored.
Take 48 kHz stereo with 32-bit floating-point samples:
| Arrangement | Samples in memory | Storage for 480 sample frames |
|---|---|---|
| Interleaved | L₀, R₀, L₁, R₁… in one buffer | One 3,840-byte buffer |
| Noninterleaved | L₀, L₁… and R₀, R₁… in separate buffers | Two 1,920-byte buffers |
Both represent 10 ms of stereo audio. Interleaved frames use eight bytes; each noninterleaved channel plane uses four bytes per frame. The number of buffers has not doubled the duration.
A callback receiving 480 frames needs the appropriate amount of storage in each buffer. Treating a per-plane byte count as the total interleaved count can truncate data or overwrite memory. The format flags and buffer list supply the interpretation; a casual “float stereo” label is incomplete.
Compressed packets have another relationship to frames. A codec packet can decode to many PCM sample frames. Packet count is not a substitute for the sample clock.
A preference is not the actual device format
An application can request a preferred sample rate or I/O buffer duration through its audio session. Those values express a preference. The actual route and other active audio requirements can produce another result.
After activation or a route change, the application queries the actual sample rate and buffer duration. If a source is 44.1 kHz and the active output route is 48 kHz, some stage must supply suitable 48 kHz samples. A file's original rate remains useful metadata but does not prove the final route's rate.
A headphone disconnection, Bluetooth selection or other route event can also change capabilities and timing. The player needs to adapt its stream rather than continuing to interpret buffers using an obsolete configuration.
Rendering on the output deadline
The output consumes frames at the established clock rate. Callbacks or queues keep it supplied. Data retrieval and expensive preparation need enough advance time so that the render work can meet its deadline.
For movie playback, the system's audio timing helps the player align video presentation with the output. The duration of one callback is a processing interval, while the complete route can include additional queued and device delay.