# OSD audio support

## Introduction

The audio support in Mame tries to allow the user to freely map between the emulated system audio outputs (called speakers) and the host system audio. A part of it is the OSD support, where a host-specific module ensures the interface between Mame and the host. This is the documentation for that module.

Note: this is currently output-only, but input should follow.

## Capabilities

The OSD interface is designed to allow three levels of support, depending on what the API allows and the amount of effort to expend. Those are:

- Level 1: One or more audio targets, only one stream allowed per target (aka exclusive mode)

- Level 2: One or more audio targets, multiple streams per target

- Level 3: One or more audio targets, multiple streams per target, user-visible per-stream-channel volume control

In any case we support having the user use an external interface to change the target of a stream and, in level 3, change the volumes. By support we mean storing the information in the per-game configuration and keeping the internal UI in sync.

## Terminology

For this module, we use the terms:

- node: some object we can send audio to. Can be physical, like speakers, or virtual, like an effect system. It should have a unique, user-presentable name for the UI.

- port: a channel of a node, has a name (non-unique, like "front left") and a 3D position

- stream: a connection to a node with allows to send audio to it

## Reference documentation

### Adding a module

Adding a module is done by adding a cpp file to src/osd/modules/sound which follows this structure,

    // License/copyright
    #include "sound_module.h"
    #include "modules/osdmodules.h"

    #ifdef MODULE_SUPPORT_KEY

    #include "modules/lib/osdobj_common.h"

    // [...]
    namespace osd {
    namespace {

    class sound_module_class : public osd_module, public sound_module
    {
       sound_module_class() : osd_module(OSD_SOUND_PROVIDER, "module_name"),
                              sound_module()
       // ...
    };

    }
    }
    #else
    namespace osd { namespace {
      MODULE_NOT_SUPPORTED(sound_module_class, OSD_SOUND_PROVIDER, "module_name")
    }}
    #endif

    MODULE_DEFINITION(SOUND_MODULE_KEY, osd::sound_module_class)

In that code, four names must be chosen:

- MODULE_SUPPORT_KEY some #define coming from the genie scripts to tell that this particular module can be compiled (like NO_USE_PIPEWIRE or SDLMAME_MACOSX)

- sound_module_class is the name of the class which makes up the module (like sound_coreaudio)

- module_name is the name to be used in -sound <xxx> to select that particular module (like coreaudio)

- SOUND_MODULE_KEY is a symbol that represents the module internally (like SOUND_COREAUDIO)

The file path needs to be added to scripts/src/osd/modules.lua in osdmodulesbuild() and the module reference to src/osd/modules/lib/osdobj_common.cpp in osd_common_t::register_options with the line:

    REGISTER_MODULE(m_mod_man, SOUND_MODULE_KEY);

This should ensure that the module is reachable through -sound <xxx> on the appropriate hosts.

### Interface

The full interface is:

    virtual bool split_streams_per_source() const override;
    virtual bool external_per_channel_volume() const override;

    virtual int init(osd_interface &osd, osd_options const &options) override;
    virtual void exit() override;

    virtual uint32_t get_generation() override;
    virtual osd::audio_info get_information() override;
    virtual uint32_t stream_sink_open(uint32_t node, std::string name, uint32_t rate) override;
    virtual uint32_t stream_source_open(uint32_t node, std::string name, uint32_t rate) override;
    virtual void stream_set_volumes(uint32_t id, const std::vector<float> &db) override;
    virtual void stream_close(uint32_t id) override;
    virtual void stream_sink_update(uint32_t id, const int16_t *buffer, int samples_this_frame) override;
    virtual void stream_source_update(uint32_t id, int16_t *buffer, int samples_this_frame) override;

The class sound_module provides defaults for minimum capabilities: one stereo target and stream at default sample rate. To support that, only init, exit and stream_update need to be implemented. init is called at startup and exit when quitting and can do whatever they need to do. stream_sink_update will be called on a regular basis with a buffer of sample_this_frame * 2 * int16_t with the audio to play. From this point in the documentation we'll assume more than a single stereo channel is wanted.

### Capabilities

Two methods are used by the module to indicate the level of capability of the module:

- split_streams_per_source() should return true when having multiple streams for one target is expected (e.g. Level 2 or 3)

- external_per_channel_volume() should return true when the streams have per-channel volume control that can be externally controlled (e.g. Level 3)

### Hardware information and generations

The core runs on the assumption that the host hardware capabilities can change at any time (bluetooth devices coming and going, usb hot-plugging...) and that the module has some way to keep tabs on what is happening, possibly using multi-threading. To keep it lightweight-ish, we use the concept of a generation which is a 32-bit number that is incremented by the module every time something changes. The core checks the current generation value at least once every update (once per frame, usually) and if it changed asks for the new state and detects and handles the differences. generation should be "eventually stable", e.g. it eventually stops changing when the user stops changing things all the time. A systematic increment every frame would be a bad idea.

    virtual uint32_t get_generation() override;

That method returns the current generation number. It's called at a minimum once per update, which usually means per frame. It should be reasonably lightweight when nothing special happens.

This method must provide all the information about the current state of the host and the module. This state is:

- m_generation: The current generation number

- m_nodes: The vector available nodes (node_info)

  - m_name: The name of the node to be used in non-user-visible configurations (can be a uuid or equivalent)

  - m_display_name: The name of the node to be used in the user interfaces (should be readable)

  - m_id: The numeric ID of the node

  - m_rate: The minimum, maximum and preferred sample rate for the node

  - m_port_names: The vector of port names

  - m_port_positions: The vector of 3D position of the ports. Refer to src/emu/speaker.h for the "standard" positions

  - m_sinks: Number of sinks (inputs)

  - m_sources: Number of sources (outputs)

- m_default_sink: ID of the node that is the current "system default" for audio output, 0 if there's no such concept

- m_default_source: same for audio input (currently unused)

- m_streams: The vector of active streams (stream_info)

  - m_id: The numeric ID of the stream

  - m_node: The target node of the stream

  - m_volumes: empty if external_per_channel_volume is false, current volume value per-channel otherwise

IDs, for nodes and streams, are (independent) 32-bit unsigned non-zero values associated to respectively nodes and streams. IDs should not be reused. A node that goes away then comes back should get a new ID. A stream closing does not allow reuse of its ID.

If a node has both sources and sinks, the sources are monitors of the sinks, e.g. they're loopbacks. They should have the same count in such a case.

Nodes must be independent. It must be possible to open streams to two different nodes at the same time. Be careful of multi-api libraries that collide between apis. In addition, with monitoring streams (input on an output), it must be possible to open different streams for input and output. If it's not the case, do not publish the monitoring inputs.

When external control exists, a module should change the value of stream_info::m_node when the user changes it, and same for stream_info::m_volumes. Generation number should be incremented when this happens, so that the core knows to look for changes.

Volumes are floats in dB, where 0 means 100% and -96 means no sound. audio.h provides osd::db_to_linear and osd::linear_to_db if such a conversion is needed.

Positions have two special values: unknown() means the position of the speaker or microphone is unknown, but it should be used anyway. The used position will be centered. map_on_request_only() means the input or output should not be used on full mappings but only when explicitly requested with a channel mapping.

There is an inherent race condition with this system, because things can change at any point after returning for the method. The idea is that the information returned must be internally consistent (a stream should not point to a node ID that does not exist in the structure, same for default sink) and that any external change from that state should increment the generation number, but that's it. Through the generation system the core will eventually be in sync with reality.

### Input and output streams

    virtual uint32_t stream_sink_open(uint32_t node, std::string name, uint32_t rate) override;
    virtual uint32_t stream_source_open(uint32_t node, std::string name, uint32_t rate) override;
    virtual void stream_set_volumes(uint32_t id, const std::vector<float> &db) override;
    virtual void stream_close(uint32_t id) override;
    virtual void stream_sink_update(uint32_t id, const int16_t *buffer, int samples_this_frame) override;
    virtual void stream_source_update(uint32_t id, int16_t *buffer, int samples_this_frame) override;

Streams are the concept used to send or receive audio from/to the host audio system. A stream is first opened through stream_sink_open for speakers and stream_source_open for microphones and targets a specific node at a specific sample rate. It is given a name for use by the host sound services for user UI purposes (currently the game name if split_streams_per_source is false, the speaker_device/microphone_device tag if true). The returned ID must be a non-zero, never-used-before for streams value in case of success. Failures, like when the node went away between the get_information call and the open one, should be silent and return zero.

stream_set_volumes is used only when external_per_channel_volume is true and is used by the core to set the per-channel volume. The call should just be ignored if the stream ID does not exist (or is zero). Do not try to apply volumes in the module if the host API doesn't provide for it, let the core handle it.

stream_close closes a stream, The call should just be ignored if the stream ID does not exist (or is zero).

Opening a stream, closing a stream or changing the volume does not need to touch the generation number.

stream_sink_update is the method used to send data to the node through a given stream. It provides a buffer of samples_this_frame * node channel count channel-interleaved int16_t values. The lifetime of the data in the buffer or the buffer pointer itself is undefined after return from the method call. The call should just be ignored if the stream ID does not exist (or is zero).

stream_source_update is the equivalent to retrieve data from a node, writing to the buffer instead of reading from it. The constraints are identical.

When a stream goes away because the target node is lost it should just be removed from the information, and the core will pick up the node departure and close the stream.

Given the assumed raceness of the interface, all the methods should be tolerant of obsolete or zero IDs being used by the core, and that is why ID reuse must be avoided. Also the update methods and the open/close/volume ones may be called at the same time in different threads.

### Helper class abuffer

    class abuffer {
    public:
        abuffer(uint32_t channels);
        void get(int16_t *data, uint32_t samples);
        void push(const int16_t *data, uint32_t samples);
        uint32_t channels() const;
    };

The class abuffer is a helper provided by sound_module to buffer audio input or output. It automatically drops data when there is an overflow and duplicates the last sample on underflow. It must first be initialized with the number of channels, which can be retrieved with channels() if needed. push sends samples * channels 16-bit samples in the buffer. get retrieves samples * channels 16-bit samples from the buffer, on a FIFO basis.

It is not protected against multithreading, but uses no class variables. So just don't read and write from one specific abuffer instance at the same time. The system sound interface mandated locking should be enough to ensure that.