Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add local time sync ref (for AbletonLink etc.) selection to SoundManager #14164

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add local time sync ref (for AbletonLink etc.) selection to SoundManager #14164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2484,6 +2484,7 @@ add_executable(
src/test/signalpathtest.cpp
src/test/skincontext_test.cpp
src/test/softtakeover_test.cpp
src/test/soundmanager_test.cpp
src/test/soundproxy_test.cpp
src/test/soundsourceproviderregistrytest.cpp
src/test/sqliteliketest.cpp
Expand Down
163 changes: 115 additions & 48 deletions src/soundio/soundmanager.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,40 @@ void SoundManager::queryDevicesMixxx() {
m_devices.append(currentDevice);
}

SoundDevicePointer SoundManager::selectLocalTimeSyncRef(
const QHash<SoundDevicePointer, QList<AudioOutput>>& deviceOutputs,
const QList<SoundDevicePointer>& devices) {
const std::array<AudioPathType, 5> priorityOrder = {
AudioPathType::Main,
AudioPathType::Deck,
AudioPathType::Bus,
AudioPathType::Headphones,
AudioPathType::Booth};

SoundDevicePointer pNewLocalTimeSyncRef = nullptr;
for (const auto& pDevice : std::as_const(devices)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devices is already const. Did clazy complain?

if (pDevice->getDeviceId().name == kNetworkDeviceInternalName) {
continue;
}
QList<AudioOutput> outputs = deviceOutputs[pDevice];
for (const auto& type : priorityOrder) {
auto it = std::find_if(outputs.begin(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use a const iterator here.

outputs.end(),
[type](const AudioOutput& out) {
return out.getType() == type;
});
if (it != outputs.end()) {
pNewLocalTimeSyncRef = pDevice;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we return here already?

break;
}
}
if (pNewLocalTimeSyncRef) {
break;
}
}
return pNewLocalTimeSyncRef;
}

SoundDeviceStatus SoundManager::setupDevices() {
// NOTE(rryan): Big warning: This function is concurrent with calls to
// pushBuffer and onDeviceOutputCallback until closeDevices() below.
Expand Down Expand Up @@ -351,10 +385,6 @@ SoundDeviceStatus SoundManager::setupDevices() {
// callback from the logic in SoundDevicePortAudio. They should communicate
// via message passing over a request/response FIFO.

// Instead of clearing m_pClkRefDevice and then assigning it directly,
// compute the new one then atomically hand off below.
SoundDevicePointer pNewMainClockRef;

m_audioLatencyOverloadCount.set(0);

// load with all configured devices.
Expand All @@ -363,7 +393,77 @@ SoundDeviceStatus SoundManager::setupDevices() {

// pair is isInput, isOutput
QVector<DeviceMode> toOpen;
bool haveOutput = false;

// Get all outputs for each device
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
for (const auto& pDevice : std::as_const(m_devices)) {
deviceOutputs[pDevice] = m_config.getOutputs().values(pDevice->getDeviceId());
// Statically connect the Network Device to the Sidechain
if (pDevice->getDeviceId().name == kNetworkDeviceInternalName) {
AudioOutput out(AudioPathType::RecordBroadcast,
0,
mixxx::audio::ChannelCount::stereo(),
0);
deviceOutputs[pDevice].append(out);
}
}

// Select pNewLocalTimeSyncRef
// The local time sync reference is the device that is used for the
// synchronization of processes outside the audio processing engine
// to the DAC timing of the local sounddevice the DJ hears.
// This sync reference shall be used for:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is missleading. Because for my undertsanding we only have one clock everything is synced too.
pNewLocalTimeSyncRef is only a temporary, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is about selecting the sounddevice that is to be used as reference for external syncronisation. This is the sounddevice with the DAC that generates the sound the DJ hears.
Time syncronization has nothing to do with the clock and this comment section does not contain this term therefore.
Output Latency is specific for each sounddevice and contains various buffers inside and outside of Mixxx. All DAC channels of the same sound-device have the same output-latency, but different sound-devices may have different output-latency. Therefore we need to determine the sound-device that is the reference, before we can gather the correct output-latency.
Logically, the DJ always hears the sound via one of the local PortAudio devices and never via the stream broadcasted over the network. Accordingly, a soundDeviceNetwork can never be useful as a reference for synchronization.

// 1.) VSync of the waveforms
// 2.) Sync of external audio device (e.g. drum machine) with feedback
// to an auxiliary input of the mixer in a external mixing setup
// 3.) Sync of lighting (DMX) or video (VJ)
SoundDevicePointer pNewLocalTimeSyncRef = selectLocalTimeSyncRef(deviceOutputs, m_devices);

// Select pNewMainClockRef
// The main clock reference is the device that is used for the
// audio processing in the engine. It can be either a local
// PortAudio device or the Network clock in case of broadcasting.
// There are four use cases:
// 1.) No broadcasting->Always Soundcard Clock
// 2.) JACK API used->Always Soundcard Clock
// 3.) Broadcasting of internal mixed Main signal->Always Network Clock
// 4.) Broadcasting of Record/Broadcast input SoundDevice->Always Soundcard Clock
SoundDevicePointer pNewMainClockRef = nullptr;
if (m_config.getForceNetworkClock() && !jackApiUsed()) {
for (const auto& pDevice : std::as_const(m_devices)) {
if (pDevice->getDeviceId().name == kNetworkDeviceInternalName) {
pNewMainClockRef = pDevice;
break;
}
}
} else {
pNewMainClockRef = pNewLocalTimeSyncRef;
}

// Fallback to keep waveforms running if no local SoundDevice is configured
// If pNewLocalTimeSyncRef or pNewMainClockRef is still nullptr,
// set it to the first network clock device.
if (!pNewLocalTimeSyncRef || !pNewMainClockRef) {
for (const auto& pDevice : std::as_const(m_devices)) {
if (pDevice->getDeviceId().name == kNetworkDeviceInternalName) {
if (!pNewLocalTimeSyncRef) {
qWarning() << "No local sound device configured, local "
"sync reference not set! Using"
<< pDevice->getDisplayName();
pNewLocalTimeSyncRef = pDevice;
}
if (!pNewMainClockRef) {
qWarning() << "Output sound device clock reference not set! Using"
<< pDevice->getDisplayName();
pNewMainClockRef = pDevice;
}
if (pNewLocalTimeSyncRef && pNewMainClockRef) {
break;
}
}
}
}

// loop over all available devices
for (const auto& pDevice : std::as_const(m_devices)) {
DeviceMode mode = {pDevice, false, false};
Expand Down Expand Up @@ -393,26 +493,10 @@ SoundDeviceStatus SoundManager::setupDevices() {
m_pEngineMixer->onInputConnected(in);
}
}
QList<AudioOutput> outputs =
m_config.getOutputs().values(pDevice->getDeviceId());

// Statically connect the Network Device to the Sidechain
if (pDevice->getDeviceId().name == kNetworkDeviceInternalName) {
AudioOutput out(AudioPathType::RecordBroadcast,
0,
mixxx::audio::ChannelCount::stereo(),
0);
outputs.append(out);
if (m_config.getForceNetworkClock() && !jackApiUsed()) {
pNewMainClockRef = pDevice;
}
}

for (const auto& out : std::as_const(outputs)) {
// Iterate over all outputs for the current device
for (const auto& out : std::as_const(deviceOutputs[pDevice])) {
mode.isOutput = true;
if (pDevice->getDeviceId().name != kNetworkDeviceInternalName) {
haveOutput = true;
}
// following keeps us from asking for a channel buffer EngineMixer
// doesn't have -- bkgood
const CSAMPLE* pBuffer = m_registeredSources.value(out)->buffer(out).data();
Expand All @@ -427,16 +511,6 @@ SoundDeviceStatus SoundManager::setupDevices() {
goto closeAndError;
}

if (!m_config.getForceNetworkClock() || jackApiUsed()) {
if (out.getType() == AudioPathType::Main) {
pNewMainClockRef = pDevice;
} else if ((out.getType() == AudioPathType::Deck ||
out.getType() == AudioPathType::Bus) &&
!pNewMainClockRef) {
pNewMainClockRef = pDevice;
}
}

// Check if any AudioSource is registered for this AudioOutput and
// call the onOutputConnected method.
for (auto it = m_registeredSources.find(out);
Expand All @@ -453,19 +527,10 @@ SoundDeviceStatus SoundManager::setupDevices() {
}
}

for (const auto& mode: toOpen) {
for (const auto& mode : toOpen) {
SoundDevicePointer pDevice = mode.pDevice;
m_pErrorDevice = pDevice;

// If we have not yet set a clock source then we use the first
// output pDevice
if (pNewMainClockRef.isNull() &&
(!haveOutput || mode.isOutput)) {
pNewMainClockRef = pDevice;
qWarning() << "Output sound device clock reference not set! Using"
<< pDevice->getDisplayName();
}

int syncBuffers = m_config.getSyncBuffers();
// If we are in safe mode and using experimental polling support, use
// the default of 2 sync buffers instead.
Expand All @@ -485,11 +550,14 @@ SoundDeviceStatus SoundManager::setupDevices() {
}
}

if (pNewMainClockRef) {
VERIFY_OR_DEBUG_ASSERT(pNewMainClockRef) {
// This should never happen, because the user can't delete the last
// broadcast device in the preferences.
qWarning() << "No output devices opened, no clock reference device set";
}
else {
qDebug() << "Using" << pNewMainClockRef->getDisplayName()
<< "as output sound device clock reference";
} else {
qWarning() << "No output devices opened, no clock reference device set";
}

qDebug() << outputDevicesOpened << "output sound devices opened";
Expand All @@ -499,8 +567,7 @@ SoundDeviceStatus SoundManager::setupDevices() {
}

m_pControlObjectSoundStatusCO->set(
outputDevicesOpened > 0 ?
SOUNDMANAGER_CONNECTED : SOUNDMANAGER_DISCONNECTED);
outputDevicesOpened > 0 ? SOUNDMANAGER_CONNECTED : SOUNDMANAGER_DISCONNECTED);

// returns OK if we were able to open all the devices the user wanted
if (devicesNotFound.isEmpty()) {
Expand Down
4 changes: 4 additions & 0 deletions src/soundio/soundmanager.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ class SoundManager : public QObject {
void queryDevicesPortaudio();
void queryDevicesMixxx();

static SoundDevicePointer selectLocalTimeSyncRef(
const QHash<SoundDevicePointer, QList<AudioOutput>>& deviceOutputs,
const QList<SoundDevicePointer>& devices);

// Opens all the devices chosen by the user in the preferences dialog, and
// establishes the proper connections between them and the mixing engine.
SoundDeviceStatus setupDevices();
Expand Down
151 changes: 151 additions & 0 deletions src/test/soundmanager_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
#include "soundio/soundmanager.h"

#include <gtest/gtest.h>

#include <QHash>
#include <QList>

#include "soundio/sounddevice.h"

class MockSoundDevice : public SoundDevice {
public:
MockSoundDevice(const QString& name)
: SoundDevice(nullptr, nullptr) {
m_deviceId.name = name;
}

SoundDeviceStatus open(bool, int) override {
return SoundDeviceStatus::Ok;
}

bool isOpen() const override {
return true;
}

SoundDeviceStatus close() override {
return SoundDeviceStatus::Ok;
}

void readProcess(SINT) override {
}

void writeProcess(SINT) override {
}

QString getError() const override {
return QString();
}

mixxx::audio::SampleRate getDefaultSampleRate() const override {
return mixxx::audio::SampleRate(44100);
}
};

TEST(SoundManagerTest, SelectLocalTimeSyncRefNoSoundDevice) {
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
QList<SoundDevicePointer> devices;

// No sound devices defined
// deviceOutputs and devices are empty

// Test case: Select local time sync reference with no sound devices
SoundDevicePointer result = SoundManager::selectLocalTimeSyncRef(deviceOutputs, devices);
EXPECT_EQ(result, nullptr);
}

TEST(SoundManagerTest, SelectLocalTimeSyncRefNoDeviceOutput) {
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
QList<SoundDevicePointer> devices;

auto portAudioDevice1 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice1"));
auto portAudioDevice2 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice2"));

// No device outputs defined
deviceOutputs[portAudioDevice1] = {};
deviceOutputs[portAudioDevice2] = {};

devices.append(portAudioDevice1);
devices.append(portAudioDevice2);

// Test case: Select local time sync reference with no device outputs
SoundDevicePointer result = SoundManager::selectLocalTimeSyncRef(deviceOutputs, devices);
EXPECT_EQ(result, nullptr);
}

TEST(SoundManagerTest, SelectLocalTimeSyncRefOneDevice) {
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
QList<SoundDevicePointer> devices;

auto portAudioDevice1 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice1"));
auto portAudioDevice2 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice2"));

deviceOutputs[portAudioDevice1] = {
AudioOutput(AudioPathType::Main,
0,
mixxx::audio::ChannelCount::stereo(),
0),
AudioOutput(AudioPathType::Headphones,
0,
mixxx::audio::ChannelCount::stereo(),
0)};
deviceOutputs[portAudioDevice2] = {};

devices.append(portAudioDevice1);
devices.append(portAudioDevice2);

// Test case: Select local time sync reference
SoundDevicePointer result = SoundManager::selectLocalTimeSyncRef(deviceOutputs, devices);
EXPECT_EQ(result, portAudioDevice1);
}

TEST(SoundManagerTest, SelectLocalTimeSyncRefTwoDevices) {
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
QList<SoundDevicePointer> devices;

auto portAudioDevice1 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice1"));
auto portAudioDevice2 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice2"));

deviceOutputs[portAudioDevice1] = {AudioOutput(AudioPathType::Headphones,
0,
mixxx::audio::ChannelCount::stereo(),
0)};
deviceOutputs[portAudioDevice2] = {AudioOutput(
AudioPathType::Main, 0, mixxx::audio::ChannelCount::stereo(), 0)};

devices.append(portAudioDevice1);
devices.append(portAudioDevice2);

// Test case: Select local time sync reference
SoundDevicePointer result = SoundManager::selectLocalTimeSyncRef(deviceOutputs, devices);
EXPECT_EQ(result, portAudioDevice1);
}

TEST(SoundManagerTest, SelectLocalTimeSyncRefWithNetworkDevice) {
QHash<SoundDevicePointer, QList<AudioOutput>> deviceOutputs;
QList<SoundDevicePointer> devices;

auto portAudioDevice1 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice1"));
auto portAudioDevice2 = SoundDevicePointer(new MockSoundDevice("PortAudioDevice2"));
auto networkDevice = SoundDevicePointer(new MockSoundDevice(kNetworkDeviceInternalName));

deviceOutputs[portAudioDevice1] = {};
deviceOutputs[portAudioDevice2] = {
AudioOutput(AudioPathType::Main,
0,
mixxx::audio::ChannelCount::stereo(),
0),
AudioOutput(AudioPathType::Headphones,
0,
mixxx::audio::ChannelCount::stereo(),
0)};
deviceOutputs[networkDevice] = {AudioOutput(
AudioPathType::Main, 0, mixxx::audio::ChannelCount::stereo(), 0)};

devices.append(portAudioDevice1);
devices.append(networkDevice);
devices.append(portAudioDevice2);

// Test case: Select local time sync reference
SoundDevicePointer result = SoundManager::selectLocalTimeSyncRef(deviceOutputs, devices);
EXPECT_EQ(result->getDeviceId(), portAudioDevice2->getDeviceId());
}
Loading