Add stem controls #13086

acolombier · 2024-04-10T20:44:41Z

Depends on #13106

This adds basic COs and UI for stem volume control. Manuel PR is here

LateNight:

Kooha-2024-04-16-21-21-09-10MB.mp4

QML:

Kooha-2024-04-11-12-39-08.mp4

JoergAtGithub · 2024-04-10T22:17:19Z

Looks really cool!

I think we need a table with all planed stems controls and all existing stems hardware to ensure, that we can cover all

Here you see the use of the Traktor S8 https://www.youtube.com/watch?v=MKJZO_0zQcM&t=1s
- For the S8 a filter per stem and a filter enable control is additional needed
Here the Rane Four https://www.youtube.com/watch?v=Pf3GQdkWByk&t=1060s
- Has small text displays, that show the name of the stem and the effect-chain assigne
  d to the stem
- Has stem-split button which connects one deck with multiple mixer channels

acolombier · 2024-04-11T00:54:51Z

I think we need a table with all planed stems controls and all existing stems hardware to ensure, that we can cover all

One of the reasons why I added the stem feature flag is so we can iterate on features and controls, definitely worth having them layed out for reference tho! Shall we create an epic like we did for S4 screens? Could you help with that since I believe it requires special permission?

Has small text displays, that show the name of the stem and the effect-chain assigned to the stem

That's the same with S4, which is part if the reason why I wanted to have the QML proxy in place. As it stands, you can already get the label for each stem.

Has stem-split button which connects one deck with multiple mixer channels

I'll have to look how that works and see if that something we can support.

daschuer · 2024-04-11T07:06:26Z

I wonder where unity is on the stem knobs. My original expectation is to put it on center like EQs.
This allows to accentuate a stem, compared to others and allows switching between EQ and Stem knobs without changing scale.

Do we want to keep this switching in the mapping domain, or do we want to add four mixer meta knob that can be mapped and will switch between EQ and Stem.

By the way we had a similar discussion with the fourth EQ band, gain and filter knob.

acolombier · 2024-04-11T10:36:17Z

Has stem-split button which connects one deck with multiple mixer channels

I'll have to look how that works and see if that something we can support.

I just watched the review - that piece of kit looks very good btw - the split button seems to perform a deck clone, and just mute the vocal, then mute everything else on the newly clone deck. IIRC, the NI stem standard suggests that the drum should always be first and the vocal always be last, so I believe this would be pretty trivial to do! We might need new CO to target deck when cloning tho (I believe at the moment, it pick a candidate out the last focus deck)

Do we want to keep this switching in the mapping domain, or do we want to add four mixer meta knob that can be mapped and will switch between EQ and Stem.

Perhaps I can add a setting in the "Interface" section? Something like "Display stem volume control instead of EQ for stem deck"? This way, user without controllers or stem support on them can still play with it? Otherwise, I think by default, we should keep stem control in the mapping domain, exception perhaps of the waveform, but that we come in a next chapter, where I start looking at analyser

Edit: I fixed the offset change, leading to un-centered crossfader and added a little animation

Kooha-2024-04-11-12-39-08.mp4

vantaboard · 2024-04-11T13:34:11Z

Been thinking about this implementation a lot lately. What do we think about this design?

Basically, you would select either the Main stem which would be all of the stems together, or you would pick one of them individually or group them together by holding down shift or something. Then, the EQ would apply to only the selected stems. Same with scratching, you could have a stem-sync mode where the stems are linked together, or you could scratch a stem independently, like if you wanted to scratch Vox. You could also put a way to re-sync the stems back to the main track which acts like the leader.

acolombier · 2024-04-11T13:44:23Z

That's a good design, but perhaps this should come a second iteration, when the waveform also supports multiple stem layer? I can imagine the selection/hover of the stem button will "highlight" the stem signal on the waveform, which may be very handy to identify section of track with a specific feature on, say no vocals or instrumental solo

EQ would apply to only the selected stems.

Not sure this is something we will be able to do due to performance impact. EQ will likely remain applicable to the deck as whole. This means you can still spare a second deck for the voice and adjust the EQ there

Same with scratching

Scratching requires transport/playback control which would go against the logic that was implemented here, so definitely won't be possible. Here again tho, you can clone the deck and scratch just a stem (or set of them) on the individual deck, similar to the demo @JoergAtGithub shared above on Serato (?)

vantaboard · 2024-04-11T13:56:05Z

That's a good design, but perhaps this should come a second iteration, when the waveform also supports multiple stem layer? I can imagine the selection/hover of the stem button will "highlight" the stem signal on the waveform, which may be very handy to identify section of track with a specific feature on, say no vocals or instrumental solo

Yeah, I agree. Definitely giving the ability to visualize the layered stems would be great, especially if we could have the waveform color in alignment with the stem color.

Scratching requires transport/playback control which would go against the logic that was implemented here, so definitely won't be possible. Here again tho, you can clone the deck and scratch just a stem (or set of them) on the individual deck, similar to the demo @JoergAtGithub shared above on Serato (?)

Same with scratching and EQ, I agree. I guess I just find cloning a bit cumbersome at the moment. Perhaps in another iteration we can cue stems for cloning, so that a clone from deck A to deck B with a cue on vocals would mute the vocals on deck A and solo them onto B with seamless playback. This would be ideal because there are some transitions where I want to continue some portion of the music into a new track, but cloning feels clunky at the moment.

vantaboard · 2024-04-11T13:58:43Z

I think I have a tendency to want to avoid creating a similar design to what @JoergAtGithub posted https://youtu.be/Pf3GQdkWByk?t=1555 because the doubling up can be really annoying and it isn't optional even with cloning in the current state.

acolombier · 2024-04-11T14:05:33Z

Yeah, I agree. Definitely giving the ability to visualize the layered stems would be great, especially if we could have the waveform color in alignment with the stem color.

That will come in future PR tho! I've been trying to keep PR shot to help ease and speed up the review.

I guess I just find cloning a bit cumbersome at the moment. Perhaps in another iteration we can cue stems for cloning, so that a clone from deck A to deck B with a cue on vocals would mute the vocals on deck A and solo them onto B with seamless playback.

Yes, that's the plan, very similar to the "split stem" feature from that demo. This way you easily "dive" in the track stem and have complete freedom on your mixing, without convoluting the deck option and control

the doubling up can be really annoying and it isn't optional even with cloning in the current state.

Agreed. When we would look into this, I think we will make sure that no doubling up gets hearable - decks get cloned but stems are exclusive to avoid that gain issue (say A plays 0 1 2, B plays 3).

vantaboard · 2024-04-11T14:06:45Z

EQ would apply to only the selected stems.

Not sure this is something we will be able to do due to performance impact. EQ will likely remain applicable to the deck as whole. This means you can still spare a second deck for the voice and adjust the EQ there

Is EQ for the whole deck still accessible somewhere in your design? I'm wondering if it even makes sense to be able to control volume for each stem in that case. Maybe my design would be mute/unmute for the different stems rather than selection.

vantaboard · 2024-04-12T00:52:56Z

Is the skin you're using available anywhere? I have built the branch, but I don't have any options to load a stem.

JoergAtGithub · 2024-04-12T05:46:48Z

What the Rane Four (and other Serato stem controllers) do is bassically a mapping between stem channels playing in any deck and mixer channels. Similar, to what we do with effect units. But exposing this in the UI similar to the FX1,FX2 buttons might be overwhelming.
Another approach would be to visually merge 2 decks if the user presses the Stem Split button on the controller. If there is only one deck, there can't be a conflict of two decks outputs connected to the same mixer channel.

daschuer · 2024-04-12T06:49:07Z

Does the Rane Four use transport controls of deck one for deck three in split mode? That sounds kind of reasonable if you want to scratch the whole Track. Maybe we can consider two split modes like coupled and Independent. That is almost like our sync scratch mode we have abandoned for Mixxx 2.4

acolombier · 2024-04-12T16:39:08Z

Is the skin you're using available anywhere? I have built the branch, but I don't have any options to load a stem.

This is the new UI which isn't yet production ready and is lacking some basic features. You can see it by building with QT6 and QML, then run mixxx with --qml

JoergAtGithub · 2024-04-14T18:13:41Z

Does the Rane Four use transport controls of deck one for deck three in split mode?

In Sertao DJ this is not the case, in the Rane Four video, I posted already above, you can see scratching of the deck with the vocal stem, while the deck with the instrumental stems is playing at normal speed: https://www.youtube.com/watch?v=Pf3GQdkWByk&t=1060s

Maybe we can consider two split modes like coupled and Independent.

Might make sense, we should write down the use cases first.

acolombier · 2024-04-14T18:20:04Z

So I guess my suggestion on creating a variant of clone deck might make sense, right? I assume that the split stem feature only work if the sibling deck is free, so we could do something similar.

Might make sense, we should write down the use cases first.

That's a fair point. Could you put this into an issue maybe so we can gather all different usecase to expect around stem? I've been trying to keep the stem PRs fairly atomic - one step at a time - so I won't be doing any "advanced" controls in this PR. That "split stem" is definitely something I will look into as that definitely would work well on my S4 Mk3

JoergAtGithub · 2024-04-14T18:28:38Z

We should clarify in general, where to put the documentation of the GSoC projects.

Maybe a STEM-Controls proposal at https://github.com/mixxxdj/proposals might be also suitable here?

Co-authored-by: JoergAtGithub <[email protected]> Co-authored-by: Daniel Schürmann <[email protected]>

src/widget/wstemcontrol.cpp

acolombier · 2024-07-28T10:40:28Z

@daschuer @JoergAtGithub I took the freedom to remove the legacy UI wiget as it looks like we are getting closer to merge this PR. If you need to do more testing, the widget is available here.

I have kept the QML part as this one is pretty functional. Arguably, we may want to change the layout in the future, but the fact that nothing happens in QMl for the last 6 months makes me sad so I'd like to have stem in there, if that's okay! :)

JoergAtGithub · 2024-07-28T11:09:07Z

The code in this PR is LGTM now, but I miss test cases for the new COs. could you please add some basic tests for them.

acolombier · 2024-07-30T07:57:26Z

@JoergAtGithub I added some tests now that should cover all new stem COs

daschuer · 2024-07-30T12:58:51Z

Thank you for the next great step.

Do we have plans regarding a widget based skin?
I am afraid without it, it will take a while until stem support is usable for all users.

acolombier · 2024-07-30T14:55:59Z

We've been discussing that in the draft PR with the widget based skin feature. The TL;DR is no easy way to support all themes correctly, but we could release the stem control under some kind of "opt-in" alpha and address some of the common issues with it

daschuer · 2024-07-30T17:03:29Z

Ah Ok. Yes and even a single Stem skin would be reasonable. Maybe it attracts other skin designers to help with the rest.

JoergAtGithub · 2024-07-30T17:57:46Z

src/test/stemcontrolobjecttest.cpp

+        m_pStem1Volume = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_1_volume");
+        m_pStem2Volume = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_2_volume");
+        m_pStem3Volume = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_3_volume");
+        m_pStem4Volume = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_4_volume");
+        m_pStem1Mute = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_1_mute");
+        m_pStem2Mute = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_2_mute");
+        m_pStem3Mute = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_3_mute");
+        m_pStem4Mute = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_4_mute");
+        m_pStem1Color = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_1_color");
+        m_pStem2Color = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_2_color");
+        m_pStem3Color = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_3_color");
+        m_pStem4Color = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_4_color");
+        m_pStemCount = std::make_unique<PollingControlProxy>(m_sGroup1, "stem_count");


Before I merge this PR, which introduces new COs, I would like to hear your opinion about the naming scheme of the stem COs ( the full list can be found in the description of #13116 ).
Do you prefer the style:
[Channel1],stem_1_volume or [Channel1Stem1],volume
The first has the benefit, that the group name is similar to other COs in the mappings,
the second would have the benefit, that we seperate channel and function clearer and we could add it as just another group syntax to the documentation of some existing COs.
The first syntax would be in line with [Channel1],hotcue_X_color, the second with [QuickEffectRack1_[ChannelXStemY]],super1.
If we go with the first syntax, than I think [QuickEffectRack1_[Channe1Stem1]],super1 should become [QuickEffectRack1_[Channel1]],stem1_super1 for consistency.
What's your opinion?

Honestly, I think using [ChannelXStemY] globally would make a lot of sense, but I remember stumbling across something and settling for [ChannelX],stem_Y_....
Now that my Mixxx knowledge has grew, I will retry and see if I can get ChannelXStemY working for consistency.

Ok, that was surprisingly easy finally, I do not know how come I didn't do this before 😅
Also updated the manual and will update the epic right away.
Note that I have opted for track_color so it matches 1-to-1 with the other channels

src/mixer/basetrackplayer.cpp

JoergAtGithub · 2024-07-30T19:53:22Z

Thank you very much!

acolombier · 2024-07-30T21:49:28Z

Thanks for your review! Next stop is either #13268 or #13123

ronso0 · 2024-09-23T08:58:36Z

src/test/stemcontrolobjecttest.cpp

+            "StemVolumeControlFull");
+}
+
+TEST_F(StemControlTest, VolumeResetOnLoad) {


For some reason this test failed here
https://github.com/mixxxdj/mixxx/actions/runs/10981245295/job/30488031679?pr=13077#step:11:1823
(in case the link doesn't work, the PR is #13077)

any idea why @acolombier ?

No idea. It looks like it is failing with an unrelated assertion, not quite sure why this would happen

oh, I see it's loading one of the dummy tracks. sorry for the noise then

github-actions bot added engine build code quality soundsource analyzer qml labels Apr 10, 2024

acolombier marked this pull request as draft April 10, 2024 20:45

acolombier force-pushed the feat/add-stem-controls branch from de099b9 to c9f2c2d Compare April 11, 2024 11:43

acolombier force-pushed the feat/add-stem-controls branch from c9f2c2d to eaefb08 Compare April 11, 2024 22:22

acolombier force-pushed the feat/add-stem-controls branch from eaefb08 to 0eb12c2 Compare April 12, 2024 17:45

github-actions bot added library ui waveform labels Apr 12, 2024

acolombier force-pushed the feat/add-stem-controls branch from eed35f2 to a17c578 Compare July 25, 2024 18:54

Apply suggestions from code review

de5bfaf

Co-authored-by: JoergAtGithub <[email protected]> Co-authored-by: Daniel Schürmann <[email protected]>

acolombier force-pushed the feat/add-stem-controls branch from a17c578 to de5bfaf Compare July 25, 2024 19:16

acolombier requested review from JoergAtGithub and daschuer July 26, 2024 08:05

JoergAtGithub reviewed Jul 28, 2024

View reviewed changes

src/widget/wstemcontrol.cpp Outdated Show resolved Hide resolved

acolombier added 4 commits July 28, 2024 11:19

Fix invalid QML index and layout

43acd47

Make constant plural

9d83a3c

Revert "feat: add UI control for stem"

6eed2eb

Merge remote-tracking branch 'upstream/main' into feat/add-stem-controls

7fdd9e0

acolombier requested a review from JoergAtGithub July 28, 2024 10:40

Add unit test for stem control

ffeca7e

acolombier force-pushed the feat/add-stem-controls branch from f5f6dd9 to ffeca7e Compare July 30, 2024 07:56

daschuer approved these changes Jul 30, 2024

View reviewed changes

JoergAtGithub reviewed Jul 30, 2024

View reviewed changes

src/mixer/basetrackplayer.cpp Outdated Show resolved Hide resolved

Rename stem COs to match quick FX group names

0a4ba6d

acolombier force-pushed the feat/add-stem-controls branch from 2a3b142 to 0a4ba6d Compare July 30, 2024 19:03

JoergAtGithub approved these changes Jul 30, 2024

View reviewed changes

JoergAtGithub merged commit 9b31059 into mixxxdj:main Jul 30, 2024
13 checks passed

ronso0 reviewed Sep 23, 2024

View reviewed changes

fwcd mentioned this pull request Dec 30, 2024

Regression: Mixxx 2.6 reanalyzes tracks on every load #14089

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add stem controls #13086

Add stem controls #13086

acolombier commented Apr 10, 2024 •

edited

Loading

JoergAtGithub commented Apr 10, 2024

acolombier commented Apr 11, 2024

daschuer commented Apr 11, 2024

acolombier commented Apr 11, 2024 •

edited

Loading

vantaboard commented Apr 11, 2024 •

edited

Loading

acolombier commented Apr 11, 2024

vantaboard commented Apr 11, 2024 •

edited

Loading

vantaboard commented Apr 11, 2024

acolombier commented Apr 11, 2024

vantaboard commented Apr 11, 2024

vantaboard commented Apr 12, 2024

JoergAtGithub commented Apr 12, 2024

daschuer commented Apr 12, 2024

acolombier commented Apr 12, 2024 •

edited

Loading

JoergAtGithub commented Apr 14, 2024

acolombier commented Apr 14, 2024

JoergAtGithub commented Apr 14, 2024

acolombier commented Jul 28, 2024

JoergAtGithub commented Jul 28, 2024

acolombier commented Jul 30, 2024

daschuer commented Jul 30, 2024

acolombier commented Jul 30, 2024

daschuer commented Jul 30, 2024

JoergAtGithub Jul 30, 2024

acolombier Jul 30, 2024

acolombier Jul 30, 2024 •

edited

Loading

JoergAtGithub commented Jul 30, 2024

acolombier commented Jul 30, 2024

ronso0 Sep 23, 2024

acolombier Sep 24, 2024

ronso0 Sep 24, 2024

Add stem controls #13086

Add stem controls #13086

Conversation

acolombier commented Apr 10, 2024 • edited Loading

JoergAtGithub commented Apr 10, 2024

acolombier commented Apr 11, 2024

daschuer commented Apr 11, 2024

acolombier commented Apr 11, 2024 • edited Loading

vantaboard commented Apr 11, 2024 • edited Loading

acolombier commented Apr 11, 2024

vantaboard commented Apr 11, 2024 • edited Loading

vantaboard commented Apr 11, 2024

acolombier commented Apr 11, 2024

vantaboard commented Apr 11, 2024

vantaboard commented Apr 12, 2024

JoergAtGithub commented Apr 12, 2024

daschuer commented Apr 12, 2024

acolombier commented Apr 12, 2024 • edited Loading

JoergAtGithub commented Apr 14, 2024

acolombier commented Apr 14, 2024

JoergAtGithub commented Apr 14, 2024

acolombier commented Jul 28, 2024

JoergAtGithub commented Jul 28, 2024

acolombier commented Jul 30, 2024

daschuer commented Jul 30, 2024

acolombier commented Jul 30, 2024

daschuer commented Jul 30, 2024

JoergAtGithub Jul 30, 2024

Choose a reason for hiding this comment

acolombier Jul 30, 2024

Choose a reason for hiding this comment

acolombier Jul 30, 2024 • edited Loading

Choose a reason for hiding this comment

JoergAtGithub commented Jul 30, 2024

acolombier commented Jul 30, 2024

ronso0 Sep 23, 2024

Choose a reason for hiding this comment

acolombier Sep 24, 2024

Choose a reason for hiding this comment

ronso0 Sep 24, 2024

Choose a reason for hiding this comment

acolombier commented Apr 10, 2024 •

edited

Loading

acolombier commented Apr 11, 2024 •

edited

Loading

vantaboard commented Apr 11, 2024 •

edited

Loading

vantaboard commented Apr 11, 2024 •

edited

Loading

acolombier commented Apr 12, 2024 •

edited

Loading

acolombier Jul 30, 2024 •

edited

Loading