Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[musicxml2hum] Fix missing lines when nowevents includes nonzerodur elements #88

Merged
merged 18 commits into from
Dec 12, 2023

Conversation

WolfgangDrescher
Copy link
Contributor

This PR fixes the missing data lines problem explained in #87.

However, it introduces a new bug. The parts and staves are now in the wrong order:

**kern	**kern	**kern	**kern	**fb	**kern	**kern	**kern	**kern
*part6	*part5	*part6	*part5	*part5	*part4	*part3	*part2	*part1
*staff8	*staff6	*staff7	*staff5	*	*staff4	*staff3	*staff2	*staff1

Expected:

**kern	**fb	**kern	**kern	**kern	**kern	**kern	**kern	**kern
*part6	*part6	*part6	*part5	*part5	*part4	*part3	*part2	*part1
*staff8	*	*staff7	*staff6	*staff5	*staff4	*staff3	*staff2	*staff1

Also note that the **fb spine will be attached to the wrong part and staff now. Do you have any idea how to debug this further?


Sourcefile: Schiorring - 04 Af dybeſte Nød raaber jeg til dig.mxl.zip

Screenshot of the output in VHV:

Bildschirmfoto 2023-12-04 um 11 15 44
Click to see new output of musicxml2hum
!!!COM: Schiørring, Niels
!!!CDT: 1743-1798
!!!OPR@@DA: Choral-Bog
!!!ONM: 4
!!!OTL@@DA: Af dybeſte Nød raaber jeg til dig
**kern	**kern	**kern	**kern	**fb	**kern	**kern	**kern	**kern
*part6	*part5	*part6	*part5	*part5	*part4	*part3	*part2	*part1
*staff8	*staff6	*staff7	*staff5	*	*staff4	*staff3	*staff2	*staff1
*clefF4	*clefF4	*clefC1	*clefC1	*	*clefF4	*clefC4	*clefC3	*clefC1
*mclefF4	*	*mclefG2	*mclefF4	*mclefG2	*mclefF4	*mclefGv2	*mclefG2	*mclefG2
*k[]	*k[]	*k[]	*k[]	*	*k[]	*k[]	*k[]	*k[]
*M4/4	*M4/4	*M4/4	*M4/4	*	*M4/4	*M4/4	*M4/4	*M4/4
*met(c)	*met(c)	*met(c)	*met(c)	*	*met(c)	*met(c)	*met(c)	*met(c)
*MM180	*	*MM180	*MM180	*MM180	*MM180	*MM180	*MM180	*MM180
*	*^	*	*^	*	*	*	*	*
2E	2G	2E	2b	2b	2e	.	2E	2G	2e	2b
=1	=1	=1	=1	=1	=1	=1	=1	=1	=1	=1
2G	2B	2G	2e	2e	2e	.	2G	2B	2e	2e
2E	2G	2E	2b	2b	2e	.	2E	2G	2e	2b
=2	=2	=2	=2	=2	=2	=2	=2	=2	=2	=2
2AA	2A	2AA	2cc	2cc	2e	.	2AA	2A	2e	2cc
2BB	2F#X	2BB	4b	4b	2d#X	.	2BB	2F#X	2d#X	(4b
.	.	.	4a	4a	.	.	.	.	.	4a)
=3	=3	=3	=3	=3	=3	=3	=3	=3	=3	=3
1C	1E	1C	2g	2g	1e	.	2C	2E	2e	2g
.	.	.	2a	2a	.	.	2C	2A	2e	2a
=4	=4	=4	=4	=4	=4	=4	=4	=4	=4	=4
2BB	2F#X	2BB	2b	2b	2d#X	.	2BB	2F#X	2d#X	2b
2E	2G#X	2E	2b	2b	2e	.	2E	2G#X	2e	2b
=5	=5	=5	=5	=5	=5	=5	=5	=5	=5	=5
2A	[2A	2A	2cc	2cc	2e	.	2A	2A	2e	2cc
2D	4A]	2Dni	2dd	2ddni	2fni	.	2D	(4A	2f	2dd
.	4B	.	.	.	.	6	.	4B)	.	.
=6	=6	=6	=6	=6	=6	=6	=6	=6	=6	=6
2E	2.c	2E	2cc	2cc	2g	.	2E	2c	2g	2cc
2F	.	2F	2a	2a	2f	.	2F	(4c	2f	2a
.	4d	.	.	.	.	6	.	4d)	.	.
=7	=7	=7	=7	=7	=7	=7	=7	=7	=7	=7
2C#X	4ryy	2C#X	(2g	4e	2e	.	(2C#X	(4e	(2e	(2g
.	4A	.	.	4ryy	.	.	.	2A	.	.
2D	2A	2D	2f)	2f	4d	.	2D)	.	4d	2f)
.	.	.	.	.	4cni	.	.	4A)	4c)	.
=	=	=	=	=	=	=	=	=	=	=
2E	2G#X	2E	2e	2e	2B	.	2E	2G#X	2B	2e
=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!	=8:|!
[2F	2A	[2F	2a	2a	2d	.	[2F	2A	2d	2a
=9	=9	=9	=9	=9	=9	=9	=9	=9	=9	=9
2F]	2B	2F]	2g	2g	2d	.	2F]	2B	2d	2g
*	*	*	*	*	*^	*	*	*	*	*
4E	4c	4E	2cc	2cc	4g	4e\	.	(4E	(4c	(4g	2cc
4F#X	4A	4F#X	.	.	4a	4d\	.	4F#X)	4A)	4a)	.
=10	=10	=10	=10	=10	=10	=10	=10	=10	=10	=10	=10
1G	2d	2G	2b	2b	4g	2d\	.	2G	2d	(4g	2b
.	.	.	.	.	4fni	.	n7	.	.	4f)	.
.	2A	2G	2a	2a	2e	2c#X\	.	2G	2c#X	2e	2a
*	*	*	*	*	*v	*v	*	*	*	*	*
=11	=11	=11	=11	=11	=11	=11	=11	=11	=11	=11
4F	2.A	4F	4dd	4dd	4d	.	(4F	(4d	[2a	(4dd
4E	.	4E	4cc	4ccni	4e	.	4E)	4e)	.	4cc)
4D	.	4D	2b	2b	4f	.	(4D	(4f	(4a]	2b
4E	4G#X	4E	.	.	4e	.	4E)	4e)	4g#X)	.
=12	=12	=12	=12	=12	=12	=12	=12	=12	=12	=12
*	*	*	*	*	*^	*	*	*	*	*
2AA	2A	2AA	2a	2a	2c	2e/	.	2AA	2c	2a	2a
*	*	*	*	*	*v	*v	*	*	*	*	*
2F#X	2A	2F#X	2cc	2cc	2d	.	2F#X	2d	2a	2cc
=13	=13	=13	=13	=13	=13	=13	=13	=13	=13	=13
2G	2G	2G	2b	2b	2d	.	2G	2d	2g	2b
2C	2G	2C	2cc	2cc	2e	.	2C	2e	2g	2cc
=14	=14	=14	=14	=14	=14	=14	=14	=14	=14	=14
2BB	2G	2BB	2dd	2dd	2d	.	2BB	2d	2g	2dd
2E	2G	2E	2g	2g	2B	.	2E	2B	[2g	2g
=15	=15	=15	=15	=15	=15	=15	=15	=15	=15	=15
2AA	2.G	2AA	2cc	2cc	2e	.	2AA	2c	2g]	2cc
2D	.	2D	2a	2a	2d	.	2D	2d	(4g	2a
.	4F#X	.	.	.	.	#	.	.	4f#X)	.
=16	=16	=16	=16	=16	=16	=16	=16	=16	=16	=16
*	*	*	*	*	*^	*	*	*	*	*
2GG	2G	2GG	2g	2g	2B	2d/	.	2GG	2B	2g	2g
*	*	*	*	*	*v	*v	*	*	*	*	*
2F	2B	2Fni	2g	2g	2d	.	2F	2B	2d	2g
=17	=17	=17	=17	=17	=17	=17	=17	=17	=17	=17
2E	2c	2E	2cc	2cc	2g	.	2E	2c	2g	2cc
4D	4d	4D	2b	2b	4f	.	(4D	(4d	(4f	2b
4E	4G	4E	.	.	4g	.	4E)	4G)	4e)	.
=18	=18	=18	=18	=18	=18	=18	=18	=18	=18	=18
4F	2c	4F	2a	2a	2f	.	(4F	2A	2c	2a
4G	.	4G	.	.	.	.	4G)	.	.	.
2A	2A	2A	2e	2e	2c	.	2A	2A	2c	2e
=19	=19	=19	=19	=19	=19	=19	=19	=19	=19	=19
2C#X	1A	2C#X	(2g	2g	2e	.	(2C#X	1A	(2e	(2g
2D	.	2D	2f)	2f	4d	.	2D)	.	4d	2f)
.	.	.	.	.	4cni	7	.	.	4c)	.
=20	=20	=20	=20	=20	=20	=20	=20	=20	=20	=20
2E	2G#X	2E	2e	2e	2B	.	2E	2G#X	2B	2e
*	*v	*v	*	*	*	*	*	*	*	*
*	*	*	*v	*v	*	*	*	*	*
==	==	==	==	==	==	==	==	==
*-	*-	*-	*-	*-	*-	*-	*-	*-
!!!system-decoration: {(s1,s2,s3,s4)}{(s5,s6)}{(s7,s8)}
!!!RDF**kern: i = editorial accidental
!!!RDF**kern: > = above
!!!RDF**kern: < = below
!!!AGN: chorale
!!!EED: Derek Remeš
!!!EED2: Victor Phan
!!!END: 2023/11/28
!!!ENC: Wolfgang Drescher
!!!EEV: 2023/12/04
!!!title: @{ONM}. @{OTL}

@WolfgangDrescher WolfgangDrescher changed the title Fix missing lines when nowevents includes nonzerodur elements [musicxml2hum] Fix missing lines when nowevents includes nonzerodur elements Dec 4, 2023
@WolfgangDrescher
Copy link
Contributor Author

I debugged this further and it looks like there are two different problems. I use this file for debugging 004-p5-6-m5-7.xml.zip

It looks like it's currently not possible to add figured bass numbers when there is no note in the bass. MuseScore also seems to have problems with this when I open the debugging file:

Bildschirmfoto 2023-12-05 um 11 44 37

Both 6 figures should actually assigned to the last quarter note of the measures, and visually below the quarter passing note of piano 1. This will also mess up musicxml2hum and push the **fb spine to another field index.


Regardless of this the **kern spines are not ordered correctly:

**kern	**kern	**kern	**kern
*part2	*part1	*part2	*part1
*staff4	*staff2	*staff3	*staff1

Expected:

**kern	**kern	**kern	**kern
*part2	*part2	*part1	*part1
*staff4	*staff3	*staff2	*staff1

@WolfgangDrescher
Copy link
Contributor Author

Never mind about the wrong order of the systems. This was caused by my import script to relocate the figured bass numbers from the upper to the lower system of piano 2…

https://github.com/WolfgangDrescher/schiorring-choral-bog/blob/29c04e254cff1f817f38835270377ec8cdbeff75/convert#L59

So the problems seems to be that **fb is mapped to the wrong system:

!!!OTL: Choral Bog
!!!OMV: Af dybeſte Nød raaber jeg til dig
**kern	**kern	**kern	**kern	**fb
*part2	*part2	*part1	*part1	*part1
*staff4	*staff3	*staff2	*staff1	*
*I"Piano 2	*	*I"Piano 1	*	*
=1	=1	=1	=1	=1
*	*	*^	*^	*
2A	2cc	[2A	2A	2cc	2e	.
2D	2dd	4A]	2Dni	2ddni	2fni	.
.	.	4B	.	.	.	6
=2	=2	=2	=2	=2	=2	=2
2E	2cc	2.c	2E	2cc	2g	.
2F	2a	.	2F	2a	2f	.
.	.	4d	.	.	.	6
=3	=3	=3	=3	=3	=3	=3
2C#X	(2g	4ryy@	2C#X	4e	2e	.
.	.	4A	.	4ryy@	.	.
2D	2f)	2A	2D	2f	4d	.
.	.	.	.	.	4cni	.
*	*	*v	*v	*	*	*
*	*	*	*v	*v	*
=	=	=	=	=
*-	*-	*-	*-	*-
!!!system-decoration: {(s1,s2)}{(s3,s4)}
!!!RDF**kern: i = editorial accidental

I will have a look at this.

@WolfgangDrescher
Copy link
Contributor Author

I have improved the assignment of the figured bass numbers, but it is not perfect yet. I have created a few files that allow to debug this easier: fb-examples.zip

Here is the current status for each example file:

Example 0

MuseScore

fb0

VHV (after)

Bildschirmfoto 2023-12-06 um 16 30 41

This is fine. But it's strange that musicxml2hum creates a **recip spine.

Example 1

MuseScore

fb1

VHV before VHV after
fb1-before Bildschirmfoto 2023-12-06 um 17 43 41

I'm not sure why the figured bass numbers in the oboe are not displayed. I will keep debugging this.

Example 2

MuseScore

fb2

VHV before VHV after
fb2-before Bildschirmfoto 2023-12-06 um 17 44 14

As there is only one **fb spine per part (note that this is one part with 2 staves): I think it is fine that the fb numbers of both staves are displayed in a single **fb spine.

Example 3

MuseScore

fb3

VHV before VHV after
fb3-before Bildschirmfoto 2023-12-06 um 17 45 14

Example 4

MuseScore

fb4

VHV before VHV after
fb4-before Bildschirmfoto 2023-12-06 um 17 45 48

Also here I think it is fine, that the **fb spine is placed after the piano part and thus the numbers are displayed below the upper staff.

Example 5

MuseScore

fb5

VHV before VHV after
fb5-before Bildschirmfoto 2023-12-06 um 17 46 23

Note that before there was a **recip spine as first spine. I'm not sure why the numbers in the oboe are not displayed. Probably the same issue as in example 1.

Example 6

MuseScore

fb6

VHV before VHV after
syntax error syntax error

Notice the **recip spine. I think that for this example the numbers of both staves should be grouped together and 8 1 should be the first data record of the fb spine.

@WolfgangDrescher
Copy link
Contributor Author

WolfgangDrescher commented Dec 6, 2023

With 9d4510c the results are now much better. But example 6 is still failing though.

Additionally I run the updated musicxml2hum script against the chorales by Niels Schiørring. The results have gotten much better, but I have found some other problems, especially with spine merges and the duration of rests. I will try to fix these as well.

@WolfgangDrescher
Copy link
Contributor Author

I improved the detection for "slahed accidentals" by allowing vertical,back-slash and sharp-sharp as values in <prefix> and <suffix>.

This does comply with the specs in:
https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/prefix/
https://www.w3.org/2021/06/musicxml40/musicxml-reference/elements/suffix/

Prefix should not contain vertical and back-slash but to keep it compatible with the existing code I allow them in <prefix> as well.

// ...
} else if (suffix == "flat-flat") {
	accidental = "--r";
} else if (suffix == "flat") {
	accidental = "-r";
} else if (suffix == "double-sharp" || suffix == "sharp-sharp") {
	accidental = "##r";
} else if (suffix == "sharp") {
	accidental = "#r";
} else if (suffix == "natural") {
	accidental = "nr";
}

Why does the accidental include r at the end when it is in the suffix? This will probably result in the data token being interpreted as a rest. If this is intended? Should I remove the r there?

@craigsapp
Copy link
Owner

The r is to "reverse" the position of the accidental:

Screenshot 2023-12-07 at 04 32 41

@WolfgangDrescher
Copy link
Contributor Author

In b45e76d I removed adding slicep->getDuration() to the duration for ryy@ tokens in data holes. I'm not sure if this is a good idea or if this will cause any other unexpected errors. But it solved may duration problems when importing the Schiørring chorales. Here is a demo MusicXML file: 011-den-lyse-dag-forgangen-er.musicxml.zip

Measures 4-5 before the change:

=4	=4	=4	=4	=4	=4	=4	=4	=4	=4	=4
1D	1dd	.	1D	1dd	1f	1a\	1D	1ff	1a	1dd
*	*	*	*^	*	*	*	*	*	*	*
2F#X	2ccni	5	2F#X	2d	2ccni	2%7ryy@	[2a\	2F#X	2dd	[2a	2ccni
*	*	*	*	*	*	*v	*v	*	*	*	*
=5	=5	=5	=5	=5	=5	=5	=5	=5	=5	=5
1G	1b-	9	1G	1d	1b-	2a]	1G	1dd	(2a]	1b-
.	.	8	.	.	.	2g	.	.	2g)	.
2B-	2g	6	2B-	2d	2g	[2g	2B-	2dd	[2g	2g

After:

=4	=4	=4	=4	=4	=4	=4	=4	=4	=4	=4
1D	1dd	.	1D	1dd	1f	1a\	1D	1ff	1a	1dd
*	*	*	*^	*	*	*	*	*	*	*
2F#X	2ccni	5	2F#X	2d	2ccni	2ryy@	[2a\	2F#X	2dd	[2a	2ccni
*	*	*	*	*	*	*v	*v	*	*	*	*
=5	=5	=5	=5	=5	=5	=5	=5	=5	=5	=5
1G	1b-	9	1G	1d	1b-	2a]	1G	1dd	(2a]	1b-
.	.	8	.	.	.	2g	.	.	2g)	.
2B-	2g	6	2B-	2d	2g	[2g	2B-	2dd	[2g	2g

2%7ryy@ got replaced with 2ryy@.


Measures 14-15 before the change

=14	=14	=14	=14	=14	=14	=14	=14	=14	=14	=14	=14
2D	1f	.	2D	2A	1f	2ryy@	1d\	(2D	(2a	1d	1f
*	*	*	*v	*v	*	*	*	*	*	*	*
2GG#X	.	7 5	2GG#X	.	2Bn	.	2GG#X)	2bn)	.	.
*	*	*	*^	*	*	*	*	*	*	*
2AA	2e	#	2AA	4A	2e	4ryy@	2c#X\	2AA	2a	2c#X	2e
.	.	.	.	4Gni	.	1..ryy@	.	.	.	.	.
*	*	*	*	*	*	*v	*v	*	*	*	*
=15	=15	=15	=15	=15	=15	=15	=15	=15	=15	=15
1.D	1.d	.	1.D	1.F	1.d	1.A	1.D	1.f	1.d	1.d
*	*	*	*v	*v	*	*	*	*	*	*

After:

=14	=14	=14	=14	=14	=14	=14	=14	=14	=14	=14	=14
2D	1f	.	2D	2A	1f	2ryy@	1d\	(2D	(2a	1d	1f
*	*	*	*v	*v	*	*	*	*	*	*	*
2GG#X	.	7 5	2GG#X	.	2Bn	.	2GG#X)	2bn)	.	.
*	*	*	*^	*	*	*	*	*	*	*
2AA	2e	#	2AA	4A	2e	4ryy@	2c#X\	2AA	2a	2c#X	2e
.	.	.	.	4Gni	.	4ryy@	.	.	.	.	.
*	*	*	*	*	*	*v	*v	*	*	*	*
=15	=15	=15	=15	=15	=15	=15	=15	=15	=15	=15
1.D	1.d	.	1.D	1.F	1.d	1.A	1.D	1.f	1.d	1.d
*	*	*	*v	*v	*	*	*	*	*	*

1..ryy@ got replaced with 4ryy@.

@WolfgangDrescher
Copy link
Contributor Author

d309393 does improve the handling of cross staff layers (same voice number). If notes are generally in the lower staff but occasionally also appear in the upper staff this works fine as I can add the the total number of voices of the new staff to the current voice index (in case multiple layers change the staff). But is does not yet work well when notes that are generally in the upper staff but occasionally are pushed to the lower staff. The stems are wrong, because the voice should be the new voice index 0 of the new staff and all other voices should be increased by +1 (or the number of voices that change their staff).

Do you have an idea how to solve this problem? MxmlEvent is not aware of the other voices so I cannot modify the voice index of all other voices that need +1 there.

Maybe using high voice indices such as 99 and 98 could be used for layers that need to be pushed in front of the voices index array. ThenreindexMeasure could resolve this. But it is a hacky solution.

MuseScore Before After
Bildschirmfoto 2023-12-12 um 12 27 16 Bildschirmfoto 2023-12-12 um 12 27 39 Bildschirmfoto 2023-12-12 um 12 28 42
Bildschirmfoto 2023-12-12 um 12 29 32 Bildschirmfoto 2023-12-12 um 12 30 07 Bildschirmfoto 2023-12-12 um 12 30 35

voice-mapping-musescore-example-files.zip

@craigsapp
Copy link
Owner

Here is a good encoding of the example (which may or may not be reflected in the data):

Screenshot 2023-12-12 at 03 55 12
**kern	**kern
*clefF4	*clefG2
*M4/4	*M4/4
=1	=1
*^	*
4G	4C	4cc
4A	4C	4cc
4B	4C	4cc
4c>	4C	4cc
=2	=2	=2
4d>	4C	4cc
4e>	4C	4cc
4f>	4C	4cc
4g>	4C	4cc
*v	*v	*
==	==
*-	*-
!!!RDF**kern: > = above

@WolfgangDrescher
Copy link
Contributor Author

7df2873 improves this behavior so that a voice (=same voice number) always has one staff to which it is assigned. If notes should be rendered in another staff than this "root staff" > and < signifiers and RDF**kern above/below lines are added.

Is it okay to push this to humlib like this? I could add an option to musicxml2hum so that this behavior has to be activated with -x / --cross-staff or similar.

@craigsapp
Copy link
Owner

7df2873 improves this behavior so that a voice (=same voice number) always has one staff to which it is assigned. If notes should be rendered in another staff than this "root staff" > and < signifiers and RDF**kern above/below lines are added.

Is it okay to push this to humlib like this? I could add an option to musicxml2hum so that this behavior has to be activated with -x / --cross-staff or similar.

Preserving voicing information as you are doing is better than changing the voice to the display staff. We will see what that does in the general case. Then perhaps -X and --no-cross-staff could be used to avoid this behavior.

So I should merge this PR now into humlib?

@WolfgangDrescher
Copy link
Contributor Author

Yes, I'm happy with it for now. Example 6 of fb-examples.zip – where ideally figured bass numbers should be combined from all voices and stacked – is still not perfect, but it's a special case which I don't need right now. So it's ready for review and to merge. I can implement it, when someone needs it.

@WolfgangDrescher WolfgangDrescher marked this pull request as ready for review December 12, 2023 18:16
@craigsapp craigsapp merged commit 6047795 into craigsapp:master Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants