Example that merge pgn files including comments?

[olleeriksson]

I ran across https://github.com/permutationlock/merge-pgn which merges PGN files (I mean really merges them, not just concatenates them) but it doesn't handle comments or software comments (annotations, arrows and circles). I started looking into how it uses python-chess and before I delve deeper into this...

Does anybody know of an example application that already merges PGN files including comments?

[tissatussa]

take a look at https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/help.html - not sure, but it's famous and has many options.

[olleeriksson]

Actually I've already looked at it, and even talked to the author. And no it doesn't do what I am looking for. He was open to implement something similar in the future though.

There is a bunch of tools that do the opposite, split one game into multiple games, but almost none that does the opposite. Probably because it's harder, though I suspect it should be so hard using python-chess.

[tissatussa]

OK .. when you say "merge" you mean the move comments of all (exactly) same games will be joined into PGN having complete info ?

[tissatussa]

..if i understand correctly .. so, notation like d4!? or d4 $4 will be detected and all { ..comment.. } ? Should be rather easy to write a python script for that .. and detect all same games in all input PGN files ?

[olleeriksson]

What I mean is merge all games into one game with variations and have all comments and all annotations (like %cal, %csl, and nags) be "merged" so that if the same position in game 1 and 2 both had a comment and some %cal arrows or %csl circles or nags, they would all be included in that position in the merged game.

[tissatussa]

well said .. now i understand .. yes, it seems a realistic idea, indeed i wonder if anyone ever wanted to create such .. maybe the reason is programs like SCID can also manage / sort many games, but it uses some DB format .. sometimes i ask myself questions or wish a function nobody talks about .. then you might discover another way, at first not obvious for you ?

[brianprichardson]

SCID can do this, I think.
And, while it does use an internal DB format, you can select a database and export it in PGN format.

[tissatussa]

you can select a database and export it in PGN format

but then all games have their own header info and only their original variations & annotations .. not merging ? I use SCID mainly to run MultiPV evals, not its DB.

[olleeriksson]

Interesting idea, that maybe there's a real reason why hardly any tools do this.

So what I am trying to do is create what some people refer to as a mono-repertoire. One big PGN game that contains all openings and all variations. Lichess has this limit on chapter length so I have to split up my repertoire into multiple chapters and when you export them they end up being separate games in the same PGN file.

I've tried before to do this in SCID and I don't think it's there. Chesstempo.com allows you to do this and that's what I use right now, but I've been looking for a command line tool to do it so speed up the process of converting the Lichess study to one large PGN game.

But, I've been spending some time tonight, actually having the new AI ChatGPT genereate some code for me to facilitate the merging of the comments and %cal/&csl annotations and I think I very soon have something that works. :) I'll post it when I do.

[tissatussa]

good luck and i'm looking forward to any developments !

[olleeriksson]

By the way, it seems python-chess doesn't support multiple comments per node, right? I am not even sure if the PGN standard even allows it, but I see some software used it, where you get something like:

1... e5 { First comment } { Second comment } { [%cal GNc6] }

Since I can't see that this can be generated by python-chess I will have to simply merge the comments and separate them by two newlines or something.

[tissatussa]

..python-chess doesn't support multiple comments per node, right?

i don't know .. regarding PGN specifications, i use the links on this page : https://github.com/fsmosca/PGN-Standard

[olleeriksson]

Hmm.. I seem to run into this issue still. Lichess.org for example stores text comments separate from the %cal/%csl annotations, like so:

1 e4 { A text comment } { [%cal Ge1e3] }

And the Javascript chess pgn-parser that I am using in a frontend client seem to require them to be in a separate {} section.

I created an issue on this:
#946

[olleeriksson]

So here is the python script that merges multiple PGNs including comments and software comments/annotations (arrows and circles etc). It's not perfect and the solution is a bit hacky to work around the limitation in python-chess but it works for my purposes.

https://github.com/olleeriksson/merge-pgn