Subscripts in the source line of an interlinear example are not properly rendered

[CLRafaelR]

Professor Dr. @cysouw

I use the latest pandoc-ling 2a1e55a as of June 26, 2022. I want to add subscripts that indicate coindexation to the source line of an interlinear example, as shown below.

However, I found I cannot achieve this either using ~...~ notation of pandoc markdown or using `\textsubscript{...}`{=latex} notation, although these notations successfully work in the translation line.

Using ~...~ notation of pandoc markdown

I implemented the following notation but the subscripts are rendered as normal characters (e.g. not '_i' but 'i').

::: {#subscript-superscript-md .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō~i~=wa Jirō~j~=ga zibun~i/j~=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::

Using `\textsubscript{...}`{=latex} notation

The subscripts disappear when I added subscripts using `\textsubscript{...}`{=latex} notation.

::: {#subscript-superscript-tex .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō`\textsubscript{i}`{=latex}=wa Jirō~`\textsubscript{j}`{=latex}~=ga zibun`\textsubscript{i/j}`{=latex}=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::

Would you mind telling me whether there is any workaround or what I can do to resolve the issue?

MWE

---
author: CLRR
to: pdf
title: Sub/Superscript workaround
latexPackage: gb4e
---

I want to add subscripts that indicate coindexation to the source line of an interlinear example, as shown below.

```{=latex}
\begin{exe} \judgewidth{}
  \ex [] { 
       Japanese coindex using subscript
  \gll \emph{Tarō\textsubscript{i}=wa} \emph{Jirō\textsubscript{j}=ga} \emph{zibun\textsubscript{i/j}=o} \emph{suki-da-to}
\emph{omot-te} \emph{i-ru.} \\
       T.=\textsc{top} J.=\textsc{nom} self=\textsc{acc}
like-\textsc{aff}-\textsc{quot} think-\textsc{cvb}
\textsc{prog}-\textsc{npst} \\
  \glt `Tarō\textsubscript{i} assumes that Jirō\textsubscript{j} likes
him\textsubscript{i}/himself\textsubscript{j}.' }
  \label{subscript-superscript-tex}
\end{exe}
```

However, using `~...~` notation in the source line fails to add subscripts.
The subscripts are rendered as normal characters
(e.g. not '~i~' but 'i'),
as demonstrated in [@subscript-superscript-md].

```
::: {#subscript-superscript-md .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō~i~=wa Jirō~j~=ga zibun~i/j~=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::
```


::: {#subscript-superscript-md .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō~i~=wa Jirō~j~=ga zibun~i/j~=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::

Using ```\textsubscript{i}`{=latex}`` notation in the source line also fails to add subscripts.
In this case, the subscripts disappear, as illustrated in [@subscript-superscript-tex].

```
::: {#subscript-superscript-tex .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō`\textsubscript{i}`{=latex}=wa Jirō~`\textsubscript{j}`{=latex}~=ga zibun`\textsubscript{i/j}`{=latex}=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::
```

::: {#subscript-superscript-tex .ex formatGloss=true}
| Japanese coindex using subscript
| Tarō`\textsubscript{i}`{=latex}=wa Jirō~`\textsubscript{j}`{=latex}~=ga zibun`\textsubscript{i/j}`{=latex}=o suki-da-to omot-te i-ru.
| T.=TOP J.=NOM self=ACC like-AFF-QUOT think-CVB PROG-NPST
| Tarō~i~ assumes that Jirō~j~ likes him~i~/himself~j~.
:::

[CLRafaelR]

I think a formatting function similar to formatGlossLine is necessary for splitSource so that a string ~...~ is converted into small caps via pandoc.SmallCaps(), and non-alphabetical symbols like / are rendered as they are (i.e. / is not italicised).

pandoc-ling/pandoc-ling.lua

Lines 444 to 465 in 2a1e55a

	function formatGlossLine (s)
	-- turn uppercase in gloss into small caps
	local split = {}
	for lower,upper in string.gmatch(s, "(.-)([%u%d][%u%d]+)") do
	if lower ~= "" then
	lower = pandoc.Str(lower)
	table.insert(split, lower)
	end
	upper = pandoc.SmallCaps(pandoc.text.lower(upper))
	table.insert(split, upper)
	end
	for leftover in string.gmatch(s, "[%u%d][%u%d]+(.-[^%u%s])$") do
	leftover = pandoc.Str(leftover)
	table.insert(split, leftover)
	end
	if #split == 0 then
	if s == "~" then s = "   " end -- sequence "space-nobreakspace-space"
	table.insert(split, pandoc.Str(s))
	end
	-- result is list of inlines
	return split
	end

[cysouw]

The basic problem is the automatic formatting that is done by panda-ling, which can be controlled by the option 'formatGloss`. Basically, this is a convenience option that works for many glosses, but it will never work for all options that people would like to throw at their glossing.

For that reason I added the possibility to prevent this automatic formatting on a per-example basis. So, if you find that a certain gloss is not formatted as you would like it to be, then you should start the example with ::: {.ex formatGloss=false} and then simply exactly format the gloss as you would like it to be.