Transformation results in error if document contains multiple identically-named link targets

[andrew-selvia]

Here's the error I'm trying to resolve:

laika.parse.markup.DocumentParser$InvalidDocuments: One or more invalid documents: One or more errors processing document '...': More than one link target with id '...' in path ...

A document which could reproduce this error (id: 'pros' or 'cons') is:

# Comparison

## Option 1

### Pros

### Cons

## Option 2

### Pros

### Cons

I'm new to Laika (liking it so far, great documentation 😄). I tried separating parsing and rendering so I could alter the Document AST to name each link independently, but it seems the parsing itself is failing so I never get the Document. Is there something obvious I'm missing?

[jenshalm]

Hello Andrew and welcome!

Unfortunately what you run into is really a feature not a bug. :-) Laika allows to cross-link to any sections which is based on ids which are derived from the header text and duplicate ids would be illegal in all supported output formats. Manual assignments of ids right in markup is not supported yet, and would require some design thought to support it consistently between different markup languages.

There are a few workarounds I can see right now. One is to not use header syntax for the level where there is repetition, e.g. simple strong markup:

**Pros**

or if that is not sufficient, assigning styles:

@:style(header3) Pros @:@

which is a Laika directive that will give you a header3 class attribute in the rendered HTML (assuming HTML is your target format).

There is also a workaround for actual manual assignments of ids via an extension route, as shown in the next comment.

[jenshalm]

Sorry, I actually forgot to mention the most relevant extension hook for this feature: the slug builder which generates the ids based on the header text and which is pluggable. This might be the most elegant solution for you.

object MyBundle extends ExtensionBundle {
  val description: String = "Assigning custom ids"
  override def slugBuilder: Option[String => String] = Some(headerText => SlugBuilder.default(headerText))
}

The above would be equivalent to the default. I'd recommend to piggy-back on the default builder to ensure you have legal ids and then add custom logic on top.

And then you can specify this extenstion bundle in the usual way:

Transformer
  .from(Markdown)
  .to(HTML)
  .using(MyBundle)
  .build

[andrew-selvia]

Thanks for the explanation @jenshalm! The MyBundle approach unblocked me. I just appended a random integer to headerText as a POC.

[andrew-selvia]

As it turns out, I don't think the bundle will work. slugBuilder is called multiple times for a given document. While I can generate a random or incrementing number to append to each slug, keeping them consistent during subsequent calls to slugBuilder doesn't seem possible since there's no state available to differentiate subsequent calls. This results in no slugs being produced at all.

In case you'd like to reproduce, this is what I've been using:

object Main extends IOApp {
  implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
  def run(args: List[String]): IO[ExitCode] = {
    Blocker[IO].use { blocker =>
      val directoryTransformer = Transformer
        .from(Markdown)
        .to(HTML)
        .using(GitHubFlavor, MyBundle)
        .io(blocker)
        .parallel[IO]
        .build
      directoryTransformer
        .fromDirectory("input")
        .toDirectory("output")
        .transform
    }.as(ExitCode.Success)
  }
}

object MyBundle extends ExtensionBundle {
  val description = "Assigning custom ids"
  private val duplicateHeaderCountsByHeader = mutable.Map[String, Int]()
  override def slugBuilder: Option[String => String] = Some(headerText => {
    val occurrence = duplicateHeaderCountsByHeader.getOrElse(headerText, 0) + 1
    duplicateHeaderCountsByHeader(headerText) = occurrence
    val value = s"$headerText${if (occurrence == 1) "" else s"-$occurrence"}"
    SlugBuilder.default(value)
  })
}

[jenshalm]

Apologies, yes, I forgot to mention that slugBuilder is meant to be idempotent. It would blow up the complexity a lot if it's not so I would not want to go down that route in future Laika versions.

Which means that there are either the other alternatives I showed in my first response. Or, if that is not satisfactory, you can add preliminary support for manually assigning ids in markup, similar to what Laika might support in a future release:

# Comparison

## Option 1

### {pros-1} Pros

### {cons-1} Cons

## Option 2

### {pros-2} Pros

### {cons-2} Cons

To support this manual assignment (the ids in curly braces), surprisingly little effort is required:

object Temp extends IOApp {
  implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)  
  def run(args: List[String]): IO[ExitCode] = {
    Blocker[IO].use { blocker =>
      val directoryTransformer = Transformer
        .from(Markdown)
        .to(HTML)
        .using(GitHubFlavor, ManualIds)
        .io(blocker)
        .parallel[IO]
        .mapDocuments(ManualIds.removeIds)
        .build
      directoryTransformer
        .fromDirectory("input")
        .toDirectory("output")
        .transform
    }.as(ExitCode.Success)
  }
}

object ManualIds extends ExtensionBundle {
  val description = "Processing manual ids from header text"
  
  def hasId(txt: String): Boolean = txt.startsWith("{") && txt.contains("}")
  def extractId (txt: String): String = txt.drop(1).split("}", 2).head
  def removeId (txt: String): String = txt.split("}", 2).last.trim
  
  def removeIds (doc: Document): Document = doc.rewrite(RewriteRules.forBlocks {
    case h: Header => h.content match {
      case Text(txt, _) +: rest if hasId(txt) => 
        Replace(h.copy(content = Text(removeId(txt)) +: rest))
      case _ => 
        Retain
    }  
  })

  override def slugBuilder: Option[String => String] = Some(headerText => {
    val input = if (hasId(headerText)) extractId(headerText) else headerText
    SlugBuilder.default(input)
  })
}

The slugBuilder extracts the id, and the new line mapDocuments(ManualIds.removeIds) removes the id at a later processing step so that it does not show up in the header text.

I tried this with current master which you shouldn't use as it's too much in flux right now for adding theme support, but I'm quite optimistic it'll work with the last release, too.

[jenshalm]

Hello @AndrewSelvia, this ticket has been open for a while, can we close it now?

What you described is expected behaviour, and everything that could be done to improve the situation would be best kept in a new feature ticket. One option would be to support manual ids out of the box, roughly as shown in my previous example, another option would be to make the number of levels with automatic id generation configurable, e.g. for only the first two levels, so that name clashes in level-3 headers don't matter. If you'd like to see any of those, feel free to create corresponding tickets.

[andrew-selvia]

Sorry for leaving it hanging! I've been slowly rewriting articles to avoid duplicate header names. I don't want to lose the auto-generated IDs; that feature is really useful. I intend to utilize the second approach you described (i.e. {pros-1}) for cases where I really do want duplicates. FWIW, supporting that out-of-the-box seems like a good idea. I'll close this for now.