What's the best way to parse/process an org.w3c.dom.Element in 2.2.0?

[mariussoutier]

Hello,

I'm using Akka to parse XML from a streaming response, and in particular XmlParsing.subtree() to process subtrees. subtree() returns an org.w3c.dom.Element. Up until ScalaXML 2.1.0, the following was working perfectly:

def processElement(element: org.w3c.dom.Element): Node = {
  import org.apache.xalan.xsltc.trax.DOM2SAX
  val dom2sax = new DOM2SAX(element)
  val adapter = new NoBindingFactoryAdapter
  dom2sax.setContentHandler(adapter)
  dom2sax.parse()
  myAdapter.rootElem
}

However in ScalaXML 2.2.0 a couple of change have broken this approach. Especially the endDocument() method now tries to access the current XML reader, which doesn't exist in my approach. I've also tried setting the XML reader (DOM2SAX is one) manually, but to no avail. I guess my approach is not the intended use of ScalaXML/FactoryAdapter.

Is also don't understand why the FactoryAdapter is always using its private var xmlReader: Option[XMLReader] (FactoryAdapter:225) and not falling back to the overridable def reader: XMLReader (XMLLoader:51).

Is there another way to do this?

[lrytz]

@mariussoutier can you guess which pull request could have introduced this change? See

• https://github.com/scala/scala-xml/releases/tag/v2.2.0
• https://github.com/scala/scala-xml/commits/main/shared/src/main/scala/scala/xml/parsing/FactoryAdapter.scala
• v2.1.0...v2.2.0

[mariussoutier]

There you go:
0203fae#diff-7606b3470156706015f32943b67ea2ffa4086a0315c2ba45bc5ea7fc2db0fbaeR225-R239

[dubinsky]

@mariussoutier please try replacing

val adapter = new scala.xml.parsing.NoBindingFactoryAdapter
dom2sax.setContentHandler(adapter)
dom2sax.parse()
adapter.rootElem

with

scala.xml.XML.withXMLReader(dom2sax).load("unused")

[mariussoutier]

This exposes the same issue:

None.get
java.util.NoSuchElementException: None.get
	at scala.None$.get(Option.scala:627)
	at scala.None$.get(Option.scala:626)
	at scala.xml.parsing.FactoryAdapter.loadDocument(FactoryAdapter.scala:143)
	at scala.xml.factory.XMLLoader.loadDocument(XMLLoader.scala:65)
	at scala.xml.factory.XMLLoader.loadDocument$(XMLLoader.scala:65)
	at scala.xml.XML$$anon$2.loadDocument(XML.scala:76)
	at scala.xml.factory.XMLLoader.loadDocument(XMLLoader.scala:69)
	at scala.xml.factory.XMLLoader.loadDocument$(XMLLoader.scala:69)
	at scala.xml.XML$$anon$2.loadDocument(XML.scala:76)
	at scala.xml.factory.XMLLoader.load(XMLLoader.scala:81)
	at scala.xml.factory.XMLLoader.load$(XMLLoader.scala:81)
	at scala.xml.XML$$anon$2.load(XML.scala:76)

[mariussoutier]

The issue being FactoryAdapter not using the overridden reader.

[dubinsky]

@mariussoutier

The issue being FactoryAdapter not using the overridden reader.

I do not think this is the issue.
Currently, FactoryAdapter extends XMLLoader - but it shouldn't: FactoryAdapter does not override nor use any methods from XMLLoader; fixing this requires breaking binary compatibility, so the problematic inheritance remains for now...

The overridden reader is used in XMLLoader, and is passed into the FactoryAdapter:

def loadDocument(inputSource: InputSource): Document = adapter.loadDocument(inputSource, reader)

where it is configured, stashed into private var xmlReader: Option[XMLReader], and its xmlReader.parse(inputSource) method is called.

The issue that you are seeing seems to be caused by the following:

• FactoryAdapter.loadDocument() returns document.get
• FactoryAdapter.document is set in the FactoryAdapter.endDocument() method
• endDocument() must be called by any SAX parser
• DOM2SAX.parse() correctly adds the calls to startDocument() and andDocument() if the root of the DOM is an element and not a document
• DOM2SAX.parse(inputSource) does NOT call endDocument() if the root of the DOM is an element and not a document

This looks like a bug in Xalan to me: DOM2SAX.parse(inputSource) should just call DOM2SAX.parse() instead of doing what it is doing currently. Scala XML should probably relax its assumptions and accommodate non-compliant SAX parsers better.

This Xalan bug can be worked around by overriding one method on the DOM2SAX:

class DOM2SAXFixed(root: org.w3c.dom.Node) extends DOM2SAX(root) {
  override def parse(unused: InputSource): Unit = parse()
}

  ...
  scala.xml.XML.withXMLReader(new DOM2SAXFixed(element)).load("unused")

[dubinsky]

@mariussoutier did you have a chance to verify that the above approach works for you? Thanks!

[mariussoutier]

Tests have passed, thanks! Looks a bit weird though ;-)

    val dom2sax = new DOM2SAX(element) {
      override def parse(unused: InputSource): Unit = parse()
    }
    scala.xml.XML.withXMLReader(dom2sax).load("unused")

[dubinsky]

Tests have passed, thanks!

Glad it worked!

Looks a bit weird though ;-)

There are two things that look weird: one is the DOM2SAX subclass, which will go away if and when Xalan team accepts and ships the fix for the issue (apache/xalan-java#83); the other one - the need to supply a fake systemId "unused" - is needed because the XMLReader's parse() method requires an InputSource...

[mariussoutier]

Yep, then let's hope they'll accept your PR.

[dubinsky]

Yep, then let's hope they'll accept your PR.

They did! With the next release of Xalan you will be able to remove a bit of weirdness from the code ;)