948 Reflected the comments of the CG on the specification of scan-left and scan-right

[dnovatchev]

Reflected the comments of the CG on the specification of scan-left and scan-right

[Arithmeticus]

The CG also discussed the inconsistency between the summary descriptions of fold/scan-left/right. It was noted that these summaries should be communicative, and not pure definitions. I don't claim to have solved the problem but I would like to put on the table the following summary templates:

fn:fold-left: Recursively applies a function to a sequence of items, left to right, returning only the final result.
fn:scan-left: Recursively applies a function to a sequence of items, left to right, returning the accumulation of all results, intermediate and final.

These versions are a bit shorter and I think clearer than what currently exists.

Versions for right would switch to "right to left".

[michaelhkay]

fn:fold-left: Recursively applies a function to a sequence of items, left to right, returning only the final result.
fn:scan-left: Recursively applies a function to a sequence of items, left to right, returning the accumulation of all results, intermediate and final.

Definitely better than what's there at the moment, but I think an improvement might be

fn:fold-left: Applies a function to each of a sequence of items, left to right, returning an accumulated result.
fn:scan-left: Applies a function to each of a sequence of items, left to right, returning a sequence of accumulated results.

[ChristianGruen]

Related (#864 (comment)), I wonder if the current notes could be rewritten as well. I have a hard time to understand the following sentence:

[…] The accumulated result is initially set to the value of the $zero argument, which is conventionally a value […] that causes the function to return the value of the other argument unchanged.

[ChristianGruen]

I’m adding a reference to Joel’s comment in the original issue. It contains some more observations that I think should still be addressed (such as the unification of the two pseudo-codes; using the same variable names as in the function signature; etc.): #957 (comment).

And a minor typo: A space is missing twice in “XPath(return”.

[dnovatchev]

I’m adding a reference to Joel’s comment in the original issue. It contains some more observations that I think should still be addressed (such as the unification of the two pseudo-codes; using the same variable names as in the function signature; etc.): #957 (comment).

It is beyond the scope of the work on scan-left/right functions to modify the pseudocode of the corresponding fold-left/right functions. We need a separate task/action for this.

I will take care of using the same variable names...

And a minor typo: A space is missing twice in “XPath(return”.

and of the missing spaces.

[ChristianGruen]

It is beyond the scope of the work on scan-left/right functions to modify the pseudocode of the corresponding fold-left/right functions. We need a separate task/action for this.

That's the comment I referred to, which I agree with:

The fourth parameter $self adds a layer of indirection that is unnecessarily obfuscating; the code for fn:fold-left, where fold-left-helper refers to itself by name, is clearer. I realize that this a conscious choice, to define the function exclusively in terms of XPath, and not of a host language. But fn:fold-left's presentation in XQuery more effectively communicates an abstract concept.

[dnovatchev]

It is beyond the scope of the work on scan-left/right functions to modify the pseudocode of the corresponding fold-left/right functions. We need a separate task/action for this.

That's the comment I referred to, which I agree with:

The fourth parameter $self adds a layer of indirection that is unnecessarily obfuscating; the code for fn:fold-left, where fold-left-helper refers to itself by name, is clearer. I realize that this a conscious choice, to define the function exclusively in terms of XPath, and not of a host language. But fn:fold-left's presentation in XQuery more effectively communicates an abstract concept.

We agree to disagree.

The single, pure XPath runnable code has important advantages over providing two different pseudocode instances - one for XQuery and one for XSLT.

I will open a separate issue that we need to improve this situation and have for the pseudo-code of all functions just one, XPath implementation provided.

[dnovatchev]

Related (#864 (comment)), I wonder if the current notes could be rewritten as well. I have a hard time to understand the following sentence:

[…] The accumulated result is initially set to the value of the $zero argument, which is conventionally a value […] that causes the function to return the value of the other argument unchanged.

There is no such text in the Notes of the two functions.

Probably this was referring to something outside of the scope of the current text?

[dnovatchev]

It is beyond the scope of the work on scan-left/right functions to modify the pseudocode of the corresponding fold-left/right functions. We need a separate task/action for this.

That's the comment I referred to, which I agree with:

The fourth parameter $self adds a layer of indirection that is unnecessarily obfuscating; the code for fn:fold-left, where fold-left-helper refers to itself by name, is clearer. I realize that this a conscious choice, to define the function exclusively in terms of XPath, and not of a host language. But fn:fold-left's presentation in XQuery more effectively communicates an abstract concept.

@Arithmeticus ,

1. We all already registered the fact that the specifications of both fold-left and fold-right are wrong in their current form - the problem of specifying the $action parameter which can be a function with different arities.

2. Also, these two functions at present have their semantics (rules) specified only in XQuery. Any XSLT implementation will therefore be dependent on how the implementors interpret this XQuery-specified semantics. Even if an additional XSLT-based semantics is added, then we are facing one new, additional and gross problem: How do we know that the two proposed semantics are actually equivalent? We do not, unless there is a formal proof. Let us face and recognize this as a significant problem: the implementations based on two different semantics could in fact do different things.

3. Both of this issues simply don't exist when a single, executable XPath semantics is provided.

We are lucky that a common expression language exists for both XSLT and XQuery and this allows us to provide a single semantics (even a runnable one!) that totally fulfills its only intended goal: to be the Oracle for right and wrong results.

But fn:fold-left's presentation in XQuery more effectively communicates an abstract concept.

If we saw that the current specification for the fold-left and fold-right is wrong (see 1. above) and taking into account all items 1, 2, and 3 (above), then how can this be an argument?

[ChristianGruen]

There is no such text in the Notes of the two functions.

Probably this was referring to something outside of the scope of the current text?

Exactly. See #864 (comment)

[dnovatchev]

There is no such text in the Notes of the two functions.
Probably this was referring to something outside of the scope of the current text?

Exactly. See #864 (comment)

Then this comment is irrelevant for this PR.

[ChristianGruen]

Then this comment is irrelevant for this PR.

I didn’t expect you to feel addressed, and to take immediate action; it was a reply to the suggestions made in #978 (comment) and the subsequent comment.

[Arithmeticus]

@dnovatchev

For your item 1, I look forward to seeing the consensus the CG reaches (#981). For item 2, I think you're going to raise a separate issue, which I think is appropriate, which I look forward to following. I think your item 3 is related to item 2 not 1 (a rule expressed in XPath can be equally flawed). I think you will need to argue, and convince others, that in the specs, when the rules of any XPath function is expressed in terms of code, it should be in terms exclusively of XPath, never XQuery and XSLT. Not that I am unsympathetic. The first time I encountered an XPath function defined in terms of XSLT and XQuery I thought it strange that in places XPath should be defined and therefore dependent upon host languages that themselves depend on XPath.

If we saw that the current specification for the fold-left and fold-right is wrong (see 1. above) and taking into account all items 1, 2, and 3 (above), then how can this be an argument?

I'm not offering an argument here, only observing that the XPath approach entails an extra layer of indirection, and lacks the clarity of the XQuery definition. If the CG comes to the consensus that XPath purity overrides clarity, then great, I'll go with that. If they go the other way, I'll go with that too. I would be more frustrated with inconsistency, which is what we currently have.

[dnovatchev]

I'm not offering an argument here, only observing that the XPath approach entails an extra layer of indirection, and lacks the clarity of the XQuery definition. If the CG comes to the consensus that XPath purity overrides clarity, then great, I'll go with that. If they go the other way, I'll go with that too.

I would be more frustrated with inconsistency, which is what we currently have.

@Arithmeticus ,

I would never be consistent with something that we all saw was wrong and that I consider truly sub-optimal (in terms of precise and correct expressing), and both missing and redundant.

I think that the fold-function specifications must be made consistent with the scan-functions ones - not the other way around.

[ChristianGruen]

1. We all already registered the fact that the specifications of both fold-left and fold-right are wrong in their current form - the problem of specifying the $action parameter which can be a function with different arities.

@dnovatchev Please avoid suggestive language, or (in the given case) define “we all”. The current function coercions rules have been jointly discussed and approved in our previous meetings; there have already been various GitHub issues referring to these rules in the meanwhile; and at least two implementations exist at the moment (namely, Saxon and BaseX) that successfully invoke functions with a lower arity following exactly these rules.

2. Also, these two functions at present have their semantics (rules) specified only in XQuery. Any XSLT implementation will therefore be dependent on how the implementors interpret this XQuery-specified semantics.

As you take the implementor view: The presented code could be written in any other language or pseudocode. I as an XQuery implementor wouldn’t care if it was expressed in XSLT as long as it’s concise, complete, and consistent. Pseudocode should generally be simple, cover all edge cases, and steer clear of syntactical intricacies.

3. Both of this issues simply don't exist when a single, executable XPath semantics is provided.

For the implementor, this doesn’t solve anything: The implementation of a feature can differ a lot from the presented pseudocode. In particular, for folds, a low-level implementation (either iterative or recursive) looks completely different from the XQuery or XPath solutions. In the given case, XQuery is a bit closer, because there’s no need to explicitly pass along a reference to itself.

For the user of the language, the choice for the best representation depends a lot on your preknowledge and personal preferences.

I’ve created #1000 for further discussion on this topic.

[ndw]

The CG agreed to accept this issue at meeting 065.