Token value comparators

[ericvergnaud]

This is an alternative approach to address contextual tokens.
See antlr/antlr4#1965 for context.
@KvanTTT please feel free to propose a draft spec

[ericvergnaud]

I guess we would only want to support equality operators ? No need for <, <=, > or >= ?

[KvanTTT]

Yes, other operators are not required.

[KvanTTT]

Probably != operator is also can be useful in some cases.

[ftomassetti]

I agree with @KvanTTT : != would be nice to have

[KvanTTT]

On the other hand, we have negation operator ~ that can be combined with token value comparison: ~ID='keyword'. I'll think about that.

[ftomassetti]

To me this seems something easy to learn and very useful in practice. I am not sure if it can solve all problems with contextual tokens (as I do not know enough about the contextual tokens problem), but it seems to me that it can certainly cover many cases with a clear solution. So I think it would be great to have this.

[ericvergnaud]

The operator bit sounds easy enough, but where things might become complicated is with the LHS and the RHS ?
How does one get the tokens being compared ?

[KvanTTT]

The operator bit sounds easy enough, but where things might become complicated is with the LHS and the RHS ?

Could you clarify what you mean by LHS and RHS? Only RHS is needed. Moreover, at the left side label can be used:

propertyAssignment
    : propertyName=ID ':' singleExpression=STRING
    | get=ID=='get' ID '(' ')' functionBody
    ;

How does one get the tokens being compared ?

The following grammar with context tokens:

x
    : ID=='keyword' 'a'
    | ID=='keyword' 'b'
    | ID
    ;

Can be rewritten to ordinary form:

x
    : KEYWORD 'a'
    | KEYWORD 'b'
    | ID
    ;

id : ID | KEYWORD;

ANTLR should resolve context tokens in runtime.

I've implemented basic support, see KvanTTT/antlr4#8

[ericvergnaud]

LGTM!