Fragments do not work as expected

[monrealis]

I'm working on a plantuml grammar and fragments do not work as expected:

This works well:

relationType
:
	(
		'|o'
		| '||'
		| '}o'
		| '}|'
		| '}'
	)
	(
		(
			'-'+
			(
				'left'
				| 'right'
				| 'top'
				| 'down'
			) '-'+
		)
		| '-'+
	)
	(
		'{'
		| 'o{'
		| 'o|'
		| '|{'
		| '||'
	)
	| '-->'
;

This gives line 34:16 no viable alternative at input '-------down' :

relationType
:
	(
		'|o'
		| '||'
		| '}o'
		| '}|'
		| '}'
	)
	(
		(
			'-'+ DIRECTION '-'+
		)
		| '-'+
	)
	(
		'{'
		| 'o{'
		| 'o|'
		| '|{'
		| '||'
	)
	| '-->'
;

fragment
DIRECTION
:
	'left'
	| 'right'
	| 'top'
	| 'down'
;

Line:
A7 }o-------down----------{ B7

[monrealis]

4.9 gives " implicit definition of token DIRECTION in parser", while 4.4 is silent.

[monrealis]

This works too:

RELATION_TYPE
:
	(
		'|o'
		| '||'
		| '}o'
		| '}|'
		| '}'
	)
	(
		(
			'-'+ (DIRECTION) '-'+
		)
		| '-'+
	)
	(
		'{'
		| 'o{'
		| 'o|'
		| '|{'
		| '||'
	)
	| '-->'
;
fragment
DIRECTION
:
	'left'
	| 'right'
	| 'top'
	| 'down'
;

[monrealis]

[KvanTTT]

4.9 gives " implicit definition of token DIRECTION in parser", while 4.4 is silent.

That's just correct behavior because you can't use fragment token inside parse rule. They are only allowed inside lexer rules. Probably the error message a bit confusing.

[monrealis]

I see two minor issues here:

• The behavior changes when you inline fragment inside parser rule. That's not how inlining should work, as understood by most IT people. Is there a reason for that?
• Google says that fragment will never be counted as a token, but the tool says "implicit definition of token DIRECTION in parser"

[KvanTTT]

The behavior changes when you inline fragment inside parser rule. That's not how inlining should work, as understood by most IT people. Is there a reason for that?

There is no common understanding of how inline lexer fragments work in IT because grammars are a specific topic. In ANTLR fragment lexer rules don't exist themselves, they are used within other real tokens for lexer simplification.

Google says that fragment will never be counted as a token, but the tool says "implicit definition of token DIRECTION in parser"

Yes, DIRECTION token does not exist that's why it's being created in an implicit way. But I agree, this error is quite confusing and should be renamed to something like unable to use fragment rule in parser.

[KvanTTT]

Also, it's a duplicate of #2886

[KvanTTT]

@parrt @ericvergnaud please close since it's duplicate.