74472

ANTLR4 DefaultErrorStrategy fails to inject missing token

Question:

I'm trying to run in TestRig the following grammar:

grammar COBOLfragment; // hidden tokens WS : [ ]+ -> channel(HIDDEN); NL : '\n' -> channel(HIDDEN); // keywords PERIOD : '.'; DIVISION : 'DIVISION'; SECTION : 'SECTION'; DATA : 'DATA'; WORKING_STORAGE : 'WORKING-STORAGE'; FILE : 'FILE'; FD : 'FD'; EXTERNAL : 'EXTERNAL'; GLOBAL : 'GLOBAL'; BLOCK : 'BLOCK'; CONTAINS : 'CONTAINS'; CHARACTERS : 'CHARACTERS'; // data INTEGER : [0-9]+; ID : [A-Z][A-Z0-9]*; dataDivision : DATA DIVISION PERIOD fileSection? workingStorageSection? ; fileSection : FILE SECTION PERIOD fileDescription* ; fileDescription : FD fileName=ID // (IS? GLOBAL)? // 1. IS GLOBAL clause // (IS? EXTERNAL)? // 2. IS EXTERNAL clause blockClause? PERIOD ; blockClause : BLOCK CONTAINS? blockSize=INTEGER CHARACTERS ; workingStorageSection : WORKING_STORAGE SECTION PERIOD ;

with the following input:

DATA DIVISION. FILE SECTION. FD FD01 WORKING-STORAGE SECTION.

Clearly the third line of input ("FD FD01") is missing the terminator PERIOD asked for in fileDescription rule.

The DefaultErrorStrategy correctly acknowledges this and conjures up the missing token:

<img alt="parse tree with missing token correcly injected" class="b-lazy" data-src="https://i.stack.imgur.com/ulfve.png" data-original="https://i.stack.imgur.com/ulfve.png" src="https://etrip.eimg.top/images/2019/05/07/timg.gif" />

On stderr the correct report is displayed: line 4:0 missing '.' at 'WORKING-STORAGE'.

But if the fragments commented out are enabled (that is, the clauses 'IS EXTERNAL' and 'IS GLOBAL' are brought in the grammar again), then single token insertion fails:

<img alt="parse tree missing the missing token" class="b-lazy" data-src="https://i.stack.imgur.com/b8mcC.png" data-original="https://i.stack.imgur.com/b8mcC.png" src="https://etrip.eimg.top/images/2019/05/07/timg.gif" />

On stderr the misleading report is displayed: line 4:0 no viable alternative at input 'WORKING-STORAGE'

<strong>How to enable the full grammar (with IS EXTERNAL and IS GLOBAL clauses) retaining the ability to correct the missing PERIOD?</strong>

Side note 1: if I enable either IS EXTERNAL or IS GLOBAL, but not both clauses, then the DefaultErrorStrategy works nicely and injects the missing token.

Side note 2: the code generated for a grammar with both clauses enabled has the following extra code (compared to a grammar with just one of them enabled):

public final FileDescriptionContext fileDescription() ... { ... try { ... switch ( getInterpreter().adaptivePredict(_input,4,_ctx) ) { case 1: { setState(31); _la = _input.LA(1); if (_la==IS) { { setState(30); match(IS); } } setState(33); match(GLOBAL); } break; } ... } catch (RecognitionException re) { ...

And the adaptivePredict() call is the culprit, because it throws no viable alternative at input 'WORKING-STORAGE' before the parser has a chance to match(PERIOD) (in the generated code, not pasted here).

Answer1:

I've managed to solve it adding a new clause for both IS clauses:

(here just the fragments changed)

... fileDescription : FD fileName=ID isClauses? blockClause? PERIOD ; isClauses : IS? GLOBAL (IS? EXTERNAL)? | IS? EXTERNAL ; ...

Now the DefaultErrorStrategy does its work and injects the missing PERIOD.

<strong>Why not isClauses : (IS? GLOBAL?) (IS? EXTERNAL)?;</strong> Well, I tried that first, of course. But got a warning (warning(154): rule 'fileDescription' contains an optional block with at least one alternative that can match an empty string) and no missing PERIOD injected.

Recommend

  • Cannot connect to beagle bone black
  • Moving through toggle buttons using arrow keys
  • Why am I getting EXC_BAD_ACCESS with these c++ functions?
  • Grab the frame from gst pipeline to opencv with python
  • Troubles using jquery get to retrieve data from the server and place in a div
  • Adding javascript function to jMeter using jsr223
  • Rijndael managed: plaintext length detction
  • How to check the text of an NSTextField (Label)
  • How to enum 32bit process's modules from a 64bit application
  • how do disable disk cache in c# invoke win32 CreateFile api with FILE_FLAG_NO_BUFFERING
  • Need code translation from VB to C#
  • Error processing multiple files
  • How can I run DataNucleus Bytecode Enhancer from SBT?
  • pip in virtualenv gets ConnectTimeoutError
  • How do I shift the decimal place in Python?
  • Floated image with variable width and heading with background image
  • Plotting line graph with factors in R
  • pillow imaging ImportError
  • I18n locale disregarding fallbacks
  • close() was never explicitly called on database
  • Z3: Convert between FP and BitVector?
  • Date Conversion from yyyy-mm-dd to dd-mm-yyyy
  • Groovy: Unexpected token “:”
  • Do I need to reset a Perl hash index?
  • Magento Fatal error: Maximum execution error solution, on WAMP
  • Word Open XML Mail Merge
  • Bad request using file_get_contents for PUT request in PHP
  • Python CGI os.system causing malformed header
  • Spring Data JPA custom method causing PropertyReferenceException
  • Java Scanner input dilemma. Automatically inputs without allowing user to type
  • FB SDK and cURL: Unknown SSL protocol error in connection to graph.facebook.com:443
  • Using $this when not in object context
  • How do I fake an specific browser client when using Java's Net library?
  • Counter field in MS Access, how to generate?
  • How reduce the height of an mschart by breaking up the y-axis
  • Perl system calls when running as another user using sudo
  • Convert array of 8 bytes to signed long in C++
  • what is the difference between the asp.net mvc application and asp.net web application
  • Matrix multiplication with MKL
  • Binding checkboxes to object values in AngularJs