ArrayIndexOutOfBounds Exception when the input expression is greater than 8192 Characters

[ironhide0396]

Changing maxNextCharIndex = DEFAULT_BUFF_SIZE (currently it is 4096 ) has made it work .
The change is made in expandBuff() of AbstractCharStream class
I am using the following dependency

          `.<groupId>com.helger.maven</groupId>
            <artifactId>ph-javacc-maven-plugin</artifactId>
            <version>4.1.4</version>`

[phax]

A little more input would be appreciated. What version are you using? What configuration are you using?

[ironhide0396]

@phax Any update on this issue ?

[jhargraveiii]

I'm having a similar issue. Any input that has a token size > buffer will crash. in our case this is json values that hold large embedded HTML snippets. Is there an option for this?

Using the maven plugin 4.1.4 as above

options for json parser
{
SUPPORT_CLASS_VISIBILITY_PUBLIC=true;
UNICODE_INPUT=true;
JAVA_UNICODE_ESCAPE=true;
CHOICE_AMBIGUITY_CHECK = 3;
OTHER_AMBIGUITY_CHECK = 2;
ERROR_REPORTING = true;
SANITY_CHECK = true;
KEEP_LINE_COLUMN = true;
JAVA_TEMPLATE_TYPE="modern";
//DEBUG_PARSER = true;
//DEBUG_LOOKAHEAD = true;
//DEBUG_TOKEN_MANAGER = true;
}

ava.lang.ArrayIndexOutOfBoundsException: 4096

at net.sf.okapi.filters.json.parser.JavaCharStream.readByte(JavaCharStream.java:69)
at net.sf.okapi.filters.json.parser.JavaCharStream.readChar(JavaCharStream.java:112)
at net.sf.okapi.filters.json.parser.JsonParserTokenManager.jjMoveNfa_0(JsonParserTokenManager.java:755)
at net.sf.okapi.filters.json.parser.JsonParserTokenManager.jjStartNfa_0(JsonParserTokenManager.java:60)
at net.sf.okapi.filters.json.parser.JsonParserTokenManager.jjMoveStringLiteralDfa1_0(JsonParserTokenManager.java:132)
at net.sf.okapi.filters.json.parser.JsonParserTokenManager.jjMoveStringLiteralDfa0_0(JsonParserTokenManager.java:72)
at net.sf.okapi.filters.json.parser.JsonParserTokenManager.getNextToken(JsonParserTokenManager.java:1058)
at net.sf.okapi.filters.json.parser.JsonParser.getNextToken(JsonParser.java:550)

[phax]

@jhargraveiii can you please PM me the grammar so I can debug into it? See the pom.xml for my email address. Thx

[jhargraveiii]

@phax any chance this can be included in the 1.1.4 release? If you are low on time I could attempt a PR. Assuming its an easy fix.

[phax]

@jhargraveiii go ahead - a PR is always welcome :)

[jhargraveiii]

I've tracked down the issue to "JAVA_UNICODE_ESCAPE=true;". JavaCharStream isn't allocating enough buffer to accommodate large tokens. CharStream works without issue. I'll dig some more.

[phax]

@jhargraveiii see also #26 for some infos

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.