Convert my .g4 to .flex and .bnf for IDEA syntax highlighter

Question

Convert my .g4 to .flex and .bnf for IDEA syntax highlighter

1.3k views Asked by michaelsnowden At 14 September 2024 at 14:52

I have never used JFlex before, and I have no idea how it works. Basically, I've built a runtime for a scheme-esque language in Java, and the parser I have for it was generated using Antlr 4, so I have a .g4 file, which looks like this:

program: expression EOF;
expression: lambdaAbstraction | application| literal;

lambdaAbstraction: LP WS? LAMBDA (WS IDENTIFIER)* WS expression WS? RP;
application: LP WS? expression (WS expression)* WS? RP;
literal: FLOAT_LITERAL | INTEGER_LITERAL | IDENTIFIER | STRING_LITERAL;

STRING_LITERAL: '"' ~["]* '"' | '\'' ~[']* '\'' ;
FLOAT_LITERAL: [0-9]+ '.' [0-9]+;
INTEGER_LITERAL: [0-9]+;
IDENTIFIER: [a-zA-Z+/*-=?@]+;
LAMBDA: '\\';
LP: '(';
RP: ')';
WS: [ \n\r\t]+;

Some examples (so you can get a feel for the grammar).

(+ 1 2) = 3
((\ a b (+ a b)) 1 2) = 3
(+ "Hello, " "World") = "Hello, World"

What I'm trying to do is build a syntax-highlighter plugin for IntelliJ-IDEA for this language using this guide: http://www.jetbrains.org/intellij/sdk/docs/tutorials/custom_language_support_tutorial.html.

Here is my .flex file

package com.michaelsnowden.yalcil_plugin;

import com.intellij.lexer.FlexLexer;
import com.intellij.psi.tree.IElementType;
import com.michaelsnowden.yalcil_plugin.psi.YalcilTypes;
import com.intellij.psi.TokenType;

%%

%class YalcilLexer
%implements FlexLexer
%unicode
%function advance
%type IElementType
%eof{  return;
%eof}

STRING_LITERAL= "\"" [^\"&]* "\"";
FLOAT_LITERAL=[0-9]+ '.' [0-9]+;
INTEGER_LITERAL=[0-9]+;
IDENTIFIER=[a-zA-Z+/*-=?@]+;
LAMBDA="\\";
LP="(";
RP=")";
WS=[ \n\r\t]+;
%%

<YYINITIAL> {
    {LAMBDA} { yybegin(YYINITIAL); return YalcilTypes.LAMBDA; }
    {FLOAT_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.LAMBDA; }
    {INTEGER_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.INTEGER_LITERAL; }
    {IDENTIFIER} { yybegin(YYINITIAL); return YalcilTypes.IDENTIFIER; }
    {LP} { yybegin(YYINITIAL); return YalcilTypes.LP; }
    {RP} { yybegin(YYINITIAL); return YalcilTypes.RP; }
    {WS} { yybegin(YYINITIAL); return YalcilTypes.WS; }
    {STRING_LITERAL} { yybegin(YYINITIAL); return YalcilTypes.STRING_LITERAL; }
}

. { return TokenType.BAD_CHARACTER; }

And here is my .bnf file

{
  parserClass="com.michaelsnowden.yalcil_plugin.YalcilParser"

  extends="com.intellij.extapi.psi.ASTWrapperPsiElement"

  psiClassPrefix="Yalcil"
  psiImplClassSuffix="Impl"
  psiPackage="com.michaelsnowden.yalcil_plugin.psi"
  psiImplPackage="com.michaelsnowden.yalcil_plugin.psi.impl"

  elementTypeHolderClass="com.michaelsnowden.yalcil_plugin.psi.YalcilTypes"
  elementTypeClass="com.michaelsnowden.yalcil_plugin.psi.YalcilElementType"
  tokenTypeClass="com.michaelsnowden.yalcil_plugin.psi.YalcilTokenType"
}

program ::= expression;
expression ::= lambdaAbstraction | application| literal;
lambdaAbstraction ::= LP WS? LAMBDA (WS IDENTIFIER)* WS expression WS? RP;
application ::= LP WS? expression (WS expression)* WS? RP;
literal ::= FLOAT_LITERAL | INTEGER_LITERAL | IDENTIFIER | STRING_LITERAL;

I get basically nothing but errors when I run this.