Consider replacing javacc with existing Antlr 4 grammar

[timboudreau]

Antlr 4 is considerably more powerful than javacc, and javacc's future is a bit less certain these days; a Rust grammar is available that could probably just be picked up and used, but it would mean some rework.

More beneficially, it would allow for considerably more sophisticated syntax highlighting, such as distinguishing macro uses and more.

[timboudreau]

Some work in progress on that

[Chris2011]

Wow, great @timboudreau :) Nice to see someone works on the project.

[Chris2011]

@drrb what is your status atm moment? Do you still work on the project too?

[Chris2011]

I wanted to change the grammar to use the textmate rust file. @timboudreau can antlr and textmate coexist? I mean using the rust textmate for syntax highlighting and the antlr for everything else or doesn't make it sense? Maybe we can change to textmate and using the language server for the rest.

[timboudreau]

I kind of forked off from working on this to build a set of modules for supporting Antlr languages in NetBeans in general - i.e. define syntax highlighting and a few other things as an annotation that references the Antlr-generated lexer and parser, and the annotation processor generates 90% of the module including navigator panels and a bunch of other stuff - if you add some analysis stuff (also mostly annotations), you get things like finding references.

So, eventually I plan to get back to Rust+Antlr, building on that - I found one Antlr grammar that sort of worked but was kind of insanely written, and have been using that as the basis for a replacement.

[Chris2011]

Hey @timboudreau what is the state of your rewrite process? I see that the last commit of you was also back in 2019. Maybe the grammar ist better nowadays? Just guessing. Found this just now: https://github.com/rrevenantt/antlr4rust

[timboudreau]

Funny, I just wrote a post about that a few minutes before seeing this on the netbeans-dev mailing list. I have a better Antlr grammar than that one in the works - lexer modes are really useful for handling some of the more painful corners such as macro_rules!. Will likely share it on Github in the next couple of weeks.

Digging into this project led me off on somewhat of a tangent into building generic Antlr language support (a few annotations to generate all of the antlr-to-netbeans parser and lexer glue code: https://github.com/timboudreau/ANTLR4-Plugins-for-NetBeans and only circled back to Rust support specifically around the time I needed to get a job, so it's been slow going since then).

[Chris2011]

Great to hear :). Thx for your investigation.