Add Ripper-based parser and ast printer script for comparison against tree-sitter-ruby output.#210
Add Ripper-based parser and ast printer script for comparison against tree-sitter-ruby output.#210aibaars wants to merge 2 commits intotree-sitter:masterfrom
Conversation
fd50419 to
b95296a
Compare
|
BTW: I consider the https://github.com/whitequark/parser AST as the supreme ruby tooling AST compared to the ripper one, especially as its more stable and better documented. It may make sense to investigate it as an addition/replacement for ripper based tests in the future. |
That's a good point. The reason I picked Ripper is because it is just an external interface to the actual MRI parser. The API and documentation of Ripper are indeed rather terse. I would definitely prefer using |
c314853 to
c3cc057
Compare
c3cc057 to
a0dfea8
Compare
This pull request adds a script named
script/print-ast.rbwhich uses the Ruby::Ripper parserto parse a Ruby file and prints its AST in the same form as tree-sitter. It can be used to
generate expected output for inclusion in test cases.
The PR also adds the script
script/compare-ast.shwhich can be used to compare the AST's producedby
script/print-ast.rbagainst the ones produced bytree-sitter parse. These "back-to-back" comparisonof tree-sitter-ruby against Ruby::Ripper can help find cases where the two parsers diverge, likely indicating
bugs in the tree-sitter-ruby grammar.
The PR also integrates the "back-to-back" tests to the CI job to test for regressions. This makes the CI job quite a bit slower . It now takes 7 minutes to complete instead of 1 minute. However, I think the improved regression testing is worth it.
@maxbrunsfeld @nickrolfe
Checklist: