Skip to content

[Refactor] Query Parser Fix After AST Support#100

Open
HazelYuAhiru wants to merge 2 commits intomainfrom
parser_fix
Open

[Refactor] Query Parser Fix After AST Support#100
HazelYuAhiru wants to merge 2 commits intomainfrom
parser_fix

Conversation

@HazelYuAhiru
Copy link
Collaborator

@HazelYuAhiru HazelYuAhiru commented Mar 17, 2026

Overview

This PR updates the query parser logic to support more complex test cases. All existing test cases now pass, except for the two listed in the following section.

Remaining Questions / TODOs

  • test_basic_parse: Should we treat d.name in the GROUP BY clause as a new node, instead of using the alias dept_name based on our previous discussion?
  • test_query_42: The original query is -EXTRACT(DOW FROM created_at). Is OperatorNode(LiteralNode(0), "-", extract_dow) (becomes 0 - EXTRACT(DOW FROM created_at)) an acceptable translation? (I assume no)
  • Support UNION

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the SQL-to-AST query parser to handle a broader set of mo_sql_parsing constructs (e.g., DISTINCT/DISTINCT ON, IN literal lists, CASE, INTERVAL/EXTRACT, NULL/IS NULL), and updates the test suite to re-enable many previously disabled parser/AST equality assertions.

Changes:

  • Expanded core/query_parser.py to parse additional SQL constructs into the project’s AST node types (e.g., ListNode, CaseNode, IntervalNode, DataTypeNode, DISTINCT ON support).
  • Adjusted alias handling across clauses (notably GROUP BY / WHERE comments and resolution behavior).
  • Re-enabled many tests/test_query_parser.py assertions, while leaving test_basic_parse and test_query_42 effectively disabled pending design decisions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
tests/test_query_parser.py Re-enables most parser-vs-expected-AST assertions, but leaves two key tests disabled via commented asserts.
core/query_parser.py Adds parsing support for DISTINCT(+ON), IN lists/subqueries, NULL/IS NULL, CASE, INTERVAL, EXTRACT, and tweaks alias resolution logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors and extends the SQL-to-AST query parser to support more complex query patterns emitted by mo_sql_parsing, with the goal of turning previously-disabled parser tests back on.

Changes:

  • Expanded QueryParser to handle additional SQL constructs (e.g., DISTINCT / DISTINCT ON, IN literal lists, INTERVAL, EXTRACT, CASE, NULL normalization).
  • Adjusted alias resolution behavior across clauses (notably GROUP BY, ORDER BY, and WHERE) to better match expected AST semantics.
  • Re-enabled many previously-commented test assertions (while leaving a couple still disabled).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
core/query_parser.py Adds parsing support for more AST node types and improves clause parsing/alias resolution to satisfy more real-world queries.
tests/test_query_parser.py Re-enables many parser equality assertions; still has two key tests with assertions commented out.
data/asts.py Minor comment/TODO update on GROUP BY ordinal handling in Query 42 expected AST.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants