Feature/upgrade cypher grammar#6
Merged
Merged
Conversation
…ase 1) - CypherLexer.flex: 65 new keyword definitions (ACCESS through WRITE) - Cypher.bnf: extend ReservedWord with all new K_* tokens - Cypher.bnf: FunctionName now uses SymbolicNameString (fixes keyword-as-function parsing) - KeywordCaseConverter: add new keywords to TO_LOWER_CASE_SPECIAL set - CypherParameterImplMixin: fix getText() for keyword-named parameters - Regenerate 30 parsing test reference files (.txt)
- Statement now dispatches to SystemCommand before Query - Add MultidatabaseCommand: SHOW/CREATE/DROP/START/STOP DATABASE - Add helper rules: IfExists, IfNotExists, OrReplace - Add statement_recover keywords for admin commands - Add DatabaseCommands parsing test
- Add CreateNodeKeyConstraint / DropNodeKeyConstraint to Command - Add NodeKeyConstraintSyntax rule - Add parsing tests for NODE KEY constraints
- Add UserCommand: SHOW USERS, CREATE/DROP/ALTER USER - Add SetPassword, SetStatus, UserStatus, PasswordStatus rules - Add UserCommands parsing test
- Add RoleCommand: SHOW ROLES, CREATE/DROP ROLE - Support ALL/POPULATED, WITH USERS, AS COPY OF variants - Add RoleCommands parsing test
- Add PrivilegeCommand: SHOW/GRANT/DENY/REVOKE PRIVILEGES - Add privilege scopes: DatabaseScope, GraphScope - Add privilege types: DatabasePrivilege, GraphPrivilege, DbmsPrivilege - Add PropertiesList, Roles, RevokePart rules - Add PrivilegeCommands parsing test
- Add 65 new admin/DDL keywords to CypherKeywords for autocompletion
- Add ShowCommand rule to resolve SHOW keyword ambiguity - Factor out K_SHOW from ShowDatabase/ShowUsers/ShowRoles/ShowPrivileges - Fix ShowRoles parsing (SHOW ROLES WITH USERS) - Fix CreateUser/AlterUser SET PASSWORD/SET STATUS parsing - Regenerate parsing test reference files
- Add K_PRIVILEGE token to lexer and ReservedWord - Change AlterUser pin=1 to pin=3 (prevents future ALTER DATABASE conflict) - SetPassword now accepts Parameter in addition to StringLiteral - DropIndex uses IfExists BNF rule instead of K_IF_EXISTS compound token - Add K_CHANGE to ReservedWord - Remove stray semicolon in QueryOptions - Update DropIndex IF EXISTS test reference (new IfExists PSI node)
- Add OP_AMPERSAND and OP_EXCLAMATION tokens to lexer and BNF - Add LabelExpression rules with OR/AND/NOT/wildcard support - Add SymbolicLabelNameString (excludes K_NOT, K_NULL) for IS form - Replace NodeLabels with LabelExpression in NodePattern, RelationshipDetail, Expression2 - Fix formatter spacing for LabelExpression before Properties - Add testLabelExpressions parsing test (267 tests passing)
- Add CreateConstraint with FOR/REQUIRE, named/unnamed, IF NOT EXISTS - Add DropConstraint with named constraints and IF EXISTS - Add CypherTypeName for type predicates (STRING, INTEGER, BOOLEAN, etc.) - Add :: (coloncolon) operator token to lexer - Wire CreateConstraint and DropConstraint into Command dispatch - Keep legacy ASSERT constraint rules for backward compatibility - Add constraint.cyp test file (268 tests passing)
- Add UseClause rule (USE / USE GRAPH expression) - Add K_USE keyword to lexer, ReservedWord and LabelReservedWord - Wire UseClause into ReadingClause - Add useClause.cyp test file
- Add YieldClause and YieldItem rules for SHOW result filtering - Add SHOW INDEXES, CONSTRAINTS, PROCEDURES, FUNCTIONS, TRANSACTIONS, CURRENT USER - Upgrade existing SHOW DATABASE/USERS/ROLES/PRIVILEGES with YIELD support - Add K_EXECUTABLE, K_TRANSACTION, K_TRANSACTIONS, K_SETTING, K_SETTINGS to lexer - Add showCommands.cyp test file (270 tests passing)
…in the project.
Phase F — Modern expressions.
STEP 1: Upgrade ExistentialSubQuery
Current: ExistentialSubQuery ::= K_EXISTS "{" K_MATCH? PatternWithWhereClause "}"
Change to support both regularQuery and pattern form:
ExistentialSubQuery ::= K_EXISTS "{" (RegularQuery | K_MATCH? PatternWithWhereClause) "}"
The g4 calls this existsExpression: EXISTS LCURLY (regularQuery | matchMode? patternList whereClause?) RCURLY
STEP 2: Upgrade CountSubQuery
Current: CountSubQuery ::= K_COUNT "{" K_MATCH? PatternWithWhereClause "}"
Same change:
CountSubQuery ::= K_COUNT "{" (RegularQuery | K_MATCH? PatternWithWhereClause) "}"
STEP 3: Add CollectExpression
- CollectExpression ::= K_COLLECT "{" RegularQuery "}"
- Add to Expression1 alternatives
- Add K_COLLECT keyword to lexer, ReservedWord and LabelReservedWord
STEP 4: Add type predicates to Expression3
Current Expression3 has: (K_IS K_NULL) | (K_IS K_NOT K_NULL)
Expand to also support:
- K_IS "::" CypherTypeName (IS :: STRING)
- K_IS K_TYPED CypherTypeName (IS TYPED STRING)
- K_IS K_NOT "::" CypherTypeName (IS NOT :: STRING)
- K_IS K_NOT K_TYPED CypherTypeName (IS NOT TYPED STRING)
- "::" CypherTypeName (shorthand :: STRING)
Note: CypherTypeName should already exist from Phase C. If not, add it.
Be careful with ordering: K_IS K_NOT K_NULL must still work — put the
NULL alternatives BEFORE the type predicate alternatives in Expression3.
STEP 5: Create ONE test file modernExpressions.cyp with these cases:
RETURN EXISTS { MATCH (n)-[:KNOWS]->(m) WHERE m.name = 'Alice' }
RETURN EXISTS { (n)-[:KNOWS]->() }
RETURN COUNT { MATCH (n:Person) RETURN n }
RETURN COUNT { (n)-[:KNOWS]->() }
RETURN COLLECT { MATCH (p:Person) RETURN p.name }
MATCH (n) WHERE n.name IS :: STRING RETURN n
MATCH (n) WHERE n.name IS NOT :: STRING RETURN n
MATCH (n) WHERE n IS NULL RETURN n
MATCH (n) WHERE n IS NOT NULL RETURN n
After changes, run .\gradlew generateCypherLexer generateCypherParser build test
Do NOT run any git commands.
- Add Quantifier rule ({n}, {n,m}, +, *)
- Modify PatternElementChain to support optional Quantifier after RelationshipPattern
- Add quantifiedPaths.cyp test file (272 tests passing)
- Add ParenthesizedPath rule (grouped patterns with optional WHERE and quantifier) - Add Selector rule (ANY SHORTEST, ALL SHORTEST, SHORTEST k PATHS GROUP, etc.) - Add K_PATH, K_PATHS, K_GROUP, K_GROUPS, K_SHORTEST keywords to lexer - Update PatternPart to support Selector prefix - Replace PatternElement grouping with ParenthesizedPath - Update integration test reference for Match (LabelExpression PSI change) - Add advancedPaths.cyp test file (273 tests passing)
- Replace simplified privilege rules with full g4 privilege structure - Add Privilege dispatch (allPrivilege, createPrivilege, dropPrivilege, showPrivilege, setPrivilege, removePrivilege, databasePrivilege, dbmsPrivilege, writePrivilege, qualifiedGraphPrivileges) - Add GrantCommand, DenyCommand, RevokeCommand with IMMUTABLE support - Add GraphQualifier, PropertyResource, LabelResource, SettingQualifier, Globs - Add DatabaseScopeList, GraphScopeList with HOME/DEFAULT support - Add role management (GrantRole, RevokeRole, RoleManagementPrivilege) - Add K_IMMUTABLE, K_HOME, K_ALIAS, K_COMPOSITE, K_IMPERSONATE, K_RENAME, K_SERVER, K_SERVERS, K_PASSWORDS to lexer - Enrich privileges test file (273 tests passing)
- Add CreateCompositeDatabase, AlterDatabase with SET ACCESS/TOPOLOGY/OPTION - Add WaitClause (WAIT n SECONDS / NOWAIT) - Add CreateAlias, DropAlias, AlterAlias, ShowAliases - Update DropDatabase with COMPOSITE, DUMP/DESTROY DATA support - Update CreateDatabase with TOPOLOGY and WaitClause - Update StartDatabase, StopDatabase with WaitClause - Add K_TOPOLOGY, K_PRIMARY, K_PRIMARIES, K_SECONDARY, K_SECONDARIES, K_WAIT, K_NOWAIT, K_SEC, K_SECOND, K_SECONDS, K_DUMP, K_DESTROY, K_DATA, K_TARGET, K_DRIVER, K_PROPERTIES, K_ONLY, K_OPTION, K_AT to lexer - Add databaseAdmin.cyp test file (274 tests passing)
- Add SubqueryInTransactionsParameters with batch, error, report options - Support ON ERROR CONTINUE/BREAK/FAIL - Support REPORT STATUS AS variable - Batch parameter now accepts Expression instead of IntegerLiteral - Add K_CONTINUE, K_BREAK, K_FAIL, K_REPORT, K_ROW, K_ERROR to lexer - Add subqueryTransactions.cyp test file (275 tests passing)
- Add EnableServerCommand, AlterServer, DropServer, ShowServers - Add AllocationCommand (DEALLOCATE/REALLOCATE with DRYRUN) - Add TerminateCommand for TERMINATE TRANSACTIONS - Add RenameCommand (RENAME ROLE/USER/SERVER) - Add K_ENABLE, K_DRYRUN, K_DEALLOCATE, K_REALLOCATE to lexer - Add serverAndAdmin.cyp test file (277 tests passing)
- Fix composite constraint property list (REQUIRE (p.name, p.email) IS NODE KEY) - Fix ExistentialSubQuery PEG ordering (pattern form before RegularQuery) - Fix CountSubQuery PEG ordering (same fix)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cypher Grammar Migration — Neo4j 5.x / 2025+ alignment
Summary
Migrates the Cypher BNF grammar from Neo4j ~4.0-4.3 baseline to Neo4j 5.x/2025+
alignment based on the official CypherParser.g4 reference.
Changes (Phases A-K)
Stats
Known issues