Skip to content

fix(bigquery): support @@ system variables in lexer#2529

Open
bermi wants to merge 2 commits into
quarylabs:mainfrom
bermi:fix/bigquery-system-variable-line-break
Open

fix(bigquery): support @@ system variables in lexer#2529
bermi wants to merge 2 commits into
quarylabs:mainfrom
bermi:fix/bigquery-system-variable-line-break

Conversation

@bermi
Copy link
Copy Markdown
Contributor

@bermi bermi commented Apr 9, 2026

BigQuery supports both @param and @@system_variable syntax. The lexer regex only matched single @ prefixes, causing @@var to be split into two tokens. This produced unwanted line breaks during formatting (e.g. SET foo = \n@@bar;) and parse errors in CALL statements using system variables like @@error.message.

Widen the at_sign_literal regex from @[a-zA-Z_][\w]* to @{1,2}[a-zA-Z_][\w.]* to match both forms in a single token, with dot support for qualified names like @@error.message.

@benfdking
Copy link
Copy Markdown
Collaborator

Hey 👋

Thank you for the contribution! While this is correct, it would be better to distinguish between at_sign_literal and double_at_sign_literal, which have very different meanings.

In BigQuery, @param is a query parameter while @@error.message is a system variable — these are semantically distinct and SQLFluff treats them as separate token types.

SQLFluff introduced this in sqlfluff/sqlfluff#5307, where they define two separate lexer matchers:

RegexLexer(
    "at_sign_literal",
    r"@[a-zA-Z_][\w]*",
    ...
),
RegexLexer(
    "double_at_sign_literal",
    r"@@[a-zA-Z_][\w\.]*",
    ...
),

Along with a dedicated SystemVariableSegment that wraps the DoubleAtSignLiteralSegment, giving @@ variables their own node type in the AST.

It would be great to follow this pattern here — adding a separate DoubleAtSignLiteral syntax kind rather than combining both into a single @{1,2} regex. This keeps the token types semantically meaningful for downstream rules and analysis.

bermi added 2 commits April 16, 2026 17:28
BigQuery supports both @param and @@system_variable syntax. The lexer
regex only matched single @ prefixes, causing @@var to be split into
two tokens. This produced unwanted line breaks during formatting
(e.g. `SET foo = \n@@bar;`) and parse errors in CALL statements using
system variables like @@error.message.

Widen the at_sign_literal regex from `@[a-zA-Z_][\w]*` to
`@{1,2}[a-zA-Z_][\w.]*` to match both forms in a single token, with
dot support for qualified names like @@error.message.
@bermi bermi force-pushed the fix/bigquery-system-variable-line-break branch from 5bdca04 to 16c8e01 Compare April 17, 2026 10:46
@bermi
Copy link
Copy Markdown
Contributor Author

bermi commented Apr 17, 2026

@benfdking thanks for the feedback. Let me know if I got the SystemVariableSegment right or if we need to handle it differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants