feat(dbapi): wire timeout parameter through Connection to execute_sql#1535
feat(dbapi): wire timeout parameter through Connection to execute_sql#1535waiho-gumloop wants to merge 1 commit intogoogleapis:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a timeout property to the Connection class in the Spanner DB-API, allowing users to set a gRPC deadline for SQL operations. The timeout value is propagated to execute_sql calls in run_statement, _do_execute_update_in_autocommit, and _handle_DQL_with_snapshot. Corresponding unit tests have been added to verify the property behavior and its correct application during execution. Feedback suggests refactoring the execute_sql call in _handle_DQL_with_snapshot to use a consistent keyword argument pattern for param_types, matching other updated call sites.
google/cloud/spanner_dbapi/cursor.py
Outdated
| kwargs = dict(request_options=self.request_options) | ||
| if self.connection._timeout is not None: | ||
| kwargs["timeout"] = self.connection._timeout | ||
| self._result_set = snapshot.execute_sql( | ||
| sql, | ||
| params, | ||
| get_param_types(params), | ||
| request_options=self.request_options, | ||
| **kwargs, | ||
| ) |
There was a problem hiding this comment.
For consistency with the changes in run_statement and _do_execute_update_in_autocommit, consider including param_types in the kwargs dictionary instead of passing it as a positional argument. This would make all three modified call sites for execute_sql follow the same pattern, improving maintainability.
| kwargs = dict(request_options=self.request_options) | |
| if self.connection._timeout is not None: | |
| kwargs["timeout"] = self.connection._timeout | |
| self._result_set = snapshot.execute_sql( | |
| sql, | |
| params, | |
| get_param_types(params), | |
| request_options=self.request_options, | |
| **kwargs, | |
| ) | |
| kwargs = dict( | |
| param_types=get_param_types(params), | |
| request_options=self.request_options, | |
| ) | |
| if self.connection._timeout is not None: | |
| kwargs["timeout"] = self.connection._timeout | |
| self._result_set = snapshot.execute_sql( | |
| sql, | |
| params, | |
| **kwargs, | |
| ) |
The DBAPI layer calls _SnapshotBase.execute_sql() in three code paths (snapshot reads, transaction reads/writes, autocommit DML) but never passes the timeout= argument. This causes all queries to use the gRPC default timeout of 3600 seconds. Add a timeout property to Connection and pass it through to execute_sql() in cursor._handle_DQL_with_snapshot(), cursor._do_execute_update_in_autocommit(), and connection.run_statement(). Fixes googleapis#1534
b73fa31 to
f93bed6
Compare
Summary
Add a
timeoutproperty toConnectionand passtimeout=toexecute_sql()in the three DBAPI code paths that currently omit it: snapshot reads, transaction statements, and autocommit DML.Fixes #1534
Background
_SnapshotBase.execute_sql()accepts atimeoutparameter that controls the gRPC deadline for theExecuteStreamingSqlRPC. When not provided, it defaults togapic_v1.method.DEFAULT, which resolves todefault_timeout=3600.0in the transport layer.The DBAPI calls
execute_sql()in three locations, none of which passtimeout=:cursor._handle_DQL_with_snapshot()callssnapshot.execute_sql(sql, params, param_types, request_options=...)withouttimeout=connection.run_statement()callstransaction.execute_sql(sql, params, param_types=..., request_options=...)withouttimeout=cursor._do_execute_update_in_autocommit()callstransaction.execute_sql(sql, params=..., param_types=..., last_statement=True)withouttimeout=This means DBAPI consumers (SQLAlchemy, Django, raw DBAPI) cannot control the gRPC deadline for individual statements — all queries use the 3600-second default.
Timeline
9b7fcd6(PR #6536)timeout=parameter added to_SnapshotBase.execute_sql()in the Spanner client2493fa1(PR #160)cursor._handle_DQL_with_snapshot()callsexecute_sql()withouttimeout=d59d502(PR #168)connection.run_statement()added — callsexecute_sql()withouttimeout=1a7c9d2(PR #278)timeout=expanded toread(),partition_read(),partition_query()in the clientcd3b950(PR #475)_handle_DQL_with_snapshot()extracted as separate method — still notimeout=ab768e4(PR #838)request_optionsadded to_handle_DQL_with_snapshot()—timeout=not addedee9662f(PR #1262)request_tag/transaction_tagadded to cursor/connection —timeout=not addedOther execution parameters (
request_options,request_priority,transaction_tag,request_tag) were each wired through the DBAPI incrementally. Thetimeoutparameter was not included in any of these additions.Changes
connection.pyself._timeout = Noneto__init__timeoutproperty and settertimeout=self._timeoutinrun_statement()when setcursor.pytimeout=self.connection._timeoutin_handle_DQL_with_snapshot()when settimeout=self.connection._timeoutin_do_execute_update_in_autocommit()when setWhen
timeoutisNone(the default),timeout=is not passed, preserving the existing behavior of usinggapic_v1.method.DEFAULT.Usage
Tests
Added 7 unit tests:
test_timeout_default_none— verifies default isNonetest_timeout_property— verifies getter/settertest_timeout_passed_to_run_statement— verifiestimeout=is passed inrun_statement()test_timeout_not_passed_when_none— verifiestimeout=is omitted whenNonetest_do_execute_update_with_timeout— verifiestimeout=in autocommit DML pathtest_handle_DQL_with_snapshot_timeout— verifiestimeout=in snapshot read pathtest_handle_DQL_with_snapshot_no_timeout— verifies omission in snapshot read pathAll 205 existing DBAPI tests continue to pass.
Related
timeout=onexecute_sql(): PR chore: add samples for transaction timeout configuration #1380 (spanner_set_statement_timeout), PR feat: add retry and timeout for batch dml #1107 (spanner_set_custom_timeout_and_retry)