-
Notifications
You must be signed in to change notification settings - Fork 104
Description
Summary
The Spanner DBAPI layer (spanner_dbapi) does not pass the timeout parameter when calling _SnapshotBase.execute_sql(), causing all queries to use the gRPC default timeout of 3600 seconds. This prevents DBAPI consumers (SQLAlchemy, Django, raw DBAPI) from controlling per-statement gRPC deadlines.
Background
_SnapshotBase.execute_sql() accepts a timeout parameter that controls the gRPC deadline for the ExecuteStreamingSql RPC. When not provided, it defaults to gapic_v1.method.DEFAULT, which resolves to default_timeout=3600.0 in the gRPC transport layer.
The DBAPI calls execute_sql() in three code paths, none of which pass timeout=:
- Snapshot reads —
cursor._handle_DQL_with_snapshot()callssnapshot.execute_sql(sql, params, param_types, request_options=...)withouttimeout= - Transaction reads/writes —
connection.run_statement()callstransaction.execute_sql(sql, params, param_types=..., request_options=...)withouttimeout= - Autocommit DML —
cursor._do_execute_update_in_autocommit()callstransaction.execute_sql(sql, params=..., param_types=..., last_statement=True)withouttimeout=
Since timeout defaults to gapic_v1.method.DEFAULT, which resolves to default_timeout=3600.0 in the gRPC transport layer (services/spanner/transports/base.py), all DBAPI queries have a 3600-second gRPC deadline regardless of the caller's intent.
Timeline
| Date | Commit | Event |
|---|---|---|
| Nov 2018 | 9b7fcd6 (PR #6536) |
timeout= added to _SnapshotBase.execute_sql() |
| Nov 2020 | 2493fa1 (PR #160) |
DBAPI created — calls execute_sql() without timeout= |
| Nov 2020 | d59d502 (PR #168) |
run_statement() added — calls execute_sql() without timeout= |
| Mar 2021 | 1a7c9d2 (PR #278) |
timeout= expanded to read(), partition_read(), partition_query() |
| Oct 2021 | cd3b950 (PR #475) |
_handle_DQL_with_snapshot() extracted — still no timeout= |
| Oct 2022 | ab768e4 (PR #838) |
request_options added to _handle_DQL_with_snapshot() — timeout= not added |
| Jan 2025 | ee9662f (PR #1262) |
request_tag/transaction_tag added — timeout= not added |
Other execution parameters (request_options, request_priority, transaction_tag, request_tag) were each wired through incrementally via the same pattern: a Connection property plus pass-through in cursor methods. The timeout parameter was not included in any of these additions.
Proposed Change
Add a timeout property to Connection, following the same pattern used for staleness, read_only, and request_priority. Pass timeout= in the three cursor/connection methods that call execute_sql().
Files changed
connection.py— Addself._timeout = Noneto__init__, addtimeoutproperty/setter, passtimeout=inrun_statement()cursor.py— Passtimeout=in_handle_DQL_with_snapshot()and_do_execute_update_in_autocommit()
Usage
from google.cloud.spanner_dbapi import connect
conn = connect(instance_id, database_id, project=project)
conn.timeout = 60 # 60-second gRPC deadline for subsequent statements
cursor = conn.cursor()
cursor.execute("SELECT * FROM my_table")This also enables framework integration. The companion change in python-spanner-sqlalchemy can wire this through execution_options:
engine.execution_options(timeout=60)Related
- Google's own samples demonstrate
timeout=onexecute_sql(): PR chore: add samples for transaction timeout configuration #1380 (spanner_set_statement_timeoutsample), PR feat: add retry and timeout for batch dml #1107 (spanner_set_custom_timeout_and_retrysample) - Companion SQLAlchemy dialect change: feat(sqlalchemy-spanner): wire timeout execution option through to DBAPI Connection.timeout google-cloud-python#16467