Skip to content

Ensure to use psycopg3 is used while connecting to postgres DB when PGADMIN_CONFIG_CONFIG_DATABASE_URI is specified. #9701#9914

Open
yogeshmahajan-1903 wants to merge 3 commits into
pgadmin-org:masterfrom
yogeshmahajan-1903:master
Open

Ensure to use psycopg3 is used while connecting to postgres DB when PGADMIN_CONFIG_CONFIG_DATABASE_URI is specified. #9701#9914
yogeshmahajan-1903 wants to merge 3 commits into
pgadmin-org:masterfrom
yogeshmahajan-1903:master

Conversation

@yogeshmahajan-1903
Copy link
Copy Markdown
Contributor

@yogeshmahajan-1903 yogeshmahajan-1903 commented May 5, 2026

Summary by CodeRabbit

  • Bug Fixes
    • Normalizes PostgreSQL-style database URIs so the app consistently uses the correct driver prefix, improving reliability when connecting to external configuration databases and avoiding connection setup errors.
  • Tests
    • Added unit tests to validate URI normalization behavior and ensure idempotent, safe handling of non-Postgres URIs.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 5, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

PostgreSQL connection URIs are normalized: occurrences of postgres:// or postgresql:// are rewritten to postgresql+psycopg:// before being used. This normalization is applied when assigning app.config['SQLALCHEMY_DATABASE_URI'], inside evaluate_config, and in check_external_config_db(database_uri) prior to engine creation; other logic is unchanged.

Changes

PostgreSQL URI Scheme Normalization

Layer / File(s) Summary
Normalization helper
web/pgadmin/utils/db_utils.py
Adds normalize_database_uri(uri) which rewrites leading postgres:// or postgresql:// to postgresql+psycopg:// via a single regex substitution.
App wiring: create_app assignment
web/pgadmin/__init__.py
Import normalize_database_uri and call it when assigning app.config['SQLALCHEMY_DATABASE_URI'] from config.CONFIG_DATABASE_URI.
evaluate_config usage
web/pgadmin/evaluate_config.py
Import normalize_database_uri and replace previous inline normalization by calling the helper for custom_config_settings['CONFIG_DATABASE_URI'].
Utility: external-config DB check
web/pgadmin/utils/check_external_config_db.py
Add import re and apply a single substitution rewrite of database_uri from postgres:// / postgresql:// to postgresql+psycopg:// before creating the SQLAlchemy engine and performing the existing server-table check.
Unit tests
web/pgadmin/utils/tests/test_db_utils.py
Add tests asserting rewrite of postgres:// and postgresql:// URIs, idempotency for postgresql+psycopg://, and no-change for sqlite:///.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main objective of the PR: ensuring psycopg3 is used when connecting to PostgreSQL via PGADMIN_CONFIG_CONFIG_DATABASE_URI, which is reflected in the database URI normalization changes across multiple modules.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
web/pgadmin/utils/check_external_config_db.py (1)

22-30: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

connection is unbound in the finally block when engine.connect() raises.

If engine.connect() at line 23 throws (bad credentials, host unreachable, etc.), connection is never assigned. The except Exception: return False branch begins executing, but Python runs finally before the return, hitting connection.close()NameError: name 'connection' is not defined. This NameError propagates out, completely overriding the intended False return, so callers never get the graceful failure.

🐛 Proposed fix
+    connection = None
     try:
         connection = engine.connect()
         if inspect(engine).has_table("server"):
             return True
         return False
     except Exception:
         return False
     finally:
-        connection.close()
+        if connection is not None:
+            connection.close()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/pgadmin/utils/check_external_config_db.py` around lines 22 - 30, The
finally block closes a connection variable that may not be assigned if
engine.connect() raises; initialize connection = None before the try (or use a
context manager with engine.connect()) and in the finally only call
connection.close() if connection is not None (or rely on the context manager to
auto-close). Update the try/except/finally around engine.connect(),
inspect(engine).has_table("server"), and connection.close() to guard against an
unbound NameError.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/pgadmin/utils/check_external_config_db.py`:
- Around line 18-20: The E125 pycodestyle error comes from the broken
continuation in the if statement using database_uri.startswith across two lines;
change the condition to use a single startswith call with a tuple (e.g.
database_uri.startswith(("postgresql://", "postgres://"))) so the if is
single-line, then keep the existing replacement logic that rewrites ":// " to
"+psycopg://" on database_uri (referencing the database_uri variable, the
startswith check, and the existing replace call).

---

Outside diff comments:
In `@web/pgadmin/utils/check_external_config_db.py`:
- Around line 22-30: The finally block closes a connection variable that may not
be assigned if engine.connect() raises; initialize connection = None before the
try (or use a context manager with engine.connect()) and in the finally only
call connection.close() if connection is not None (or rely on the context
manager to auto-close). Update the try/except/finally around engine.connect(),
inspect(engine).has_table("server"), and connection.close() to guard against an
unbound NameError.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 923b9d2a-08a5-407f-a036-74b2a2bafd5d

📥 Commits

Reviewing files that changed from the base of the PR and between 2f211dc and 02ffd14.

📒 Files selected for processing (1)
  • web/pgadmin/utils/check_external_config_db.py

Comment thread web/pgadmin/utils/check_external_config_db.py Outdated
@asheshv
Copy link
Copy Markdown
Contributor

asheshv commented May 5, 2026

Code review

Found 2 issues:

  1. Fix is incomplete — same rewrite is missing where SQLALCHEMY_DATABASE_URI is actually set, so the bug from issue using with docker-compose #9701 will still reproduce. check_external_config_db() is invoked once from pkg/docker/entrypoint.sh:153 as a pre-flight check. After that pre-check passes, pgadmin/__init__.py:340 assigns the unmodified config.CONFIG_DATABASE_URI directly to app.config['SQLALCHEMY_DATABASE_URI'], and run_migration_for_others()db_upgrade(app) → SQLAlchemy then tries to load the psycopg2 dialect for the bare postgresql:// URI. This is exactly the second traceback the reporter pasted in using with docker-compose #9701 (File "/pgadmin4/pgadmin/__init__.py", line 476import psycopg2 → ModuleNotFoundError). The same postgresql://postgresql+psycopg:// normalization needs to be applied at the URI-assignment site (or in config.py/__init__.py) for the fix to actually work.

    ##########################################################################
    if config.CONFIG_DATABASE_URI is not None and \
    len(config.CONFIG_DATABASE_URI) > 0:
    app.config['SQLALCHEMY_DATABASE_URI'] = config.CONFIG_DATABASE_URI
    else:
    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///{0}?timeout={1}' \
    .format(config.SQLITE_PATH.replace('\\', '/'),

  2. postgres:// branch produces an invalid SQLAlchemy URL. Verified against the project's SQLAlchemy 2.0.49: postgres:// and postgres+psycopg:// both raise NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres[.psycopg] (the postgres:// alias was removed in SQLAlchemy 1.4). For input postgres://..., the current code produces postgres+psycopg://..., which still fails at create_engine. To handle the postgres:// form, the rewrite needs to swap the scheme to postgresql+psycopg://, e.g.:

    if database_uri.startswith(("postgresql://", "postgres://")):
        database_uri = re.sub(r"^postgres(ql)?://", "postgresql+psycopg://", database_uri, count=1)

    """
    if database_uri.startswith("postgresql://") or database_uri.startswith(
    "postgres://"):
    database_uri = database_uri.replace("://", "+psycopg://", 1)
    engine = create_engine(database_uri)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@yogeshmahajan-1903
Copy link
Copy Markdown
Contributor Author

Code review

Found 2 issues:

  1. Fix is incomplete — same rewrite is missing where SQLALCHEMY_DATABASE_URI is actually set, so the bug from issue using with docker-compose #9701 will still reproduce. check_external_config_db() is invoked once from pkg/docker/entrypoint.sh:153 as a pre-flight check. After that pre-check passes, pgadmin/__init__.py:340 assigns the unmodified config.CONFIG_DATABASE_URI directly to app.config['SQLALCHEMY_DATABASE_URI'], and run_migration_for_others()db_upgrade(app) → SQLAlchemy then tries to load the psycopg2 dialect for the bare postgresql:// URI. This is exactly the second traceback the reporter pasted in using with docker-compose #9701 (File "/pgadmin4/pgadmin/__init__.py", line 476import psycopg2 → ModuleNotFoundError). The same postgresql://postgresql+psycopg:// normalization needs to be applied at the URI-assignment site (or in config.py/__init__.py) for the fix to actually work.

    ##########################################################################
    if config.CONFIG_DATABASE_URI is not None and \
    len(config.CONFIG_DATABASE_URI) > 0:
    app.config['SQLALCHEMY_DATABASE_URI'] = config.CONFIG_DATABASE_URI
    else:
    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///{0}?timeout={1}' \
    .format(config.SQLITE_PATH.replace('\\', '/'),

  2. postgres:// branch produces an invalid SQLAlchemy URL. Verified against the project's SQLAlchemy 2.0.49: postgres:// and postgres+psycopg:// both raise NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres[.psycopg] (the postgres:// alias was removed in SQLAlchemy 1.4). For input postgres://..., the current code produces postgres+psycopg://..., which still fails at create_engine. To handle the postgres:// form, the rewrite needs to swap the scheme to postgresql+psycopg://, e.g.:

    if database_uri.startswith(("postgresql://", "postgres://")):
        database_uri = re.sub(r"^postgres(ql)?://", "postgresql+psycopg://", database_uri, count=1)

    """
    if database_uri.startswith("postgresql://") or database_uri.startswith(
    "postgres://"):
    database_uri = database_uri.replace("://", "+psycopg://", 1)
    engine = create_engine(database_uri)

🤖 Generated with Claude Code

  • If this code review was useful, please react with 👍. Otherwise, react with 👎.

Fixed.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
web/pgadmin/utils/check_external_config_db.py (1)

25-33: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard connection cleanup when engine.connect() fails.

If connect raises, connection is never bound, and finally: connection.close() can raise UnboundLocalError, masking the intended False return path.

Proposed fix
     engine = create_engine(database_uri)
     try:
-        connection = engine.connect()
-        if inspect(engine).has_table("server"):
-            return True
-        return False
+        with engine.connect() as connection:
+            return inspect(connection).has_table("server")
     except Exception:
         return False
-    finally:
-        connection.close()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/pgadmin/utils/check_external_config_db.py` around lines 25 - 33, The
finally block can raise UnboundLocalError if engine.connect() fails because
connection is not set; update the check_external_config_db logic that calls
engine.connect() and uses connection.close() by either initializing connection =
None before the try or, better, using a context manager (engine.connect()) to
ensure safe cleanup; specifically adjust the block around engine.connect(),
inspect(engine).has_table("server") and connection.close() so you only call
connection.close() when connection is truthy (or use "with engine.connect() as
connection:" to eliminate the manual close).
🧹 Nitpick comments (1)
web/pgadmin/__init__.py (1)

340-344: ⚡ Quick win

Centralize DB URI normalization to one shared helper.

This rewrite now exists in both web/pgadmin/__init__.py and web/pgadmin/utils/check_external_config_db.py; extracting a single helper will prevent future drift between pre-check and runtime assignment paths.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/pgadmin/__init__.py` around lines 340 - 344, Extract the DB URI
normalization logic currently duplicated in the module that sets
app.config['SQLALCHEMY_DATABASE_URI'] and in check_external_config_db.py into a
single shared helper (e.g., a function named normalize_database_uri) placed in
the existing utils module used by both files; replace the inline re.sub call in
web/pgadmin/__init__.py (the block that builds _db_uri) and the corresponding
code in check_external_config_db.py to call
normalize_database_uri(config.CONFIG_DATABASE_URI) so both pre-check and runtime
assignment use the identical transformation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/pgadmin/utils/check_external_config_db.py`:
- Around line 21-23: The re.sub call that assigns database_uri exceeds the line
length; break the arguments across multiple lines to satisfy pycodestyle E501
without changing behavior. Locate the assignment to database_uri where re.sub is
called and format it like: re.sub( pattern, replacement, database_uri, count=1 )
with each argument on its own line (or pattern/replacement on one line and
database_uri/count on the next) so the long line is wrapped but the regex,
replacement string ("postgresql+psycopg://") and count=1 are unchanged.

---

Outside diff comments:
In `@web/pgadmin/utils/check_external_config_db.py`:
- Around line 25-33: The finally block can raise UnboundLocalError if
engine.connect() fails because connection is not set; update the
check_external_config_db logic that calls engine.connect() and uses
connection.close() by either initializing connection = None before the try or,
better, using a context manager (engine.connect()) to ensure safe cleanup;
specifically adjust the block around engine.connect(),
inspect(engine).has_table("server") and connection.close() so you only call
connection.close() when connection is truthy (or use "with engine.connect() as
connection:" to eliminate the manual close).

---

Nitpick comments:
In `@web/pgadmin/__init__.py`:
- Around line 340-344: Extract the DB URI normalization logic currently
duplicated in the module that sets app.config['SQLALCHEMY_DATABASE_URI'] and in
check_external_config_db.py into a single shared helper (e.g., a function named
normalize_database_uri) placed in the existing utils module used by both files;
replace the inline re.sub call in web/pgadmin/__init__.py (the block that builds
_db_uri) and the corresponding code in check_external_config_db.py to call
normalize_database_uri(config.CONFIG_DATABASE_URI) so both pre-check and runtime
assignment use the identical transformation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: da20045a-80cb-4264-bd30-6761a636dd7f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d92361 and f34a5b3.

📒 Files selected for processing (2)
  • web/pgadmin/__init__.py
  • web/pgadmin/utils/check_external_config_db.py

Comment thread web/pgadmin/utils/check_external_config_db.py
@yanehi
Copy link
Copy Markdown

yanehi commented May 15, 2026

I encountered this error while upgrading pgAdmin from version 9.14 to 9.15.

Copy link
Copy Markdown
Contributor

@asheshv asheshv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix — the core change is correct. A couple of follow-ups before merge:

1. Extract the rewrite into a shared helper.
The same re.sub(r"^postgres(ql)?://", "postgresql+psycopg://", ...) lives in both web/pgadmin/__init__.py and web/pgadmin/utils/check_external_config_db.py. A single normalize_database_uri(uri) helper (e.g. under web/pgadmin/utils/) used by both call sites prevents drift if another driver (asyncpg, cockroachdb, …) ever needs similar treatment, and removes the asymmetric startswith guard that currently exists in only one of the two sites.

2. Add a unit test for the rewrite.
Small but valuable for regression prevention. Suggested cases:

  • postgres://u:p@h/dbpostgresql+psycopg://u:p@h/db
  • postgresql://u:p@h/dbpostgresql+psycopg://u:p@h/db
  • postgresql+psycopg://… → unchanged (idempotency)
  • sqlite:///… → unchanged

Nits (optional):

  • r"^postgres(ql)?://"r"^postgres(?:ql)?://" (non-capturing group; cosmetic).
  • _db_uri in web/pgadmin/__init__.py is single-use — the re.sub result can be assigned to SQLALCHEMY_DATABASE_URI directly.
  • A release-notes entry referencing #9701 would be nice.

Otherwise LGTM — the regex is anchored, count=1, idempotent for already-prefixed URIs, and both URI-writing paths (create_app and the Docker pre-flight via check_external_config_db) are covered.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/pgadmin/evaluate_config.py`:
- Around line 119-120: The assignment to
custom_config_settings['CONFIG_DATABASE_URI'] is too long for pycodestyle; split
the expression so the line length is below the limit by wrapping the call to
normalize_database_uri across lines (for example, use parentheses to break the
RHS onto a new line or assign the input to a short temporary variable and pass
that into normalize_database_uri). Update the statement referencing
custom_config_settings['CONFIG_DATABASE_URI'] and normalize_database_uri(...)
accordingly so the resulting lines comply with the style guide.

In `@web/pgadmin/utils/db_utils.py`:
- Line 19: The long return statement that rewrites the URI using re.sub is
exceeding line length; split the expression across multiple lines to satisfy
pycodestyle E501 (for example, break after the comma and wrap the pattern/string
in parentheses or assign parts to short variables) while keeping the same call:
re.sub(r"^postgres(?:ql)?://", "postgresql+psycopg://", uri, count=1). Ensure
you preserve the exact regex, replacement and count=1 argument and only reformat
for line length.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1359f7db-a511-46f5-a733-e5fb385fc212

📥 Commits

Reviewing files that changed from the base of the PR and between 1b7ad09 and e4c029e.

📒 Files selected for processing (4)
  • web/pgadmin/__init__.py
  • web/pgadmin/evaluate_config.py
  • web/pgadmin/utils/db_utils.py
  • web/pgadmin/utils/tests/test_db_utils.py

Comment thread web/pgadmin/evaluate_config.py Outdated
Comment thread web/pgadmin/utils/db_utils.py Outdated
@yanehi
Copy link
Copy Markdown

yanehi commented May 21, 2026

@asheshv @yogeshmahajan-1903 - After updating from 9.14 to 9.15, I get the following error and PgAdmin won't start:

...
File "/pgadmin4/pgadmin/utils/check_external_config_db.py", line 18, in check_external_config_db
raise exc.ArgumentError(
"Could not parse SQLAlchemy URL from given URL string"
)
...

Unfortunately, the suggestion in #9701 (comment) to adjust the connection string didn't work for me.

@yogeshmahajan-1903
Copy link
Copy Markdown
Contributor Author

#9701 (comment)

Could you please provide an example for your DB_URI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants