Skip to content

Comments

Fix Python 3.13 compatibility with pathlib #61#62

Merged
monozoide merged 1 commit intodevfrom
chore/61-python-3.13-compatibility
Oct 16, 2025
Merged

Fix Python 3.13 compatibility with pathlib #61#62
monozoide merged 1 commit intodevfrom
chore/61-python-3.13-compatibility

Conversation

@monozoide
Copy link
Owner

@monozoide monozoide commented Oct 16, 2025

Description

This PR fixes the Python 3.13 compatibility issue causing SQL export to fail due to the removal of pathlib.Path objects supporting the context manager protocol.

The problematic pattern used importlib.resources.files().joinpath() directly as a context manager, which worked in Python 3.9 - 3.12 but was deprecated and finally removed in Python 3.13.

Root Cause

Before (Python 3.9-3.12 compatible, broken in 3.13+):

with importlib.resources.files("lib.maillogsentinel.data").joinpath(
    "maillogsentinel_sql_column_mapping.json"
) as bundled_path:
    # PosixPath object used as context manager - FAILS in Python 3.13

After (Python 3.9-3.13+ compatible):

from importlib.resources import as_file, files

with as_file(files("lib.maillogsentinel.data").joinpath(
    "maillogsentinel_sql_column_mapping.json"
)) as bundled_path:
    # as_file() provides proper context manager - WORKS in all versions

Files Modified

  • lib/maillogsentinel/sql_exporter.py - Update resource loading in SQL export functionality
  • lib/maillogsentinel/sql_importer.py - Update resource loading in SQL import functionality
  • Any other files using the deprecated importlib.resources.files().joinpath() as context manager pattern

Changes Made

  1. Import update: Added as_file import alongside files from importlib.resources
  2. Context manager fix: Wrapped files().joinpath() calls with as_file() to provide proper context manager
  3. Backward compatibility: Solution works on Python 3.9+ without breaking existing functionality

Error Fixed

Before fix:

ERROR: Could not load headers from bundled mapping for test setup: 'PosixPath' object does not support the context manager protocol
sql_export: Error loading bundled default column mapping file: 'PosixPath' object does not support the context manager protocol. Aborting SQL export.
TypeError: 'PosixPath' object does not support the context manager protocol

After fix:

  • SQL import/export completes successfully on Python 3.13+
  • No more TypeError exceptions
  • Full functionality restored

Testing

Environment Requirements

  • Python 3.13+ system (where the issue occurs)
  • MailLogSentinel installation

Test Steps

  1. Run MailLogSentinel SQL export: 
    python3 /usr/local/bin/maillogsentinel.py --sql-export
python3 /usr/local/bin/maillogsentinel.py --sql-import
  2. Verify import/export completes without errors
  3. Confirm SQL export file is generated successfully
  4. Test on Python 3.13

Expected Results

  • ✅ No TypeError: 'PosixPath' object does not support the context manager protocol
  • ✅ SQL import/export functionality works on Python 3.13+
  • ✅ Backward compatibility maintained for Python 3.9-3.12
  • ✅ No deprecation warnings

Python Version Compatibility

Python Version Status Notes
3.9 ✅ Working Original development version
3.10 ✅ Working No issues
3.11 ✅ Working Deprecation warnings (now fixed)
3.12 ✅ Working Last version with deprecated support
3.13+ ✅ Fixed This PR resolves the breaking change

Impact

  • User Impact: Restores SQL import/export functionality for users on Python 3.13+
  • Compatibility: Maintains support across Python 3.9-3.13+ range
  • Risk: Low - uses recommended modern importlib.resources patterns
  • Breaking Changes: None - purely internal implementation fix

🔗 Links

close #61

References

@monozoide
Fix Python 3.13 compatibility with pathlib #61
0100525 
Refactored the SQL import/export functionality to use `importlib.resources.as_file` instead of the deprecated `pathlib.Path` context manager. This resolves a crash on Python 3.13, where `pathlib.Path` objects no longer support the context manager protocol.
@monozoide monozoide self-assigned this Oct 16, 2025
@monozoide monozoide added the enhancement New feature or request label Oct 16, 2025
@monozoide monozoide added the chore For maintenance/infrastructure tasks label Oct 16, 2025
@monozoide monozoide moved this from Todo to In progress in MailLogSentinel Roadmap Oct 16, 2025
@monozoide monozoide added this to the v5.15.2 milestone Oct 16, 2025
@monozoide monozoide linked an issue Oct 16, 2025 that may be closed by this pull request
[COMPATIBILITY] Python 3.13 compatibility issue - PosixPath context manager removal breaks SQL export #61
Closed
@monozoide monozoide merged commit 808ae5b into dev Oct 16, 2025
4 checks passed
@github-project-automation github-project-automation bot moved this from In progress to Done in MailLogSentinel Roadmap Oct 16, 2025
@monozoide monozoide deleted the chore/61-python-3.13-compatibility branch October 16, 2025 21:03
monozoide added a commit that referenced this pull request Oct 17, 2025
@monozoide
Dev (#67)
cb55b68 
* workflows/update-gitignore-and-create-mls-ci #34 (#38)

* workflows/update-gitignore #34

Updated .gitignore to allow .log files in the repository for test data.

* workflow/add-gh-actions-workflow #34

Introduces a CI workflow that runs linting and tests on code changes to the main and dev branch, while skipping these steps for documentation-only changes. This setup uses flake8 for linting and pytest for testing, and optimizes CI runs by detecting code vs. docs changes.

* docs: Add sample email report and log files to dataset #31 (#39)

Added sample_email_report_output.txt, sample_mail.log, and sample_sasl.log to docs/dataset for documentation and testing purposes. These files provide example outputs and logs for MailLogSentinel.

* chore: add standardized PR templates for all contribution types (#35) (#42)

* Add PR templates

Introduces standardized pull request templates for bugfixes, code changes, documentation, CI/CD, and features in the .github/PR_TEMPLATES directory. These templates help ensure consistent and thorough PR descriptions, validation steps, and project checklists across different types of contributions.

* Delete PULL_REQUEST_TEMPLATE.md #35

Deleted the .github/PULL_REQUEST_TEMPLATE.md file. This change removes the default template for new pull requests.

* Revamp README with clearer setup and feature guide #32 (#43)

* Revamp README with clearer setup and feature guide #32

The README has been rewritten for clarity and conciseness, featuring a new quick start guide, clearer prerequisites, simplified command references, and improved documentation links. The overview, installation, and usage instructions are now more accessible, and advanced features are summarized with direct links to the Wiki. The new format is more user-friendly for first-time users and contributors.

* Update README links and formatting #32

Corrected documentation and sample output links, updated the contributing guide URL, and improved formatting for the closing quote in the README.

* Fix relative link to sample email report in README #32

Updated the link to the daily email report example to use the correct relative path, ensuring the documentation points to the right file location.

* ci: fix path filter for docs-only changes #44 (#45)

Enhanced the GitHub Actions workflow to better distinguish between code and documentation changes using separate path filters for pull requests and pushes. Updated the lint job to use Python 3.11 and ruff instead of flake8, and improved dependency installation for both lint and test jobs. The workflow now supports a fast path for documentation-only changes, skipping unnecessary jobs.

* Revise and expand contributing guidelines #33 (#46)

* Revise and expand contributing guidelines #33

Updated CONTRIBUTING.md with clearer, more structured quick-start instructions and recommendations. Added a new CONTRIBUTING_DETAILED.md file providing comprehensive workflow, commit signing, quality standards, and contribution requirements to help contributors follow best practices.

* Fix relative links in contributing docs #33

Updated relative paths in CONTRIBUTING.md and CONTRIBUTING_DETAILED.md to ensure links to detailed guidelines and discussions work correctly.

Closes #33

* Revise and condense maillogsentinel man page #47 (#49)

The man page for maillogsentinel was rewritten for clarity, brevity, and improved structure. Redundant and verbose sections were condensed, option descriptions were clarified, and auxiliary tool documentation was streamlined. The new version emphasizes practical usage, configuration, diagnostics, and security best practices, while removing excessive detail and outdated formatting.

* Add manpages for ipinfo and log_anonymizer #48 (#50)

Introduces manual pages for the ipinfo and log_anonymizer command-line tools, providing usage instructions, options, examples, and related information for system administrators.

* Add comprehensive FAQ documentation (#51)

Introduces a detailed FAQ (docs/wiki/FAQ.md) covering installation, configuration, usage, maintenance, integrations, troubleshooting, data analysis, security, and development for MailLogSentinel. This resource aims to assist users and contributors with common questions and operational guidance.

* Update documentation links and add manual pages #52 (#53)

Adjusted wiki links to use correct relative paths, added FAQ link, and included references to manual pages for maillogsentinel, ipinfo, and log_anonymizer in the README.

* Create readable markdown versions of manpages #54 (#55)

Introduces manual pages in Markdown format for the ipinfo, log_anonymizer, and maillogsentinel utilities. These documents provide usage instructions, options, configuration details, examples, and security considerations for each tool as part of the MailLogSentinel project.

* Add Debian install guide for MailLogSentinel #21 (#56)

Introduces a comprehensive installation and configuration guide for MailLogSentinel on Debian 12/13. The guide covers prerequisites, system preparation, installation steps, configuration, verification, service and timer setup, advanced options, troubleshooting, security, and additional resources.

* Fix linting errors in CI workflow #58 (#59)

* Fix linting errors in CI workflow #58

This commit fixes a number of linting errors that were causing the CI workflow to fail. The errors were primarily related to unused imports, f-strings without placeholders, and unused variables.

* Refactor and clean up test code #58

Removed unused imports and variables in test_maillogsentinel_setup.py and test_sql_exporter.py. Updated test_run_sql_export_basic_flow to use context manager for patching datetime and simplified the mocking of the logger. These changes improve test clarity and maintainability.

* Remove duplicate unittest.mock import #58

Consolidated the import of patch and MagicMock from unittest.mock to avoid redundancy in the test file.

* Remove unused MagicMock import #58

Cleaned up the import statements by removing MagicMock, which was not used in the test file.

* Fix Python 3.13 compatibility with pathlib #61 (#62)

Refactored the SQL import/export functionality to use `importlib.resources.as_file` instead of the deprecated `pathlib.Path` context manager. This resolves a crash on Python 3.13, where `pathlib.Path` objects no longer support the context manager protocol.

close #61

* Fix: SQL export reports success on failure #63 (#64)

* Fix: SQL export reports success on failure #63

The --sql-export command was displaying a misleading success message even when data conversion errors occurred. This was because the `format_sql_value` function would log a warning and return NULL on conversion failure, but it did not propagate the error.

This commit makes the data conversion stricter by raising an `SQLExportError` when a conversion for a NOT NULL column fails. The `run_sql_export` function now handles this exception, counts the errors, and returns `False` if any errors occurred. It also deletes the incomplete SQL file to avoid leaving invalid artifacts.

This ensures that the SQL export process provides accurate feedback and only reports success when the export is actually successful.

* Update SQL export tests #63

Tests now expect SQLExportError when None is provided for NOT NULL columns. Updated assertions to match new SQL statement formatting with quoted column names. This improves test accuracy and enforces stricter validation in SQL export logic.

Closes #63

* Improve NULL handling for NOT NULL integer columns Fix #65 (#66)

Refines the logic in format_sql_value to treat columns as nullable only if 'NOT NULL' is absent from the SQL type definition. Adds stricter validation to prevent empty strings from being converted to integers for NOT NULL columns, and introduces a new test case to verify data conversion failure for NOT NULL integer fields.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@monozoide monozoide

Labels

chore For maintenance/infrastructure tasks enhancement New feature or request

Projects

MailLogSentinel Roadmap
Status: Done

Milestone

v5.15.2

Development

Successfully merging this pull request may close these issues.

[COMPATIBILITY] Python 3.13 compatibility issue - PosixPath context manager removal breaks SQL export

1 participant

@monozoide