DFS file loading fails with Python 3.14 on Windows (error 2003) during test suite execution

[ecomodeller]

Summary

mikecore fails to load DFS files with error 2003 (F_ERR_OPEN) when running test suites with Python 3.14 on Windows. Individual tests pass, but full suite execution experiences intermittent failures. Does not occur on Linux or with Python 3.13 on Windows.

Reproduction

Discovered in modelskill test suite:

# Fails: Full suite on Windows + Python 3.14
pytest tests/ -v
# Exception: Could not load file ... (Error code 2003)

# Passes: Same test individually
pytest tests/test_pointcompare.py::test_score -v

Test Results

Platform	Python	Result
Windows	3.13	✅ Pass
Windows	3.14	❌ Intermittent failures
Linux	3.14	✅ Pass

Root Cause

Python 3.14 introduced ctypes thread safety changes for free-threaded builds. mikecore's singleton DLL loading pattern in DfsDLL.py (lines 42-61) lacks thread safety:

class DfsDLL:
    Wrapper = None  # Global static, no locking
    
    @staticmethod
    def Init(libfilepath=None):
        if DfsDLL.Wrapper is None:  # Race condition
            DfsDLL.Wrapper = ctypes.CDLL(...)

Suggested Fix

Add thread-safe initialization with proper locking around singleton check and DLL loading.

Workaround

Use Python 3.13 on Windows. See modelskill commit ff699be8.

Environment

• mikecore: 0.2.2
• Python: 3.14.2
• OS: Windows 11

[ecomodeller]

One possible approach: Python's import system is inherently thread-safe (protected by the import lock), so moving the DLL loading to module level would avoid the race condition without needing explicit locking.

Something like:

# At module level in DfsDLL.py
_wrapper = ctypes.CDLL(os.path.join(_get_lib_path(), 'ufs.dll' if os.name == 'nt' else 'libufs.so'))
_wrapper.dfsInitSystem()
# ... configure argtypes/restype ...

This is a common Python pattern for singletons - simpler than the C#-style lazy initialization and thread-safe by design.

[ryan-kipawa]

I'm not sure I fully understand the root cause of the problem. Why/where is DfsDLL.Init being called from multiple threads? How confident are we with the root cause analysis above?

It looks like all DLL's all called one time from mikecore-python package's init:

mikecore-python/mikecore/__init__.py

Lines 34 to 36 in 5359aea

	eumDLL.Init()
	MzCartDLL.Init(mikebin)
	DfsDLL.Init(mikebin)

With the exception of DfsFile also being called here again:

mikecore-python/mikecore/DfsFile.py

Line 686 in 5359aea

DfsDLL.Init()

I would suggestion perhaps investigating removal of the line above to see if that fixes the issue. Probably also good to start understanding the context of where the library is being used multithreaded, or is this somehow baked in now to ctypes / pytest / ...?

[ecomodeller]

Check the logs on this
https://github.com/DHI/modelskill/actions/runs/20771215685

[ryan-kipawa]

Check the logs on this https://github.com/DHI/modelskill/actions/runs/20771215685

I'm doubtful that the root cause analysis above is correct from looking at this, but maybe 🤷‍♂️ I'll look a bit more into it

[ryan-kipawa]

I tried running the mikecore-python test suite on my Windows machine with Python 3.14. All tests passed in about five separate runs that I did. Maybe it also makes sense to have scheduled runners on mikecore-python.

If the test suite in modelskill did reveal something actually related to mikecore-python, then perhaps it's missing a test or two.

[ryan-kipawa]

I used Process Monitor on Windows to analyze modelskill's test suite. I focused on one of the failed tests related to two_elements.dfsu, shown below:

FAILED tests/test_pointcompare.py::test_score_two_elements - Exception: Could not load file tests/testdata/two_elements.dfsu (Error code 2003)

The trace for Process Monitor is as follows:

There's no direct obvious error from this. You can notice that the file is opened and closed three times in under 20ms. At the end of the overall test suite, I also noticed there's 100+ CloseFile operations.

I'm not sure if this is related or not, but just adding at extra information. Somehow I was hypothesizing that there's a race condition involving file handles, but maybe that's not applicable if they're all read-only?

[ryan-kipawa]

I started #42 that has a minimal test example exposing this "bug" in mikecore-python instead of modelskill's.

I suspect there's a maximum number of files that can be open at one time on Windows (512) which modelskill's test suite is exceeding. See here for details: https://stackoverflow.com/questions/870173/is-there-a-limit-on-number-of-open-files-in-windows

Notice: this problem is not limited to Python 3.14. I've reproduced the failure on Python 3.10. Not sure why it became exposed in Python 3.14, but one potential explanation is the speedups caused a race condition on the file open/closes that sometimes exceeds 512.