MINIFICPP-2718 Windows based docker tests by martinzink · Pull Request #2106 · apache/nifi-minifi-cpp

martinzink · 2026-02-12T14:54:35Z

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?

For code changes:

If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE file?
If applicable, have you updated the NOTICE file?

For documentation related changes:

Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

Closes #2106 Signed-off-by: Martin Zink <martinzink@apache.org>

Copilot

Pull request overview

This PR updates the MiNiFi C++ Behave-based test framework to support running a subset of Docker-based tests on Windows hosts, while also refactoring the container abstraction and SSL certificate utilities to be more portable.

Changes:

Introduces Windows container support (Windows container base class, Windows MiNiFi container, Windows Dockerfile) and gates scenarios via a @SUPPORTS_WINDOWS tag.
Refactors the Behave container layer to distinguish Linux vs. Windows containers and centralizes controller logic via a MinifiController helper.
Replaces M2Crypto/PyOpenSSL usage in the Behave framework with cryptography and updates dependent test containers accordingly.

Reviewed changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
run_flake8.sh	Updates flake8 exclude list to ignore additional virtualenv directories.
extensions/standard-processors/tests/features/steps/tcp_client_container.py	Switches test container base class to `LinuxContainer`.
extensions/standard-processors/tests/features/steps/syslog_container.py	Switches test container base class to `LinuxContainer`.
extensions/standard-processors/tests/features/steps/minifi_controller_steps.py	Routes controller-related steps through a `.controller` helper object.
extensions/standard-processors/tests/features/steps/diag_slave_container.py	Switches test container base class to `LinuxContainer`.
extensions/standard-processors/tests/features/replace_text.feature	Marks feature as `@SUPPORTS_WINDOWS`.
extensions/standard-processors/tests/features/core_functionality.feature	Marks feature as `@SUPPORTS_WINDOWS` and adjusts some step text/timeouts.
extensions/sql/tests/features/steps/postgress_server_container.py	Switches test container base class to `LinuxContainer`.
extensions/splunk/tests/features/steps/splunk_container.py	Removes PyOpenSSL usage and uses `ssl_utils.dump_cert/dump_key`; switches to `LinuxContainer`.
extensions/opc/tests/features/steps/opc_ua_server_container.py	Switches test container base class to `LinuxContainer`.
extensions/kafka/tests/features/steps/kafka_server_container.py	Removes PyOpenSSL usage and uses `dump_cert/dump_key` with DER output; switches to `LinuxContainer`.
extensions/grafana-loki/tests/features/steps/reverse_proxy_container.py	Switches test container base class to `LinuxContainer`.
extensions/grafana-loki/tests/features/steps/grafana_loki_container.py	Removes PyOpenSSL usage and uses `dump_cert/dump_key`; switches to `LinuxContainer`.
extensions/gcp/tests/features/steps/fake_gcs_server_container.py	Switches test container base class to `LinuxContainer`.
extensions/elasticsearch/tests/features/steps/opensearch_container.py	Removes PyOpenSSL usage and uses `dump_cert/dump_key`.
extensions/elasticsearch/tests/features/steps/elasticsearch_container.py	Removes PyOpenSSL usage and uses `dump_cert/dump_key`.
extensions/elasticsearch/tests/features/steps/elastic_base_container.py	Switches test container base class to `LinuxContainer`.
extensions/couchbase/tests/features/steps/couchbase_server_container.py	Removes PyOpenSSL usage and updates `retry_check(...)` decorator invocation style.
extensions/azure/tests/features/steps/azure_server_container.py	Switches test container base class to `LinuxContainer`.
extensions/aws/tests/features/steps/s3_server_container.py	Switches test container base class to `LinuxContainer`.
extensions/aws/tests/features/steps/kinesis_server_container.py	Switches test container base class to `LinuxContainer`.
docker/installed/win.Dockerfile	Adds a Windows Server Core Dockerfile to run MiNiFi installed via MSI.
behave_framework/src/minifi_test_framework/core/ssl_utils.py	Reimplements cert generation/dumping using `cryptography`.
behave_framework/src/minifi_test_framework/core/minifi_test_context.py	Selects Linux/FHS/Windows MiNiFi container implementations at runtime; introduces protocol typing.
behave_framework/src/minifi_test_framework/core/hooks.py	Skips non-`@SUPPORTS_WINDOWS` scenarios on Windows; makes teardown tolerant when setup is skipped.
behave_framework/src/minifi_test_framework/containers/nifi_container.py	Switches NiFi container base class to `LinuxContainer`.
behave_framework/src/minifi_test_framework/containers/minifi_win_container.py	Adds a Windows MiNiFi container implementation.
behave_framework/src/minifi_test_framework/containers/minifi_protocol.py	Adds a typing Protocol for MiNiFi container capabilities.
behave_framework/src/minifi_test_framework/containers/minifi_linux_container.py	Adds a Linux MiNiFi container implementation with controller support and cert provisioning.
behave_framework/src/minifi_test_framework/containers/minifi_fhs_container.py	Adds an FHS-layout MiNiFi container implementation with controller support and cert provisioning.
behave_framework/src/minifi_test_framework/containers/minifi_controller.py	Adds a helper encapsulating `minifi-controller` CLI interactions.
behave_framework/src/minifi_test_framework/containers/minifi_container.py	Updates cert dumping to use `dump_cert/dump_key` (legacy container remains in tree).
behave_framework/src/minifi_test_framework/containers/http_proxy_container.py	Switches test container base class to `LinuxContainer`.
behave_framework/src/minifi_test_framework/containers/container_windows.py	Adds a Windows container base class for test framework containers.
behave_framework/src/minifi_test_framework/containers/container_protocol.py	Adds a typing Protocol for container operations used by steps.
behave_framework/src/minifi_test_framework/containers/container_linux.py	Renames base container class to `LinuxContainer` and aligns it with the protocol.
behave_framework/pyproject.toml	Bumps framework version and replaces M2Crypto/PyOpenSSL deps with `cryptography`.
.gitignore	Ignores additional venv directories (`venv`, `behave_venv`).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-20T08:09:33Z

behave_framework/src/minifi_test_framework/core/minifi_test_context.py

 class MinifiTestContext(Context):
-    containers: dict[str, Container]
+    containers: dict[str, ContainerProtocol]
    scenario_id: str
    network: Network
    minifi_container_image: str
    resource_dir: str | None
-    root_ca_key: crypto.PKey
-    root_ca_cert: crypto.X509
+    root_ca_key: Any
+    root_ca_cert: Any


root_ca_key / root_ca_cert are typed as Any. Since ssl_utils now uses cryptography, these can be typed more precisely (e.g., cryptography.x509.Certificate and an RSAPrivateKey type) to keep static checking useful.

Copilot · 2026-02-20T08:09:34Z

behave_framework/src/minifi_test_framework/containers/container_windows.py

+
+        # 4. Create and Start
+        try:
+            print(f"Creating and starting container '{self.container_name}'...")


print(...) is used for container startup logging. Using the existing logging calls (as in LinuxContainer.deploy) would keep test output consistent and easier to control in CI.

Suggested change

print(f"Creating and starting container '{self.container_name}'...")

logging.info(f"Creating and starting container '{self.container_name}'...")

Copilot · 2026-02-20T08:09:34Z

behave_framework/src/minifi_test_framework/containers/container_windows.py

+
+            for file in self.files:
+                self._copy_content_to_container(file.content, file.path)
+
+        except Exception as e:
+            logging.error(f"Error starting container: {e}")
+            self.clean_up()
+            raise
+        return True
+
+    def _copy_content_to_container(self, content: str, target_path: str):
+        if not self.container:
+            return
+
+        win_path = self._normalize_path(target_path)
+        dir_name = os.path.dirname(win_path)
+        file_name = os.path.basename(win_path)
+
+        self._run_powershell(f"New-Item -ItemType Directory -Force -Path '{dir_name}'")
+
+        tar_stream = io.BytesIO()
+        with tarfile.open(fileobj=tar_stream, mode='w') as tar:
+            encoded_data = content.encode('utf-8')
+            tarinfo = tarfile.TarInfo(name=file_name)
+            tarinfo.size = len(encoded_data)
+            tar.addfile(tarinfo, io.BytesIO(encoded_data))
+


_copy_content_to_container assumes content is a str and unconditionally calls .encode('utf-8'), but File.content is defined as str | bytes. If a Windows container needs to copy binary content (e.g., JKS files/certs), this will fail. Consider accepting bytes | str and only encoding when the input is text.

Copilot · 2026-02-20T08:09:34Z

behave_framework/src/minifi_test_framework/core/ssl_utils.py

+import datetime
+from cryptography import x509
+from cryptography.x509.oid import NameOID
+from cryptography.hazmat.primitives import hashes
+from cryptography.hazmat.primitives.asymmetric import rsa
+from cryptography.hazmat.primitives import serialization


This file no longer includes the ASF license header that is present across the rest of the repository. Please restore the standard Apache 2.0 header at the top of the file to maintain licensing compliance.

Copilot · 2026-02-20T08:09:34Z

behave_framework/src/minifi_test_framework/core/ssl_utils.py

 def make_client_cert(common_name, ca_cert, ca_key):
-    return _make_cert(common_name=common_name, ca_cert=ca_cert, ca_key=ca_key, extended_key_usage=b"clientAuth")
+    return _make_cert(common_name, ca_cert, ca_key, [x509.OID_CLIENT_AUTH])


 def make_server_cert(common_name, ca_cert, ca_key):
-    return _make_cert(common_name=common_name, ca_cert=ca_cert, ca_key=ca_key, extended_key_usage=b"serverAuth")
+    return _make_cert(common_name, ca_cert, ca_key, [x509.OID_SERVER_AUTH])



x509.OID_CLIENT_AUTH / x509.OID_SERVER_AUTH are not valid identifiers in cryptography.x509 (extended key usage OIDs are exposed via cryptography.x509.oid.ExtendedKeyUsageOID). As written, these calls will raise at runtime when creating certs. Switch to the correct ExtendedKeyUsageOID constants (or pass ObjectIdentifier instances) so certificate generation works.

Copilot · 2026-02-20T08:09:35Z

run_flake8.sh

 directory=${1:-.}
-flake8 --exclude .venv,venv,thirdparty,build,cmake-build-*,github_env --builtins log,REL_SUCCESS,REL_FAILURE,REL_ORIGINAL,raw_input --ignore E501,W503,F811 "${directory}"
+flake8 --exclude .venv,venv,venv2,behave-venv,thirdparty,build,cmake-build-*,github_env --builtins log,REL_SUCCESS,REL_FAILURE,REL_ORIGINAL,raw_input --ignore E501,W503,F811 "${directory}"


The repo’s Behave test virtualenv is created in ./behave_venv (see docker/RunBehaveTests.sh), but flake8 excludes behave-venv (dash). This means flake8 will still traverse the Behave venv by default. Consider excluding behave_venv here (and keep/remove behave-venv depending on what is actually used).

Copilot · 2026-02-20T08:09:35Z

docker/installed/win.Dockerfile

+
+SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
+
+RUN Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))


Downloading and executing the Chocolatey install script with iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) introduces a supply-chain remote code execution risk: if community.chocolatey.org or the connection is compromised, arbitrary code can run during the Docker build with high privileges. To harden this, rely on a trusted, versioned installation source (e.g., a base image or offline package) and/or verify the installer’s integrity via a pinned checksum or signature instead of executing it directly from the remote URL.

martinzink added low-impact Test only or trivial change that's most likely not gonna introduce any new bugs depends-on-another-PR labels Feb 12, 2026

martinzink added this to the Rust bindings milestone Feb 12, 2026

martinzink added a commit that referenced this pull request Feb 12, 2026

MINIFICPP-2718 Windows based docker tests

0e402e4

Closes #2106 Signed-off-by: Martin Zink <martinzink@apache.org>

martinzink added a commit that referenced this pull request Feb 12, 2026

MINIFICPP-2718 Windows based docker tests

9413e63

Closes #2106 Signed-off-by: Martin Zink <martinzink@apache.org>

martinzink marked this pull request as ready for review February 12, 2026 16:57

MINIFICPP-2718 Windows based docker tests

ea3f9de

martinzink force-pushed the MINIFICPP-2718 branch from 32a11ab to ea3f9de Compare February 12, 2026 18:00

martinzink added a commit that referenced this pull request Feb 19, 2026

MINIFICPP-2718 Windows based docker tests

a6128ab

Closes #2106 Signed-off-by: Martin Zink <martinzink@apache.org>

martinzink requested a review from Copilot February 20, 2026 08:03

Copilot started reviewing on behalf of martinzink February 20, 2026 08:03 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes

martinzink added 2 commits February 20, 2026 10:32

win docker improvements

d65c896

review changes

24571fc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MINIFICPP-2718 Windows based docker tests#2106

MINIFICPP-2718 Windows based docker tests#2106
martinzink wants to merge 3 commits intoapache:MINIFICPP-2711from
martinzink:MINIFICPP-2718

martinzink commented Feb 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

	print(f"Creating and starting container '{self.container_name}'...")
	logging.info(f"Creating and starting container '{self.container_name}'...")


		SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

		RUN Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

Conversation

martinzink commented Feb 12, 2026

For all changes:

For code changes:

For documentation related changes:

Note:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments