Skip to content

Disable Kafka connector and gRPC, restrict backend to REST, add hygiene checks and tests#186

Closed
neuron7xLab wants to merge 16 commits intomainfrom
codex/locate-incomplete-scripts-and-tests
Closed

Disable Kafka connector and gRPC, restrict backend to REST, add hygiene checks and tests#186
neuron7xLab wants to merge 16 commits intomainfrom
codex/locate-incomplete-scripts-and-tests

Conversation

@neuron7xLab
Copy link
Copy Markdown
Owner

Motivation

  • Make unsupported ingestion surfaces explicit by removing Kafka/gRPC as supported runtime paths and ensure connectors fail closed rather than silently stubbing behavior.
  • Prevent placeholder/TBD tokens from leaking into generated stakeholder artifacts.
  • Add focused connector hygiene tests and wire them into the PR gate so connector regressions are caught automatically.

Description

  • Reworked Kafka handling by turning the old stub into a fail-closed implementation that raises KafkaConnectorUnavailableError in src/.../connectors/kafka_source.py and exported the explicit error.
  • Removed KafkaSourceConfig and Kafka options from src/.../connectors/config.py, restricted BackendConfig.protocol and IngestionConfig.source_type to supported values (rest/file), and added validation to reject unsupported protocols.
  • Simplified and hardened RemoteBackend in src/.../connectors/runner.py to be REST-only, added a protocol check that raises on non-REST protocols, and consolidated HTTP client usage.
  • Added placeholder detection to scripts/generate_stakeholder_assets.py via PLACEHOLDER_RE and assert_no_placeholders, and cleaned up a couple of TBD entries.
  • Added new tests: tests/connectors/test_fail_closed_connectors.py, tests/connectors/test_polygon_adapter_reproducible.py, and scripts/tests/test_generate_stakeholder_assets.py, and removed the legacy tests/unit/data/adapters/test_polygon.py.
  • Updated CI (.github/workflows/pr-gate.yml) to run the new mandatory connector hygiene tests as part of the PR gate.

Testing

  • Ran the fast unit test suite with pytest tests/ -q and the run completed successfully.
  • Executed the new connector hygiene tests with pytest tests/connectors/test_fail_closed_connectors.py tests/connectors/test_polygon_adapter_reproducible.py scripts/tests/test_generate_stakeholder_assets.py -q and they passed.
  • CI workflow updated to run the new connector hygiene tests as part of the PR gate and the added tests are included in that job.

Codex Task

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@neuron7xLab neuron7xLab enabled auto-merge (squash) April 9, 2026 14:20
@neuron7xLab neuron7xLab disabled auto-merge April 9, 2026 14:20
@neuron7xLab neuron7xLab closed this Apr 9, 2026
@neuron7xLab neuron7xLab deleted the codex/locate-incomplete-scripts-and-tests branch April 9, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant