ci: subprocess + http + unix transport SQL-test coverage#1
Merged
Conversation
Run the test/sql suite over all three VGI transports in CI: - pyproject: add an `http` optional-dependency extra re-stating `vgi-python[http]` (pulls in waitress for `vgi-match --http`); relock so uv.lock carries waitress. - ci/run-integration.sh: parameterize by $TRANSPORT (subprocess|http|unix, default subprocess). For http boot `vgi-match --http --port 0 --port-file <f>` (cwd = stage dir), poll the port-file, LOCATION = http://127.0.0.1: <port>, and inject `INSTALL httpfs FROM core; LOAD httpfs;` after each `LOAD vgi;` (http leg only — required or the ATTACH errors and the runner silently skips). For unix boot `vgi-match --unix <sock>` and poll the socket. cleanup() preserves the real exit code; the run step fails the leg on a "All tests were skipped" silent-skip fake-pass. - .github/workflows/ci.yml: integration job becomes a transport x os matrix (subprocess/http/unix on ubuntu/macos); install with `--extra http`. - ci/README.md: document the three transports. The match_resolve TableBufferingFunction emits its full result in finalize as a single small batch (output = input rows + cluster_id + match_probability), so the DrainState(done) cursor is HTTP-safe — the output never exceeds the producer batch limit, so there is no mid-stream continuation. DrainState already extends ArrowSerializableDataclass. No cursor fix needed. Verified locally over all three transports against haybarn-unittest: subprocess/unix 12 assertions, http 14 (the 2 injected httpfs statements). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run the full
test/sqlsuite over all three VGI transports (subprocess, http, unix) on ubuntu + macos in CI.Changes
httpextra re-statingvgi-python[http](waitress); relock.$TRANSPORT. http bootsvgi-match --http --port 0 --port-file <f>(cwd = stage dir) + injectsINSTALL httpfs FROM core; LOAD httpfs;; unix bootsvgi-match --unix <sock>. Silent-skip guard + exit-code-preserving cleanup.transport × osmatrix; install--extra http.Streaming note
match_resolveis aTableBufferingFunctionthat emits its full result (input rows +cluster_id+match_probability) as a single small batch infinalize, so theDrainState(done)cursor is HTTP-safe — output never exceeds the producer batch limit, so no mid-stream continuation occurs.DrainStatealready extendsArrowSerializableDataclass. No cursor fix needed.Verified locally over all three transports against haybarn-unittest.
🤖 Generated with Claude Code