test: A2A ITK interoperability harness + baseline (no SDK changes)#35
Open
zeroasterisk wants to merge 1 commit into
Open
test: A2A ITK interoperability harness + baseline (no SDK changes)#35zeroasterisk wants to merge 1 commit into
zeroasterisk wants to merge 1 commit into
Conversation
Adds a test-harness that drives the unmodified A2A Elixir SDK against the
official A2A Interoperability Test Kit (ITK), plus an honest capability/gap
baseline report. This is a measuring-stick PR — no changes to lib/.
Harness:
- test/support/itk/instruction.ex ITK Instruction protobuf codec
- test/support/itk/agent.ex JSON-RPC handler / instruction interpreter
- test/itk/server.exs standalone Bandit server (v0.3 card, JSON-RPC, SSE)
- test/itk/{instruction,agent}_test.exs + fixtures
Report (docs/ITK_BASELINE.md) documents, with reproducible evidence:
- WORKS: v0.3 agent card, proto decode, interpreter, non-streaming message/send (404 tests green)
- GAPS: JSON-RPC enum encoding (ROLE_AGENT/TASK_STATE_* vs v0.3 agent/completed),
SSE streaming event envelopes, v1.0 vs v0.3 card shape
- Prioritized v1.0 gap list seeding future (separate) SDK work
Reference client: a2a-sdk 0.3.24.
Author
|
The plan is: Land this PR with dustin for v0.3 and 1.0. then upgrade. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ITK Interoperability Harness + Baseline (showcase — no SDK changes)
This PR adds a test-harness that drives the unmodified A2A Elixir SDK against the official A2A Interoperability Test Kit (ITK), plus an honest capability/gap baseline report. It's a measuring stick, not a fix — zero changes to
lib/.Why
Establish a reproducible baseline so future v1.0-compliance work is gap-driven and regression-gated, and so reviewers can see exactly what the SDK can/can't do against a real A2A client today.
What's added (harness + docs only)
test/support/itk/instruction.ex— ITK Instruction protobuf codectest/support/itk/agent.ex— JSON-RPC handler / instruction interpretertest/itk/server.exs— standalone Bandit server (v0.3 card, JSON-RPC, SSE)test/itk/{instruction,agent}_test.exs+ binary fixturesdocs/ITK_BASELINE.md— full capability/gap report with reproducible evidencegit diff --stat origin/main -- lib/is empty (verified).Baseline summary
✅ Works (404 unit tests green against pristine SDK):
preferredTransport: JSONRPC,protocolVersion: 0.3.0)return_response,steps,call_agent)message/sendround-trip❌ Gaps (vs
a2a-sdk 0.3.24Python client — documented, not fixed here):ROLE_AGENT/TASK_STATE_COMPLETED(proto-style); v0.3 client expectsagent/completedTaskStatusUpdateEvent/TaskArtifactUpdateEventunion (missingtaskId/kind)encode_agent_cardemits v1.0supportedInterfaces; v0.3 client wantspreferredTransport/additionalInterfacesPrioritized v1.0 gap list (seeds future, separate PRs)
Reproduce