Skip to content

fix(conformance): correct SEP-1330 enum schemas in test_elicitation_sep1330_enums #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kpavlov merged 35 commits into from
Mar 12, 2026
Merged

fix(conformance): correct SEP-1330 enum schemas in test_elicitation_sep1330_enums #600

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
3c36239
add conformance tests for server and client
devcrocod Mar 2, 2026
42d14f0
rewrite conformance tests, rework server-client implementation
devcrocod Mar 5, 2026
21846f5
add OAuth authorization and client credentials scenarios to conforman…
devcrocod Mar 5, 2026
c241bda
add OAuth authorization and client credentials scenarios to conforman…
devcrocod Mar 6, 2026
4c7900a
replace `System.err` with KotlinLogging in ConformanceClient, add bas…
devcrocod Mar 6, 2026
aebff7a
remove toolLists from basic client
devcrocod Mar 6, 2026
6e30638
update protocol version constants
devcrocod Mar 6, 2026
09ff656
remove unused `allowedOrigins` configuration from ConformanceServer s…
devcrocod Mar 6, 2026
0e80c11
add support for baseline file of expected failures in conformance tests
devcrocod Mar 6, 2026
b37f568
update elicitation schema to include `username` and `email`, adjust r…
devcrocod Mar 6, 2026
70ace75
delete `results` directory during `clean` task in conformance-test build
devcrocod Mar 6, 2026
928c68e
conditionally apply Detekt plugin to exclude conformance-test
devcrocod Mar 6, 2026
30ab88e
add resource parameter support and pre-registration in OAuth authoriz…
devcrocod Mar 6, 2026
914b531
remove OAuth authorization handler and simplify resource metadata dis…
devcrocod Mar 6, 2026
1ba27f8
add CSRF state parameter validation and enhance error handling in OAu…
devcrocod Mar 7, 2026
7c3b6ce
remove redundant non-null assertions in authCodeFlow implementation
devcrocod Mar 7, 2026
d0c0d96
add new expected failures to conformance test baseline
devcrocod Mar 7, 2026
5369a57
specify explicit types for cached discovery and credentials in authCo…
devcrocod Mar 7, 2026
451940f
improve error handling in `run-conformance.sh` by adding fallback for…
devcrocod Mar 7, 2026
94dfe80
update README to reflect 18 OAuth scenarios in client-auth suite and …
devcrocod Mar 7, 2026
c465ffe
add Cross-App Access scenario and validate PRM resource in authCodeFl…
devcrocod Mar 7, 2026
c7695d3
fix: handle sse 404 (#593)
devcrocod Mar 9, 2026
d9e82b2
fix: serialization empty response without id (#592)
devcrocod Mar 9, 2026
5ef15e9
add HTTP timeout configuration to ConformanceClient for improved requ…
devcrocod Mar 9, 2026
6e9c082
refactor ConformanceClient to centralize HTTP client creation with sh…
devcrocod Mar 9, 2026
7c1a1cf
parameterize JDK and Node.js versions in workflow, add new auth scena…
devcrocod Mar 9, 2026
4892b7c
add support for additional auth scenarios in conformance tests and re…
devcrocod Mar 9, 2026
0a3b4bd
feat: add SSE reconnection with retry support (#596)
devcrocod Mar 11, 2026
6f1ae61
feat(conformance): add `list` command to conformance test script
kpavlov Mar 11, 2026
3c097bd
feat(conformance): add logging to `test_tool_with_logging` and update…
kpavlov Mar 12, 2026
c1f1818
fix(server): keep SSE connection open until explicitly cancelled
kpavlov Mar 12, 2026
7ec51bd
chore(conformance): remove 5 passing server tests from conformance ba…
kpavlov Mar 12, 2026
970148a
fix(conformance): correct SEP-1330 enum schemas in test_elicitation_s…
kpavlov Mar 12, 2026
98e9485
Merge branch 'main' into kpavlov/fix-elicitation-test
kpavlov Mar 12, 2026
cde9364
docs: clean up conformance-tests README by removing outdated limitations
kpavlov Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions conformance-test/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,15 +111,8 @@ Tests the conformance server against all server scenarios:

## Known SDK Limitations

8 scenarios are expected to fail due to current SDK limitations (tracked in [
`conformance-baseline.yml`](conformance-baseline.yml).

| Scenario | Suite | Root Cause |
|---------------------------------------|--------|----------------------------------------------------------------------------------------------------------------|
| `elicitation-sep1330-enums` | server | *(same as above)* |
| `resources-templates-read` | server | SDK does not implement `addResourceTemplate()` with URI pattern matching; resources are looked up by exact URI |
| `elicitation-sep1034-client-defaults` | client | SDK does not fill in `default` values from the elicitation request schema before sending the response |

Comment on lines -117 to -122
Copy link
Copy Markdown
Contributor

@devcrocod devcrocod Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it was worth removing this from the readme, since it’s more visible there and includes a brief description of the issue

Copy link
Copy Markdown
Contributor

@devcrocod devcrocod Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, now we have two failed scenarios, so I don't think it’s a big problem

Copy link
Copy Markdown
Contributor Author

@kpavlov kpavlov Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs a bit more care, so it might not be the most valuable thing.

Scenarios, expected to fail due to current SDK limitations, are tracked in
[`conformance-baseline.yml`](conformance-baseline.yml).
These failures reveal SDK gaps and are intentionally not fixed in this module.

[list-scenarios-command]: https://github.com/modelcontextprotocol/conformance/tree/main?tab=readme-ov-file#list-available-scenarios
1 change: 0 additions & 1 deletion conformance-test/conformance-baseline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Conformance test baseline - expected failures
# Add entries here as tests are identified as known SDK limitations
server:
- elicitation-sep1330-enums
- resources-templates-read

client:
Expand Down
30 changes: 12 additions & 18 deletions ...e-test/src/main/kotlin/io/modelcontextprotocol/kotlin/sdk/conformance/ConformanceTools.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,27 +318,22 @@ fun Server.registerConformanceTools() {
)
},
)
// Legacy titled (deprecated)
// Legacy titled (deprecated) - uses enum + enumNames arrays
put(
"legacyEnum",
buildJsonObject {
put("type", JsonPrimitive("string"))
put(
"oneOf",
"enum",
JsonArray(listOf(JsonPrimitive("opt1"), JsonPrimitive("opt2"), JsonPrimitive("opt3"))),
)
put(
"enumNames",
JsonArray(
listOf(
buildJsonObject {
put("const", JsonPrimitive("opt1"))
put("title", JsonPrimitive("Option One"))
},
buildJsonObject {
put("const", JsonPrimitive("opt2"))
put("title", JsonPrimitive("Option Two"))
},
buildJsonObject {
put("const", JsonPrimitive("opt3"))
put("title", JsonPrimitive("Option Three"))
},
JsonPrimitive("Option One"),
JsonPrimitive("Option Two"),
JsonPrimitive("Option Three"),
),
),
)
Expand Down Expand Up @@ -367,17 +362,16 @@ fun Server.registerConformanceTools() {
)
},
)
// Titled multi-select
// Titled multi-select - uses items.anyOf with const/title pairs
put(
"titledMulti",
buildJsonObject {
put("type", JsonPrimitive("array"))
put(
"items",
buildJsonObject {
put("type", JsonPrimitive("string"))
put(
"oneOf",
"anyOf",
JsonArray(
listOf(
buildJsonObject {
Expand Down Expand Up @@ -405,7 +399,7 @@ fun Server.registerConformanceTools() {
"Please review and update the form fields with defaults",
schema,
)
CallToolResult(listOf(TextContent(result.content.toString())))
CallToolResult(listOf(TextContent("Elicitation completed: action=${result.action}, content=${result.content}")))
}

// 12. Dynamic tool
Expand Down
Loading