Skip to content

Upgrade schemathesis to v3.0.0 and fix all test failures#370

Open
cmil wants to merge 7 commits into
mainfrom
328-schemathesis-upgrade
Open

Upgrade schemathesis to v3.0.0 and fix all test failures#370
cmil wants to merge 7 commits into
mainfrom
328-schemathesis-upgrade

Conversation

@cmil

@cmil cmil commented Jun 27, 2026

Copy link
Copy Markdown
Member

Summary

  • Upgrades the schemathesis GitHub Action to v3.0.0 (pinned by commit hash)
  • Switches from negative to positive testing mode (--mode positive) to prevent HTTP method fuzzing (e.g. QUERY) that caused eXist-db 500 errors
  • Raises request timeout to 30 s to handle slow Wikidata SPARQL responses
  • Fixes the API bugs uncovered by the upgraded test suite

API fixes

  • /character/{id} — constrains the id path parameter to ^Q[0-9]+ so non-Wikidata IDs are rejected with 400 rather than passing schema validation
  • /corpora?include= — validates the include query parameter server-side; unknown values return 400
  • /dts/collection?nav=parents — fixes a 500 cardinality error when a play ID is not in the test corpus; returns 404 instead; restricts the nav parameter enum to [parents]
  • /dts/navigation and /dts/document — makes cross-parameter handling more lenient: ref takes precedence over start/end; incomplete ranges (one of start/end missing) are silently ignored; bare resource without any navigation parameter defaults to down=1
  • /dts/document and /dts/navigation — adds minLength: 1 to the resource parameter schema so schemathesis does not generate empty-string inputs

Test infrastructure

  • Adds test/test.json (missing corpus fixture used by CI)
  • Updates test/schemathesis.sh to match CI flags

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown

Merging this PR will leave 6 of 35 operations untested. Please review.

❌ 6 untested
🔴 6 partial
✅ 23 tested

Operations  █████████████████░░░  29/35    83%
Parameters  ██████████████░░░░░░  48/70    69%
Keywords    ████████████░░░░░░░░  55/89    62%
Responses   ████████░░░░░░░░░░░░  31/76    41%
Examples    ████████████████████    4/4   100%

❌ Untested (6)

Method Path Parameters Keywords Responses Examples
DELETE /exist/restxq/v1/corpora/{corpusname} 0/1 0/1 0/2 0/0
DELETE /exist/restxq/v1/corpora/{corpusname}/plays/{playname} 0/2 0/2 0/2 0/0
POST /exist/restxq/v1/corpora 0/2 0/8 0/3 0/0
POST /exist/restxq/v1/corpora/{corpusname} 0/2 0/5 0/3 0/0
POST /exist/restxq/v1/webhook/github 0/5 0/7 0/1 0/0
PUT /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/tei 0/3 0/4 0/3 0/0

🔴 Tested (29)

Method Path Parameters Keywords Responses Examples
🔴 GET /exist/restxq/v1/character/{id} 0/1 1/2 1/3 0/0
🔴 GET /exist/restxq/v1/corpora 0/1 1/2 1/1 0/0
🟡 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/spoken-text 6/7 7/8 1/3 0/0
🟡 GET /exist/restxq/v1/dts/collection 1/2 2/3 2/4 0/0
🟡 GET /exist/restxq/v1/dts/document 3/4 4/5 1/4 2/2
🟡 GET /exist/restxq/v1/dts/navigation 3/5 5/7 1/4 2/2
🟢 GET /exist/restxq/v1/corpora/{corpusname} 1/1 1/1 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/metadata 1/1 1/1 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/metadata/csv 1/1 1/1 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname} 2/2 2/2 1/2 0/0
19 more
Method Path Parameters Keywords Responses Examples
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/characters 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/characters/csv 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/metrics 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/csv 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/gexf 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/graphml 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/csv 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/gexf 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/graphml 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/spoken-text-by-character 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/stage-directions 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/stage-directions-with-speakers 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/tei 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/txt 2/2 2/2 1/2 0/0
🟢 GET /exist/restxq/v1/id/{id} 1/1 1/1 1/2 0/0
🟢 GET /exist/restxq/v1/wikidata/author/{id} 1/1 1/1 2/2 0/0
🟢 GET /exist/restxq/v1/dts 0/0 0/0 1/1 0/0
🟢 GET /exist/restxq/v1/info 0/0 0/0 1/1 0/0
🟢 GET /exist/restxq/v1/wikidata/mixnmatch 0/0 0/0 1/1 0/0

Powered by TraceCov v0.21.2

schemathesis 4.22.3 | http://localhost:8081/exist/restxq/v1/openapi.yaml

Download full HTML report

@cmil cmil requested a review from ingoboerner June 27, 2026 09:01
@cmil cmil force-pushed the 328-schemathesis-upgrade branch 3 times, most recently from efe2714 to b605432 Compare June 27, 2026 13:01
cmil added 7 commits July 3, 2026 15:53
Upgrade schemathesis/action from v1.1.1 to v3.0.0 (pinned by SHA),
using the new dedicated `authorization` input and dropping the obsolete
`--workers` flag.

Fix server errors exposed by schemathesis v2+ when duplicate query
parameters (e.g. ?sex=null&sex=null) are sent: RESTXQ binds these as a
sequence, causing XQuery runtime errors in boolean and string contexts.
Normalize all affected query params to a single value with [1] at the
start of each function body in api.xqm and dts.xqm.

Closes #328
- Switch schemathesis to --mode positive to suppress QUERY-method fuzzing
  that caused eXist-db 500 responses
- Add --request-timeout 30 to handle slow Wikidata SPARQL responses
- Constrain /character/{id} to pattern ^Q[0-9]+ to reject non-Wikidata IDs
- Validate ?include= param in /corpora, return 400 on unknown values
- Fix /dts/collection?nav=parents 500 by guarding against missing corpus
  (returns 404) and restricting nav enum to [parents]
- Ignore incomplete ref/start/end combinations in DTS document and
  navigation endpoints instead of returning 400
- Default DTS navigation to down=1 when no navigation param is supplied
- Add minLength: 1 to DTS resource param to prevent empty-string inputs
Since we may introduce other test suites (e.g. DTS) in the future
a more specific name seems like a good idea.
@cmil cmil force-pushed the 328-schemathesis-upgrade branch from 7c1d061 to eba8e16 Compare July 3, 2026 13:57
@cmil cmil added this to the 1.4.0 milestone Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant