Fix misleading error in private_encrypt with default OAEP padding#177
Merged
timlegge merged 3 commits intocpan-authors:mainfrom Apr 23, 2026
Conversation
Tests that private_encrypt with OAEP/PSS padding produces a padding compatibility error rather than a misleading "plaintext too long" error. These tests currently FAIL — the fix follows in the next commit. Also makes OAEP/PSS rejection tests version-independent (previously only tested on OpenSSL 3.x). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When private_encrypt() is called with the default OAEP padding (or PSS), the padding is fundamentally incompatible with the operation. Previously, check_max_message_length() ran first and could produce a misleading "plaintext too long for key size with OAEP padding" error — hiding the real issue that OAEP/PSS aren't supported for private_encrypt at all. Now validates padding mode before checking message length, ensuring the user sees "OAEP padding is not supported for private_encrypt/public_decrypt" regardless of data size. This also extends the OAEP/PSS rejection to pre-3.x OpenSSL (previously only checked in the 3.x code path inside rsa_crypt()). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
timlegge
requested changes
Apr 23, 2026
| "public_decrypt with OAEP croaks with clear message on OpenSSL 3.x"); | ||
| } else { | ||
| ok(1, "public_decrypt with OAEP behavior on pre-3.x (skipped)"); | ||
| like($@, qr/./, |
The public_decrypt tests used qr/./ (matches any string) for the pre-3.x path because the error message was unpredictable. Fix the root cause: add the same OAEP/PSS guards to public_decrypt() that private_encrypt() already has, making errors consistent across all OpenSSL versions. Remove the now-unnecessary version branching. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
|
Fixed. The Added the same padding guards to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Validates padding compatibility (OAEP/PSS rejection) in
private_encrypt()before checking message length.Why
make_rsa_obj()defaults to OAEP padding. When a user callsprivate_encrypt()without changing padding, and the data exceeds the OAEP length limit (key_size - 42),check_max_message_length()fires first with "plaintext too long for key size with OAEP padding". This is misleading — the real problem isn't data length, it's that OAEP is fundamentally incompatible withprivate_encrypt. The user needs to calluse_pkcs1_padding()oruse_no_padding(), not shrink their data.How
Added explicit OAEP/PSS padding checks in
private_encrypt()beforecheck_max_message_length(). This duplicates the checks already present inrsa_crypt()'s OpenSSL 3.x path, but ensures correct error ordering and extends coverage to pre-3.x OpenSSL where these checks didn't exist.Testing
private_encrypt.🤖 Generated with Claude Code
Quality Report
Changes: 3 files changed, 49 insertions(+), 60 deletions(-)
Code scan: clean
Tests: passed (OK)
Branch hygiene: clean
Generated by Kōan post-mission quality pipeline