KAFKA-20330: Ack handling improvement on broker restart#21824
Open
AndrewJSchofield wants to merge 3 commits intoapache:trunkfrom
Open
KAFKA-20330: Ack handling improvement on broker restart#21824AndrewJSchofield wants to merge 3 commits intoapache:trunkfrom
AndrewJSchofield wants to merge 3 commits intoapache:trunkfrom
Conversation
Collaborator
|
Could you please rebase/merge trunk to resolve the conflicts? |
3687516 to
8b53e4b
Compare
DL1231
reviewed
Mar 20, 2026
...nts-integration-tests/src/test/java/org/apache/kafka/clients/consumer/ShareConsumerTest.java
Outdated
Show resolved
Hide resolved
...ts/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java
Show resolved
Hide resolved
DL1231
approved these changes
Mar 22, 2026
Collaborator
DL1231
left a comment
There was a problem hiding this comment.
Thanks fot the update, LGTM.
apoorvmittal10
approved these changes
Mar 30, 2026
Contributor
apoorvmittal10
left a comment
There was a problem hiding this comment.
Thanks for the patch, LGTM!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A share consumer uses a share session to keep track of its acquired
records with each share-partition leader it is talking to. When the
connection breaks, the share session is lost and acknowledgements fail.
When the connection to a share-partition leader failed while there was
an outstanding request, the share consumer noticed the disconnection and
failed the acknowledgements as expected. Also, if the leader changed to
a different broker, again the share consumer noticed the leadership
change and failed the acknowledgements as expected.
In the situation where the share-partition leadership did not change
when the broker restarted AND there was no in-flight request, the share
consumer did not notice the disconnection and would try to continue the
share session. There were also a few situations in which
acknowledgements could be lost and not notified to the acknowledgement
commit callback to do with this kind of leadership non-transition.
This PR improves the situation by using a consistent exception
NotLeaderOrFollowerExceptionregardless of when the disconnection wasnoticed. It also makes sure that acknowledgements which cannot be sent
are completed properly in all cases.
It is possible that a tweak to the protocol will be needed to eliminate
ShareSessionNotFoundExceptionin all edge cases, but that would take aKIP.