fix: properly handle trailing newlines in lucene and add more tests

[mhoff]

Description

• properly handle trailing newlines in lucene
• add more tests regarding the usage of (escaped) special characters
• ensure test_requester fails for not triggering filter rules

Assignee

• The changes adhere to the contribution guidelines
• I have performed a self-review of my code
• My changes generate no new warnings (e.g. flake8/mypy/pytest/...) other than deprecations

Documentation

• README.md is up-to-date
• CHANGELOG.md is up-to-date
• Documentation (doc/source) is up-to-date
• Preview for docs looks fine
• Notebooks are up-to-date and functional
• Examples are up-to-date and functional (including compose & k8s)

Code Quality

• Patch test coverage > 95% and does not decrease
• New code uses correct & specific type hints

How did you verify that the changes work in practice?

• pytest

Reviewer

• The code is readable/maintainable and follows the contribution guidelines
• Documentation is up-to-date
• Tests (unit & acceptance) are meaningful and not over-mocked

---

The rendered docs for this PR can be found here.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.20%. Comparing base (78c0f13) to head (bb87566).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #948      +/-   ##
==========================================
- Coverage   96.21%   96.20%   -0.01%     
==========================================
  Files         218      218              
  Lines       14373    14373              
==========================================
- Hits        13829    13828       -1     
- Misses        544      545       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Pablu23]

Looks good to me. Thank you for all the tests, but please do explain what the difference is between \Z and $ because as I thought and a quick google would also agree, they should be equivalent?

[mhoff]

Looks good to me. Thank you for all the tests, but please do explain what the difference is between \Z and $ because as I thought and a quick google would also agree, they should be equivalent?

The regex is meant to find (the potentially escaped) end of the string. But $ matches trailing newlines as well, which the user intends to use as part of the payload. This leads to trailing newlines being stripped from the payload.

https://sethmlarson.dev/regex-%24-matches-end-of-string-or-newline#:~:text=Pattern%20matches
https://sethmlarson.dev/use-backslash-A-and-z-not-%5E-and-%24-with-python-regular-expressions

[mhoff]

@Pablu23 could you please also tick the review tasks in the PR description?