Skip to content

perf: lazily wrap elements in find_by_text/find_by_regex so first_match short-circuits#370

Merged
D4Vinci merged 2 commits into
D4Vinci:devfrom
yetval:perf/lazy-wrap-first-match
Jul 11, 2026
Merged

perf: lazily wrap elements in find_by_text/find_by_regex so first_match short-circuits#370
D4Vinci merged 2 commits into
D4Vinci:devfrom
yetval:perf/lazy-wrap-first-match

Conversation

@yetval

@yetval yetval commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Problem

find_by_text/find_by_regex default to first_match=True but call
self.__elements_convertor(possible_targets) first, which eagerly wraps all N
elements into Selectors before the loop. So break saves nothing: wrapping is
O(N) instead of O(k), where k is the first match's index.

Fix

-for node in self.__elements_convertor(possible_targets):
+for node in map(self.__element_convertor, possible_targets):

Lazy wrapping, one element at a time. Singular __element_convertor builds each
Selector with identical kwargs, so output is unchanged; only the wrapping becomes
lazy. Nothing after the loop needs possible_targets to be a Selectors.

Result

Wrapping on first_match=True drops from O(N) to O(k). Output-identical
(early/last/no-match, both methods), existing suite green (94 passed).

Benchmark, minimum of 50 runs, page of N=5000 elements, times in milliseconds:

Method & match position for first_match=True Before (ms) After (ms) Speedup
find_by_text   match at index 5 (early) 3.51 1.82 ~1.9x faster
find_by_text   match at last index 4999 10.36 10.24 within noise
find_by_text   no match (full scan) 10.13 9.96 within noise
find_by_regex   match at index 5 (early) 3.58 1.84 ~1.9x faster
find_by_regex   match at last index 4999 12.27 12.19 within noise
find_by_regex   no match (full scan) 12.15 12.19 within noise

Early match is ~1.9x faster; last-match and no-match stay within noise (never slower).

@D4Vinci

D4Vinci commented Jul 11, 2026

Copy link
Copy Markdown
Owner

It seems like you are studying the code to find stuff like this ahaha. Nice work, mate!
Always a pleasure to review your accurate contributions!

@D4Vinci D4Vinci added enhancement New feature or request Memory Optimizations labels Jul 11, 2026
@D4Vinci D4Vinci merged commit 97ef582 into D4Vinci:dev Jul 11, 2026
5 checks passed
@yetval

yetval commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

It seems like you are studying the code to find stuff like this ahaha. Nice work, mate! Always a pleasure to review your accurate contributions!

Haha, yes, I have been studying!! Thank You!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants