Skip to content

Commit 2071739

Browse files
Hackerboneclaude
andcommitted
Expand networkidle exclusion list to cover analytics, tracking, and heartbeats
Add Google Analytics, GTM, Hotjar, FullStory, LogRocket, Mouseflow, Clarity, Datadog, Sentry, New Relic, Forter, and generic heartbeat/keepalive/beacon patterns. These continuously-polling endpoints prevent networkidle from firing on real-world sites like partner.getyourguide.com (Looker heartbeats, Forter events, Datadog telemetry). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent debbeeb commit 2071739

2 files changed

Lines changed: 3 additions & 3 deletions

File tree

patch_driver_networkidle.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
import sys
1717
from pathlib import Path
1818

19-
CAPTCHA_PATTERNS_JS = '["challenges.cloudflare.com","google.com/recaptcha","www.gstatic.com/recaptcha","hcaptcha.com","api.funcaptcha.com","client-api.arkoselabs.com"]'
19+
CAPTCHA_PATTERNS_JS = '["challenges.cloudflare.com","google.com/recaptcha","www.gstatic.com/recaptcha","hcaptcha.com","api.funcaptcha.com","client-api.arkoselabs.com","google-analytics.com","googletagmanager.com","analytics.google.com","hotjar.com","fullstory.com","logrocket.com","mouseflow.com","clarity.ms","browser-intake-datadoghq.com","sentry.io","newrelic.com","nr-data.net","forter.com","/heartbeat","/keepalive","/keep-alive","/beacon"]'
2020

2121
CAPTCHA_CHECK = f"""const _reqUrl = request.url();
2222
if ({CAPTCHA_PATTERNS_JS}.some(p => _reqUrl.includes(p)))

patch_python_package.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -588,9 +588,9 @@ async def route_handler(route: Route) -> None:
588588
NETWORKIDLE_PATCH_CODE = '''
589589
import re as _re
590590
591-
_CAPTCHA_PATTERNS_JS = '["challenges.cloudflare.com","google.com/recaptcha","www.gstatic.com/recaptcha","hcaptcha.com","api.funcaptcha.com","client-api.arkoselabs.com"]'
591+
_EXCLUDED_PATTERNS_JS = '["challenges.cloudflare.com","google.com/recaptcha","www.gstatic.com/recaptcha","hcaptcha.com","api.funcaptcha.com","client-api.arkoselabs.com","google-analytics.com","googletagmanager.com","analytics.google.com","hotjar.com","fullstory.com","logrocket.com","mouseflow.com","clarity.ms","browser-intake-datadoghq.com","sentry.io","newrelic.com","nr-data.net","forter.com","/heartbeat","/keepalive","/keep-alive","/beacon"]'
592592
_PATCH_MARKER = '// [patchright-networkidle-blacklist]'
593-
_CAPTCHA_CHECK = f'const _reqUrl = request.url();\\n if ({_CAPTCHA_PATTERNS_JS}.some(p => _reqUrl.includes(p)))\\n return;'
593+
_CAPTCHA_CHECK = f'const _reqUrl = request.url();\\n if ({_EXCLUDED_PATTERNS_JS}.some(p => _reqUrl.includes(p)))\\n return;'
594594
595595
def _patch_networkidle_blacklist(driver_root):
596596
for dirpath, _, filenames in os.walk(driver_root):

0 commit comments

Comments
 (0)