Escalation Layers¶

recurl uses a layered approach to bypass anti-bot protection. Each layer is more powerful but also more resource-intensive.

Layer Overview¶

Layer 1: curl_engine¶

Standard curl request.

Uses the bundled upstream curl binary with the user's exact flags.

When It Works¶

• APIs without bot protection
• Static content (images, files, etc.)
• Sites with basic or no protection

When It Fails¶

• Sites checking TLS fingerprint (JA3/JA4)
• Sites with JavaScript challenges
• Sites with CAPTCHA gates

Example¶

recurl --recurl-debug https://httpbin.org/get
# [recurl] curl_engine: 200 OK
# No escalation needed

Layer 2: Impersonation¶

Browser TLS fingerprint mimicry.

Uses curl-impersonate - a modified curl that matches browser TLS signatures.

How It Works¶

curl-impersonate modifies:

• TLS handshake - cipher suites, extensions, curves
• HTTP/2 settings - SETTINGS frame values, pseudo-header order
• Headers - browser-matching User-Agent

The server sees a TLS fingerprint identical to a real browser.

Available Profiles¶

When It Works¶

• Sites using JA3/JA4 fingerprinting
• Basic TLS-based bot detection
• No JavaScript execution required

When It Fails¶

• Sites with JavaScript challenges
• Sites that require cookie-based verification
• Sites with behavioral analysis

Example¶

recurl --recurl-debug https://nowsecure.nl
# [recurl] curl_engine: 403 Cloudflare
# [recurl] Escalating: impersonation (chrome)
# [recurl] curl_chrome: 200 OK

Force Impersonation¶

# Skip curl_engine, go directly to impersonation
recurl --recurl-impersonate chrome https://example.com

# Use specific profile
recurl --recurl-impersonate firefox https://example.com

Layer 3: JS Preflight¶

Headless browser execution with cookie replay.

Launches a real Chromium browser to solve JavaScript challenges, then replays the request with curl using extracted cookies.

How It Works¶

1. Launch headless Chromium
2. Navigate to target URL
3. Wait for JS challenges to resolve
   - Cloudflare Turnstile
   - JavaScript fingerprinting
   - Cookie generation
4. Extract: cookies, final URL, headers
5. Close browser
6. Replay with curl_engine + extracted cookies

Stealth Features¶

recurl injects stealth patches to evade bot detection:

• navigator.webdriver → undefined
• Realistic navigator.plugins
• Chrome runtime objects
• WebGL vendor spoofing
• Console debug suppression

When It Works¶

• JavaScript challenges (Cloudflare, Akamai, etc.)
• Cookie-based verification
• Single-page applications
• Any site that works in a browser

When It May Fail¶

• CAPTCHA requiring human interaction
• Sites with advanced behavioral analysis
• Rate limiting at IP level

Example¶

recurl --recurl-debug https://site-with-js-challenge.com
# [recurl] curl_engine: 403 Cloudflare
# [recurl] Escalating: impersonation (chrome)
# [recurl] curl_chrome: 403 JS challenge
# [recurl] Escalating: JS preflight
# [recurl] JS preflight: starting
# [recurl] JS preflight: challenge detected, waiting...
# [recurl] JS preflight: success
# [recurl] JS preflight: extracted 3 cookies
# [recurl] Replaying with curl_engine + cookies
# [recurl] curl_engine: 200 OK

Force JS Preflight¶

# Skip all other layers
recurl --recurl-js https://spa-site.com

# Wait for specific element
recurl --recurl-js --recurl-js-wait ".content-loaded" https://spa-site.com

# Custom timeout
recurl --recurl-js --recurl-js-timeout 60000 https://slow-site.com

Get Rendered HTML¶

Instead of replaying with curl, return the DOM after JS execution:

recurl --recurl-js-rendered https://spa-site.com

Useful for single-page applications where the HTML is generated by JavaScript.

Daemon Integration¶

JS preflight benefits significantly from the daemon:

Control Daemon¶

# Force daemon usage
recurl --recurl-daemon on --recurl-js https://example.com

# Disable daemon (inline execution)
recurl --recurl-daemon off --recurl-js https://example.com

Platform Escalation Paths¶

Linux / macOS¶

curl_engine → Impersonation → JS Preflight

Full escalation chain available.

Windows¶

curl_engine → JS Preflight

Impersonation is skipped (curl-impersonate not available).

Choosing the Right Layer¶