All sites
Data API
Reddit Data API
Subreddits, posts, comments and search from across Reddit.
19endpointsCloudflare · Rate Limit handled for you
Example (captured live)
capzy.ai
ExampleJSON responseclick the page to trace a field
{
"query": "python",
"results": [
{
"id": "1vftpv1",
"fullname": "t3_1vftpv1",
"scraped_at": "2026-08-18T09:36:17.072274+00:00",
"title": "Solving the AWS WAF CAPTCHA and Token Challenge",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "AWS WAF ships a bot-control challenge that a lot of teams hit without knowing what it is. A page loads a small script (`challenge.js`, and a captcha bundle when the puzzle is interactive), the browser runs it, and only after that does the site hand you the content you asked for. The short version of the fix: you solve the challenge once, you receive an `aws-waf-token` cookie, and you carry that cookie on every request that follows. Everything else in this post is detail around those three steps.
The reason people get stuck is that they treat the token as the whole job. It is not. The token proves you cleared the challenge from a given client on a given IP, so a token minted from a burned datacenter address, or one paired with headers that do not match the client that earned it, gets rejected downstream. Solve the challenge, yes, but solve it from an environment the site is willing to trust.
# What AWS WAF actually serves
There are two flavors. The lightweight one is a silent challenge: `challenge.js` runs some checks in the background, and if it is satisfied you get the token without any user interaction. The heavier one is an interactive CAPTCHA: a puzzle bundle loads and you get a visual task, commonly a toss or rotate style puzzle where you orient an object correctly.
Both paths end at the same place. On success the response sets an `aws-waf-token` cookie. That cookie is the artifact you care about. AWS WAF checks it on later requests, and as long as it is present, valid, and coming from a consistent client and IP, the site serves you normally instead of re-challenging.
# Solve it with the Solver API
You send the challenge to the [Solver API](https://capzy.ai/solvers) with task type `AntiAwsWafTask` and the `websiteURL` of the page throwing the challenge. The API works the pattern every task uses: `/createTask` returns a task id, then you poll `/getTaskResult` until it is ready.
# 1) create the task
curl -s https://api.capzy.ai/createTask \
-H "Content-Type: application/json" \
-d '{
"clientKey": "YOUR_CLIENT_KEY",
"task": {
"type": "AntiAwsWafTask",
"websiteURL": "https://target.example.com/protected"
}
}'
# -> {"errorId":0,"taskId":"abc123"}
# 2) poll for the result
curl -s https://api.capzy.ai/getTaskResult \
-H "Content-Type: application/json" \
-d '{
"clientKey": "YOUR_CLIENT_KEY",
"taskId": "abc123"
}'
# -> {"errorId":0,"status":"ready","solution":{ ... aws-waf-token ... }}
The exact solution field names and any optional parameters (for a specific challenge subtype) are documented on the [solvers catalog](https://capzy.ai/solvers), so pull the current schema from there rather than hardcoding assumptions. The important mental model is: the solution gives you the `aws-waf-token` value, and your job is to set it as a cookie on the same session.
# Carry the token forward
Once you have the token, put it back where the browser would have: as the `aws-waf-token` cookie on your session, scoped to the target domain. Every subsequent request reuses it.
import httpx
token = solution["cookie"] # the aws-waf-token value from the solve
with httpx.Client(
headers={"user-agent": UA_THAT_MATCHES_THE_SOLVE},
cookies={"aws-waf-token": token},
) as client:
r = client.get("https://target.example.com/protected")
print(r.status_code)
Two things break people here. First, the User-Agent and header set you use afterward must match the client the token was minted for. A token earned by a Chrome-shaped client and then replayed from a bare Python client looks wrong, and AWS WAF is watching for exactly that kind of mismatch. Second, the token is IP-associated. Reuse it from the same egress you solved on. Solve on one address and replay from another and you have handed AWS WAF a clear signal that the token and the client parted ways.
A small operational detail saves a lot of debugging: set the cookie with the same domain and path scope the browser would have used, not a broader or narrower scope. A token scoped wrong is a token that silently does not get sent on the requests that need it, and then you are chasing a "rejected" token that was never actually attached.
>The token is necessary but not sufficient. AWS WAF is scoring the whole request, so a valid token from a burned IP or a mismatched client still gets bounced.
# Make the token stick: IP and fingerprint
This is where most AWS WAF failures actually live. If your egress IP is a well-known datacenter range that AWS already distrusts, you can solve the challenge cleanly and still get re-challenged on the next request. Route through residential-quality egress with the [Proxies API](https://capzy.ai/proxies) and keep the same IP for the life of the session so the token and the address stay paired.
Coherence matters just as much. The [Fingerprint API](https://capzy.ai/fingerprints) gives you a consistent client identity (User-Agent, headers, and the JS-surface signals that go with them) so the client that carries the token looks like the client that earned it. When you need a real browser in the loop rather than a raw HTTP client, drive the [Cloud Browser](https://capzy.ai/browser) or use [Capium](https://capzy.ai/capium) (`pip install capium`), our first-party stealth browser, so the challenge script runs in a genuine engine. Remember the split: the browser runs the page, the Solver API solves the challenge.
# When it still fails
If tokens keep getting rejected, work down this list. Confirm the `websiteURL` you send is the exact page that throws the challenge, not a redirect target. Confirm you are reusing the same IP across the solve and the follow-up requests. Confirm your headers did not drift after the solve. And confirm the token has not simply expired: `aws-waf-token` is not eternal, so on a long-running job you re-solve when the site starts challenging again.
Solving AWS WAF is a token-capture-and-carry problem, not a puzzle problem. Get the `aws-waf-token`, carry it on a consistent session over a clean IP, and the interactive puzzle becomes a detail the Solver API handles for you.
Ready to try it? [Create a free account](https://capzy.ai/auth/register) and run your first `AntiAwsWafTask` against the page that has been blocking you.",
"selftext_html": "<!-- SC_OFF --><div class="md"><p>AWS WAF ships a bot-control challenge that a lot of teams hit without knowing what it is. A page loads a small script (<code>challenge.js</code>, and a captcha bundle when the puzzle is interactive), the browser runs it, and only after that does the site hand you the content you asked for. The short version of the fix: you solve the challenge once, you receive an <code>aws-waf-token</code> cookie, and you carry that cookie on every request that follows. Everything else in this post is detail around those three steps.</p>
<p>The reason people get stuck is that they treat the token as the whole job. It is not. The token proves you cleared the challenge from a given client on a given IP, so a token minted from a burned datacenter address, or one paired with headers that do not match the client that earned it, gets rejected downstream. Solve the challenge, yes, but solve it from an environment the site is willing to trust.</p>
<h1>What AWS WAF actually serves</h1>
<p>There are two flavors. The lightweight one is a silent challenge: <code>challenge.js</code> runs some checks in the background, and if it is satisfied you get the token without any user interaction. The heavier one is an interactive CAPTCHA: a puzzle bundle loads and you get a visual task, commonly a toss or rotate style puzzle where you orient an object correctly.</p>
<p>Both paths end at the same place. On success the response sets an <code>aws-waf-token</code> cookie. That cookie is the artifact you care about. AWS WAF checks it on later requests, and as long as it is present, valid, and coming from a consistent client and IP, the site serves you normally instead of re-challenging.</p>
<h1>Solve it with the Solver API</h1>
<p>You send the challenge to the <a href="https://capzy.ai/solvers">Solver API</a> with task type <code>AntiAwsWafTask</code> and the <code>websiteURL</code> of the page throwing the challenge. The API works the pattern every task uses: <code>/createTask</code> returns a task id, then you poll <code>/getTaskResult</code> until it is ready.</p>
<pre><code># 1) create the task
curl -s https://api.capzy.ai/createTask \
-H &quot;Content-Type: application/json&quot; \
-d &#39;{
&quot;clientKey&quot;: &quot;YOUR_CLIENT_KEY&quot;,
&quot;task&quot;: {
&quot;type&quot;: &quot;AntiAwsWafTask&quot;,
&quot;websiteURL&quot;: &quot;https://target.example.com/protected&quot;
}
}&#39;
# -&gt; {&quot;errorId&quot;:0,&quot;taskId&quot;:&quot;abc123&quot;}
# 2) poll for the result
curl -s https://api.capzy.ai/getTaskResult \
-H &quot;Content-Type: application/json&quot; \
-d &#39;{
&quot;clientKey&quot;: &quot;YOUR_CLIENT_KEY&quot;,
&quot;taskId&quot;: &quot;abc123&quot;
}&#39;
# -&gt; {&quot;errorId&quot;:0,&quot;status&quot;:&quot;ready&quot;,&quot;solution&quot;:{ ... aws-waf-token ... }}
</code></pre>
<p>The exact solution field names and any optional parameters (for a specific challenge subtype) are documented on the <a href="https://capzy.ai/solvers">solvers catalog</a>, so pull the current schema from there rather than hardcoding assumptions. The important mental model is: the solution gives you the <code>aws-waf-token</code> value, and your job is to set it as a cookie on the same session.</p>
<h1>Carry the token forward</h1>
<p>Once you have the token, put it back where the browser would have: as the <code>aws-waf-token</code> cookie on your session, scoped to the target domain. Every subsequent request reuses it.</p>
<pre><code>import httpx
token = solution[&quot;cookie&quot;] # the aws-waf-token value from the solve
with httpx.Client(
headers={&quot;user-agent&quot;: UA_THAT_MATCHES_THE_SOLVE},
cookies={&quot;aws-waf-token&quot;: token},
) as client:
r = client.get(&quot;https://target.example.com/protected&quot;)
print(r.status_code)
</code></pre>
<p>Two things break people here. First, the User-Agent and header set you use afterward must match the client the token was minted for. A token earned by a Chrome-shaped client and then replayed from a bare Python client looks wrong, and AWS WAF is watching for exactly that kind of mismatch. Second, the token is IP-associated. Reuse it from the same egress you solved on. Solve on one address and replay from another and you have handed AWS WAF a clear signal that the token and the client parted ways.</p>
<p>A small operational detail saves a lot of debugging: set the cookie with the same domain and path scope the browser would have used, not a broader or narrower scope. A token scoped wrong is a token that silently does not get sent on the requests that need it, and then you are chasing a &quot;rejected&quot; token that was never actually attached.</p>
<blockquote>
<p>The token is necessary but not sufficient. AWS WAF is scoring the whole request, so a valid token from a burned IP or a mismatched client still gets bounced.</p>
</blockquote>
<h1>Make the token stick: IP and fingerprint</h1>
<p>This is where most AWS WAF failures actually live. If your egress IP is a well-known datacenter range that AWS already distrusts, you can solve the challenge cleanly and still get re-challenged on the next request. Route through residential-quality egress with the <a href="https://capzy.ai/proxies">Proxies API</a> and keep the same IP for the life of the session so the token and the address stay paired.</p>
<p>Coherence matters just as much. The <a href="https://capzy.ai/fingerprints">Fingerprint API</a> gives you a consistent client identity (User-Agent, headers, and the JS-surface signals that go with them) so the client that carries the token looks like the client that earned it. When you need a real browser in the loop rather than a raw HTTP client, drive the <a href="https://capzy.ai/browser">Cloud Browser</a> or use <a href="https://capzy.ai/capium">Capium</a> (<code>pip install capium</code>), our first-party stealth browser, so the challenge script runs in a genuine engine. Remember the split: the browser runs the page, the Solver API solves the challenge.</p>
<h1>When it still fails</h1>
<p>If tokens keep getting rejected, work down this list. Confirm the <code>websiteURL</code> you send is the exact page that throws the challenge, not a redirect target. Confirm you are reusing the same IP across the solve and the follow-up requests. Confirm your headers did not drift after the solve. And confirm the token has not simply expired: <code>aws-waf-token</code> is not eternal, so on a long-running job you re-solve when the site starts challenging again.</p>
<p>Solving AWS WAF is a token-capture-and-carry problem, not a puzzle problem. Get the <code>aws-waf-token</code>, carry it on a consistent session over a clean IP, and the interactive puzzle becomes a detail the Solver API handles for you.</p>
<p>Ready to try it? <a href="https://capzy.ai/auth/register">Create a free account</a> and run your first <code>AntiAwsWafTask</code> against the page that has been blocking you.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/aws-waf-captcha-solve",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vftpv1/solving_the_aws_waf_captcha_and_token_challenge/",
"url_overridden_by_dest": "https://capzy.ai/blog/aws-waf-captcha-solve",
"score": 2,
"upvote_ratio": 1,
"ups": 2,
"downs": 0,
"num_comments": 1,
"created_at": "2026-08-05T01:48:28+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/Phwrg4oH8IroY4esbIkN4-j3tgOfmD1DFARPIC3c3d0.jpeg?width=140&height=73&auto=webp&s=0edc6e1cc1107955f08112639b5f2d4270dab469",
"preview_image": "https://external-preview.redd.it/Phwrg4oH8IroY4esbIkN4-j3tgOfmD1DFARPIC3c3d0.jpeg?auto=webp&s=3898d0af9a14fd5fe20aa962dcd50f2cfa544a19",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vh8vh2",
"fullname": "t3_1vh8vh2",
"scraped_at": "2026-08-18T09:36:17.072306+00:00",
"title": "How to Solve Cloudflare Turnstile in 2026: A Developer's Guide",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "To solve Cloudflare Turnstile, you need a valid `cf-turnstile-response` token minted for the exact sitekey and origin the widget was placed on, and you need it to survive Cloudflare's server-side `siteverify` check. That is harder than it sounds in 2026, because Turnstile stopped being a checkbox a while ago and became a rolling behavioral challenge that watches your browser as much as your click.
This guide covers what the widget actually does, why a token that looks fine still gets rejected, and how to get a working token with a solving API. There is a code example you can copy near the end.
# What Turnstile is checking
Turnstile is Cloudflare's replacement for the old "I am not a robot" checkbox. It runs in one of three modes: a managed challenge that may show a checkbox, a non-interactive mode that shows a spinner and passes silently, and an invisible mode with no visible UI at all. In every mode it does the same core work. It loads a JavaScript challenge, runs a proof-of-work and a battery of browser environment checks, times how you interact with the page, and then mints a token if it decides you are legitimate.
The token lands in a hidden field named `cf-turnstile-response`. Your form submits it, and Cloudflare's backend calls its verify endpoint to confirm the token is real, unexpired, and matches the sitekey and hostname. The sitekey is public and lives right in the page HTML, usually as a `data-sitekey` attribute starting with `0x4AAAAAAA`.
Here is the part that trips people up. The token is not the whole picture. Turnstile embeds signals about the environment it ran in. If you mint a token in a browser that leaks automation, or from an IP with a bad reputation, Cloudflare can still reject it at verify time or feed you a harder challenge next round. A token is a claim; Cloudflare decides whether to believe it.
# Why your Turnstile tokens fail
When a token gets rejected, it is almost always one of these:
* **Wrong sitekey or origin.** Turnstile binds the token to the sitekey and the hostname it was issued for. Solve it against [`https://app.example.com`](https://app.example.com) and try to use it on [`https://www.example.com`](https://www.example.com) and it fails.
* **Expired token.** Turnstile tokens are short-lived, on the order of a few minutes. Mint, submit, done. Do not queue them.
* **Reused token.** Tokens are single-use. Cloudflare's verify endpoint remembers.
* **Dirty egress IP.** Even a technically valid token gets a colder reception from a flagged datacenter range. Cloudflare correlates the token's context with the IP that submits the form.
* **Fingerprint mismatch.** If the browser that minted the token looks nothing like the client that submits it, the signals do not line up.
That last pair is why "I solved the captcha but I am still blocked" is such a common complaint. You solved the widget. You did not solve the session.
>Turnstile grades the whole request, not just the click. A perfect token from a browser that leaks `navigator.webdriver` or from an IP that hosts a thousand other bots will still earn you a rougher ride.
# Solving Turnstile with an API
The reliable path is to let infrastructure that Cloudflare already trusts mint the token for you: a real browser engine with a coherent fingerprint, running from clean egress. A solving API wraps all of that. You submit the sitekey and page URL, it runs the challenge, and it returns `cf-turnstile-response`.
Here is a minimal Python example against Capzy:
import requests, time
API = "https://api.capzy.ai"
KEY = "YOUR_API_KEY"
# create the task
task = requests.post(f"{API}/createTask", json={
"clientKey": KEY,
"task": {
"type": "TurnstileTask",
"websiteURL": "https://example.com/login",
"websiteKey": "0x4AAAAAAABkMYinukE8nzY"
}
}).json()
task_id = task["taskId"]
# poll for the token
while True:
res = requests.post(f"{API}/getTaskResult", json={
"clientKey": KEY, "taskId": task_id
}).json()
if res["status"] == "ready":
token = res["solution"]["token"]
break
time.sleep(3)
print(token) # inject into cf-turnstile-response, then submit
Once you have the token, drop it into the hidden `cf-turnstile-response` input on the page (or send it directly to whatever endpoint the form posts to) and submit within the token's lifetime. If the widget is invisible and re-fires on navigation, solve again per navigation rather than reusing the old token.
For the exact field names and the Cloudflare-specific options like `action` and cData, the [Turnstile solver](https://capzy.ai/solvers/turnstile) page has the full task schema, and the broader [solver catalog](https://capzy.ai/solvers) covers the other Cloudflare-adjacent challenges you might hit on the same site.
# Getting the rest of the session right
If tokens are validating but you are still challenged repeatedly, the token is not your problem. Work the two signals Cloudflare correlates alongside it.
First, your egress. Turnstile is materially friendlier to residential-quality IPs than to datacenter ranges that carry a reputation for automation. Routing through the [Proxies API](https://capzy.ai/proxies) so your form submits from a clean IP often does more for your pass rate than any change to the token itself.
Second, your browser. If you are submitting from a raw HTTP client while the token was minted in a full browser, the mismatch is visible. Either drive the whole flow through a real [Cloud Browser](https://capzy.ai/browser), or make your client's fingerprint coherent with the [Fingerprint API](https://capzy.ai/fingerprints) so your user agent, client hints, and TLS all agree. Turnstile is looking for contradictions. Do not give it any.
A quick checklist for a healthy Turnstile flow:
* Solve against the exact `websiteURL` and `data-sitekey` from the live page.
* Submit the token immediately, once, and never reuse it.
* Submit from the same IP class and fingerprint profile the token was minted with.
* Re-solve on navigation instead of caching.
# Wrapping up
Turnstile in 2026 is a behavioral, environment-aware challenge, and `cf-turnstile-response` is only the visible output. Get the token from trusted infrastructure, submit it fresh and once, and back it with a clean IP and a coherent fingerprint so the rest of your request agrees with the token.
Start solving Turnstile today with a [free Capzy account](https://capzy.ai/auth/register), and pair it with the [Proxies API](https://capzy.ai/proxies) and [Cloud Browser](https://capzy.ai/browser) when a token alone is not enough.",
"selftext_html": "<!-- SC_OFF --><div class="md"><p>To solve Cloudflare Turnstile, you need a valid <code>cf-turnstile-response</code> token minted for the exact sitekey and origin the widget was placed on, and you need it to survive Cloudflare&#39;s server-side <code>siteverify</code> check. That is harder than it sounds in 2026, because Turnstile stopped being a checkbox a while ago and became a rolling behavioral challenge that watches your browser as much as your click.</p>
<p>This guide covers what the widget actually does, why a token that looks fine still gets rejected, and how to get a working token with a solving API. There is a code example you can copy near the end.</p>
<h1>What Turnstile is checking</h1>
<p>Turnstile is Cloudflare&#39;s replacement for the old &quot;I am not a robot&quot; checkbox. It runs in one of three modes: a managed challenge that may show a checkbox, a non-interactive mode that shows a spinner and passes silently, and an invisible mode with no visible UI at all. In every mode it does the same core work. It loads a JavaScript challenge, runs a proof-of-work and a battery of browser environment checks, times how you interact with the page, and then mints a token if it decides you are legitimate.</p>
<p>The token lands in a hidden field named <code>cf-turnstile-response</code>. Your form submits it, and Cloudflare&#39;s backend calls its verify endpoint to confirm the token is real, unexpired, and matches the sitekey and hostname. The sitekey is public and lives right in the page HTML, usually as a <code>data-sitekey</code> attribute starting with <code>0x4AAAAAAA</code>.</p>
<p>Here is the part that trips people up. The token is not the whole picture. Turnstile embeds signals about the environment it ran in. If you mint a token in a browser that leaks automation, or from an IP with a bad reputation, Cloudflare can still reject it at verify time or feed you a harder challenge next round. A token is a claim; Cloudflare decides whether to believe it.</p>
<h1>Why your Turnstile tokens fail</h1>
<p>When a token gets rejected, it is almost always one of these:</p>
<ul>
<li><strong>Wrong sitekey or origin.</strong> Turnstile binds the token to the sitekey and the hostname it was issued for. Solve it against <a href="https://app.example.com"><code>https://app.example.com</code></a> and try to use it on <a href="https://www.example.com"><code>https://www.example.com</code></a> and it fails.</li>
<li><strong>Expired token.</strong> Turnstile tokens are short-lived, on the order of a few minutes. Mint, submit, done. Do not queue them.</li>
<li><strong>Reused token.</strong> Tokens are single-use. Cloudflare&#39;s verify endpoint remembers.</li>
<li><strong>Dirty egress IP.</strong> Even a technically valid token gets a colder reception from a flagged datacenter range. Cloudflare correlates the token&#39;s context with the IP that submits the form.</li>
<li><strong>Fingerprint mismatch.</strong> If the browser that minted the token looks nothing like the client that submits it, the signals do not line up.</li>
</ul>
<p>That last pair is why &quot;I solved the captcha but I am still blocked&quot; is such a common complaint. You solved the widget. You did not solve the session.</p>
<blockquote>
<p>Turnstile grades the whole request, not just the click. A perfect token from a browser that leaks <code>navigator.webdriver</code> or from an IP that hosts a thousand other bots will still earn you a rougher ride.</p>
</blockquote>
<h1>Solving Turnstile with an API</h1>
<p>The reliable path is to let infrastructure that Cloudflare already trusts mint the token for you: a real browser engine with a coherent fingerprint, running from clean egress. A solving API wraps all of that. You submit the sitekey and page URL, it runs the challenge, and it returns <code>cf-turnstile-response</code>.</p>
<p>Here is a minimal Python example against Capzy:</p>
<pre><code>import requests, time
API = &quot;https://api.capzy.ai&quot;
KEY = &quot;YOUR_API_KEY&quot;
# create the task
task = requests.post(f&quot;{API}/createTask&quot;, json={
&quot;clientKey&quot;: KEY,
&quot;task&quot;: {
&quot;type&quot;: &quot;TurnstileTask&quot;,
&quot;websiteURL&quot;: &quot;https://example.com/login&quot;,
&quot;websiteKey&quot;: &quot;0x4AAAAAAABkMYinukE8nzY&quot;
}
}).json()
task_id = task[&quot;taskId&quot;]
# poll for the token
while True:
res = requests.post(f&quot;{API}/getTaskResult&quot;, json={
&quot;clientKey&quot;: KEY, &quot;taskId&quot;: task_id
}).json()
if res[&quot;status&quot;] == &quot;ready&quot;:
token = res[&quot;solution&quot;][&quot;token&quot;]
break
time.sleep(3)
print(token) # inject into cf-turnstile-response, then submit
</code></pre>
<p>Once you have the token, drop it into the hidden <code>cf-turnstile-response</code> input on the page (or send it directly to whatever endpoint the form posts to) and submit within the token&#39;s lifetime. If the widget is invisible and re-fires on navigation, solve again per navigation rather than reusing the old token.</p>
<p>For the exact field names and the Cloudflare-specific options like <code>action</code> and cData, the <a href="https://capzy.ai/solvers/turnstile">Turnstile solver</a> page has the full task schema, and the broader <a href="https://capzy.ai/solvers">solver catalog</a> covers the other Cloudflare-adjacent challenges you might hit on the same site.</p>
<h1>Getting the rest of the session right</h1>
<p>If tokens are validating but you are still challenged repeatedly, the token is not your problem. Work the two signals Cloudflare correlates alongside it.</p>
<p>First, your egress. Turnstile is materially friendlier to residential-quality IPs than to datacenter ranges that carry a reputation for automation. Routing through the <a href="https://capzy.ai/proxies">Proxies API</a> so your form submits from a clean IP often does more for your pass rate than any change to the token itself.</p>
<p>Second, your browser. If you are submitting from a raw HTTP client while the token was minted in a full browser, the mismatch is visible. Either drive the whole flow through a real <a href="https://capzy.ai/browser">Cloud Browser</a>, or make your client&#39;s fingerprint coherent with the <a href="https://capzy.ai/fingerprints">Fingerprint API</a> so your user agent, client hints, and TLS all agree. Turnstile is looking for contradictions. Do not give it any.</p>
<p>A quick checklist for a healthy Turnstile flow:</p>
<ul>
<li>Solve against the exact <code>websiteURL</code> and <code>data-sitekey</code> from the live page.</li>
<li>Submit the token immediately, once, and never reuse it.</li>
<li>Submit from the same IP class and fingerprint profile the token was minted with.</li>
<li>Re-solve on navigation instead of caching.</li>
</ul>
<h1>Wrapping up</h1>
<p>Turnstile in 2026 is a behavioral, environment-aware challenge, and <code>cf-turnstile-response</code> is only the visible output. Get the token from trusted infrastructure, submit it fresh and once, and back it with a clean IP and a coherent fingerprint so the rest of your request agrees with the token.</p>
<p>Start solving Turnstile today with a <a href="https://capzy.ai/auth/register">free Capzy account</a>, and pair it with the <a href="https://capzy.ai/proxies">Proxies API</a> and <a href="https://capzy.ai/browser">Cloud Browser</a> when a token alone is not enough.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/solve-cloudflare-turnstile-2026-guide",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vh8vh2/how_to_solve_cloudflare_turnstile_in_2026_a/",
"url_overridden_by_dest": "https://capzy.ai/blog/solve-cloudflare-turnstile-2026-guide",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-06T16:18:46+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/nv7JXdLRyJr_Uwgl5OCJQRkj2OHMlXh9JAa9X3jv1Aw.jpeg?width=140&height=73&auto=webp&s=a4115a4e52c4efa603fbf9d1d3a71a8ea0236408",
"preview_image": "https://external-preview.redd.it/nv7JXdLRyJr_Uwgl5OCJQRkj2OHMlXh9JAa9X3jv1Aw.jpeg?auto=webp&s=8bada03e3caa2b10fec175d1896d12d8577977c5",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vh3js3",
"fullname": "t3_1vh3js3",
"scraped_at": "2026-08-18T09:36:17.072315+00:00",
"title": "IP Reputation 101: Why the Same Script Passes on One IP and Fails on Another",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# IP Reputation 101: Why the Same Script Passes on One IP and Fails on Another
You ran the exact same scraper from two machines. On your laptop it sailed through. On the cloud box it got a 403 on the first request. The code is byte-for-byte identical, so what changed? The IP. This is IP reputation 101: every address you connect from carries a history and a classification, and anti-bot systems score that reputation before they read a single header. The same script passes on one IP and fails on another because the two IPs start with different scores.
Let me explain what "reputation" is made of and how to keep yours clean.
# What an IP's reputation is built from
An IP address is not anonymous to defenders. It comes with metadata that is cheap to look up and heavily weighted.
# ASN and IP class
Every IP belongs to an Autonomous System Number, which tells a defender who operates the range. The broad classes:
* **Residential.** Addresses assigned by consumer ISPs to homes. These carry the highest trust because that is where real users live.
* **Datacenter.** Addresses owned by hosting and cloud providers. A request from one of these is, by default, not a person sitting at a desk, so it starts with a lower score. Well-known cloud ranges are especially scrutinized because so much automated traffic originates there.
* **Mobile.** Addresses from cellular carriers, often shared behind carrier-grade NAT. High trust, but shared so heavily that a single IP maps to many users.
When you move a script from a home connection to a cloud VM, you move from the highest-trust class to one of the lowest. That single change explains most "it worked yesterday" blocks.
# History and abuse signals
Beyond class, defenders track behavior over time. Has this IP been seen hammering endpoints? Is it on public blocklists? Has it failed challenges repeatedly? Large bot managers like DataDome and Akamai maintain their own reputation databases fed by traffic across every site they protect, so an IP burned on one property arrives pre-flagged on the next.
>Reputation is shared and sticky. An IP that got burned scraping one site last week can start you at the floor on a completely different site today, because the same vendor protects both.
# Geo coherence
Your IP has a geolocation, and everything else you present should agree with it. A US East Coast IP paired with an `Asia/Tokyo` timezone and an `Accept-Language: de-DE` header is three contradictions in one request. Defenders check that your IP country, your declared timezone, and your language line up.
# Contradiction: US IP, Tokyo timezone, German language.
# Each mismatch spends score even before any challenge.
ip_country = "US"
declared_tz = "Asia/Tokyo" # wrong for a US IP
accept_language = "de-DE,de;q=0.9" # wrong for a US IP
# Coherent: pick egress, timezone, and language together.
ip_country = "US"
declared_tz = "America/New_York"
accept_language = "en-US,en;q=0.9"
# Why the score moves before your code runs
The important mental model: IP reputation is a **network-layer** signal. It is evaluated during the connection, before your headers are parsed and long before any JavaScript or captcha appears. That is why a perfect browser fingerprint cannot rescue a burned IP. You already lost points at the handshake.
This also explains a confusing symptom, where you get blocked with no challenge at all. A challenge is a chance to prove yourself. If your starting score is low enough, the defender skips the chance and blocks outright. No slider, no press-and-hold, just a 403.
# Shared IPs and the noisy-neighbor problem
Cheap proxy pools recycle a small set of addresses across thousands of customers. When someone else abuses an IP you are also using, you inherit the flag. This is the noisy-neighbor problem, and it is why a "working" proxy suddenly stops working with no change on your end. The address did not change. Its reputation did, because of traffic that was not yours.
The fix is egress you can trust: reputation-clean addresses, geo-matched to your target, rotated sensibly rather than hammered. A managed [Proxies API](https://capzy.ai/proxies) exists to solve exactly this, giving you clean addresses without you having to audit ASN history by hand.
# Rotation done wrong makes things worse
A lot of people reach for aggressive rotation as a fix and end up hurting their score. Rotating your IP on every single request is not what a human does. A real user holds one address for a whole session, clicking through pages, building up cookies and a coherent history. A scraper that swaps IP on every request looks exactly like what it is: a distributed effort to dodge counting. Some defenders specifically flag a session where the cookie set stays constant but the source IP changes on each hit, because that pairing is impossible for a real browser.
The healthier pattern is sticky sessions. Hold one clean IP for the duration of a logical session, pace your requests, and only rotate when you see signs the address is getting flagged, a sudden challenge, a 403, a slower response. Rotate on failure, not on a fixed timer. This keeps each session internally coherent while still spreading load across your pool over time.
# Reading the symptoms
Different block styles tell you where you stand on reputation:
* **Instant 403 with no challenge.** Your starting score was too low to bother challenging. Almost always an IP-class or reputation problem.
* **A challenge on the first request.** Your score was borderline. The defender is giving you a chance to prove yourself, which often means the IP is acceptable but something else (fingerprint, headers) is shaky.
* **Passing for a while, then blocks.** You burned the IP through volume or pace, or a noisy neighbor did it for you. Time to rotate that address out.
Learning to read these saves hours. A no-challenge 403 is never solved by tuning your solver, because the solver never got to run.
# How to keep your reputation clean
A short checklist that prevents most reputation-driven blocks:
* **Match IP class to the target.** Consumer-facing sites expect consumer-class traffic. Do not scrape a retail site from a well-known cloud range.
* **Match geo end to end.** Egress country, timezone, and language should all agree. If you rotate country, rotate the timezone and language with it.
* **Do not hammer a single IP.** Bursts of identical requests from one address burn it fast. Spread load and pace requests like a human session.
* **Rotate on failure, not on every request.** Sticky sessions look more human than an address that changes mid-session. Rotate when an IP shows signs of being flagged, not on a fixed per-request timer.
* **Keep the rest coherent.** Reputation gets you in the door, but a clean IP paired with an obvious Python TLS fingerprint still fails. Pair clean egress with a coherent build from the [Fingerprint API](https://capzy.ai/fingerprints).
# Why a token alone will not save you
If you buy a captcha token but mint it from a burned IP, the token is often rejected on validation because the defender scores the session it was solved in. DataDome, Akamai, PerimeterX, and Kasada all fold IP reputation into the verdict. The token has to come from a session whose network reputation already looks clean. That is why our solvers run on reputation-clean egress by default. Explore the [solver catalog](https://capzy.ai/solvers) to see how the pieces fit.
# Where Capzy fits
Capzy pairs clean, geo-coherent egress with coherent fingerprints and browser-minted tokens, so your session starts with a good score instead of fighting uphill from the floor. The [Proxies API](https://capzy.ai/proxies) handles the network layer so you stop debugging blocks that were never about your code.
Same script, different result? It is almost always the IP. [Sign up](https://capzy.ai/auth/register) and route through clean egress on your next request.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>IP Reputation 101: Why the Same Script Passes on One IP and Fails on Another</h1>
<p>You ran the exact same scraper from two machines. On your laptop it sailed through. On the cloud box it got a 403 on the first request. The code is byte-for-byte identical, so what changed? The IP. This is IP reputation 101: every address you connect from carries a history and a classification, and anti-bot systems score that reputation before they read a single header. The same script passes on one IP and fails on another because the two IPs start with different scores.</p>
<p>Let me explain what &quot;reputation&quot; is made of and how to keep yours clean.</p>
<h1>What an IP&#39;s reputation is built from</h1>
<p>An IP address is not anonymous to defenders. It comes with metadata that is cheap to look up and heavily weighted.</p>
<h1>ASN and IP class</h1>
<p>Every IP belongs to an Autonomous System Number, which tells a defender who operates the range. The broad classes:</p>
<ul>
<li><strong>Residential.</strong> Addresses assigned by consumer ISPs to homes. These carry the highest trust because that is where real users live.</li>
<li><strong>Datacenter.</strong> Addresses owned by hosting and cloud providers. A request from one of these is, by default, not a person sitting at a desk, so it starts with a lower score. Well-known cloud ranges are especially scrutinized because so much automated traffic originates there.</li>
<li><strong>Mobile.</strong> Addresses from cellular carriers, often shared behind carrier-grade NAT. High trust, but shared so heavily that a single IP maps to many users.</li>
</ul>
<p>When you move a script from a home connection to a cloud VM, you move from the highest-trust class to one of the lowest. That single change explains most &quot;it worked yesterday&quot; blocks.</p>
<h1>History and abuse signals</h1>
<p>Beyond class, defenders track behavior over time. Has this IP been seen hammering endpoints? Is it on public blocklists? Has it failed challenges repeatedly? Large bot managers like DataDome and Akamai maintain their own reputation databases fed by traffic across every site they protect, so an IP burned on one property arrives pre-flagged on the next.</p>
<blockquote>
<p>Reputation is shared and sticky. An IP that got burned scraping one site last week can start you at the floor on a completely different site today, because the same vendor protects both.</p>
</blockquote>
<h1>Geo coherence</h1>
<p>Your IP has a geolocation, and everything else you present should agree with it. A US East Coast IP paired with an <code>Asia/Tokyo</code> timezone and an <code>Accept-Language: de-DE</code> header is three contradictions in one request. Defenders check that your IP country, your declared timezone, and your language line up.</p>
<pre><code># Contradiction: US IP, Tokyo timezone, German language.
# Each mismatch spends score even before any challenge.
ip_country = &quot;US&quot;
declared_tz = &quot;Asia/Tokyo&quot; # wrong for a US IP
accept_language = &quot;de-DE,de;q=0.9&quot; # wrong for a US IP
# Coherent: pick egress, timezone, and language together.
ip_country = &quot;US&quot;
declared_tz = &quot;America/New_York&quot;
accept_language = &quot;en-US,en;q=0.9&quot;
</code></pre>
<h1>Why the score moves before your code runs</h1>
<p>The important mental model: IP reputation is a <strong>network-layer</strong> signal. It is evaluated during the connection, before your headers are parsed and long before any JavaScript or captcha appears. That is why a perfect browser fingerprint cannot rescue a burned IP. You already lost points at the handshake.</p>
<p>This also explains a confusing symptom, where you get blocked with no challenge at all. A challenge is a chance to prove yourself. If your starting score is low enough, the defender skips the chance and blocks outright. No slider, no press-and-hold, just a 403.</p>
<h1>Shared IPs and the noisy-neighbor problem</h1>
<p>Cheap proxy pools recycle a small set of addresses across thousands of customers. When someone else abuses an IP you are also using, you inherit the flag. This is the noisy-neighbor problem, and it is why a &quot;working&quot; proxy suddenly stops working with no change on your end. The address did not change. Its reputation did, because of traffic that was not yours.</p>
<p>The fix is egress you can trust: reputation-clean addresses, geo-matched to your target, rotated sensibly rather than hammered. A managed <a href="https://capzy.ai/proxies">Proxies API</a> exists to solve exactly this, giving you clean addresses without you having to audit ASN history by hand.</p>
<h1>Rotation done wrong makes things worse</h1>
<p>A lot of people reach for aggressive rotation as a fix and end up hurting their score. Rotating your IP on every single request is not what a human does. A real user holds one address for a whole session, clicking through pages, building up cookies and a coherent history. A scraper that swaps IP on every request looks exactly like what it is: a distributed effort to dodge counting. Some defenders specifically flag a session where the cookie set stays constant but the source IP changes on each hit, because that pairing is impossible for a real browser.</p>
<p>The healthier pattern is sticky sessions. Hold one clean IP for the duration of a logical session, pace your requests, and only rotate when you see signs the address is getting flagged, a sudden challenge, a 403, a slower response. Rotate on failure, not on a fixed timer. This keeps each session internally coherent while still spreading load across your pool over time.</p>
<h1>Reading the symptoms</h1>
<p>Different block styles tell you where you stand on reputation:</p>
<ul>
<li><strong>Instant 403 with no challenge.</strong> Your starting score was too low to bother challenging. Almost always an IP-class or reputation problem.</li>
<li><strong>A challenge on the first request.</strong> Your score was borderline. The defender is giving you a chance to prove yourself, which often means the IP is acceptable but something else (fingerprint, headers) is shaky.</li>
<li><strong>Passing for a while, then blocks.</strong> You burned the IP through volume or pace, or a noisy neighbor did it for you. Time to rotate that address out.</li>
</ul>
<p>Learning to read these saves hours. A no-challenge 403 is never solved by tuning your solver, because the solver never got to run.</p>
<h1>How to keep your reputation clean</h1>
<p>A short checklist that prevents most reputation-driven blocks:</p>
<ul>
<li><strong>Match IP class to the target.</strong> Consumer-facing sites expect consumer-class traffic. Do not scrape a retail site from a well-known cloud range.</li>
<li><strong>Match geo end to end.</strong> Egress country, timezone, and language should all agree. If you rotate country, rotate the timezone and language with it.</li>
<li><strong>Do not hammer a single IP.</strong> Bursts of identical requests from one address burn it fast. Spread load and pace requests like a human session.</li>
<li><strong>Rotate on failure, not on every request.</strong> Sticky sessions look more human than an address that changes mid-session. Rotate when an IP shows signs of being flagged, not on a fixed per-request timer.</li>
<li><strong>Keep the rest coherent.</strong> Reputation gets you in the door, but a clean IP paired with an obvious Python TLS fingerprint still fails. Pair clean egress with a coherent build from the <a href="https://capzy.ai/fingerprints">Fingerprint API</a>.</li>
</ul>
<h1>Why a token alone will not save you</h1>
<p>If you buy a captcha token but mint it from a burned IP, the token is often rejected on validation because the defender scores the session it was solved in. DataDome, Akamai, PerimeterX, and Kasada all fold IP reputation into the verdict. The token has to come from a session whose network reputation already looks clean. That is why our solvers run on reputation-clean egress by default. Explore the <a href="https://capzy.ai/solvers">solver catalog</a> to see how the pieces fit.</p>
<h1>Where Capzy fits</h1>
<p>Capzy pairs clean, geo-coherent egress with coherent fingerprints and browser-minted tokens, so your session starts with a good score instead of fighting uphill from the floor. The <a href="https://capzy.ai/proxies">Proxies API</a> handles the network layer so you stop debugging blocks that were never about your code.</p>
<p>Same script, different result? It is almost always the IP. <a href="https://capzy.ai/auth/register">Sign up</a> and route through clean egress on your next request.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/ip-reputation-101",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vh3js3/ip_reputation_101_why_the_same_script_passes_on/",
"url_overridden_by_dest": "https://capzy.ai/blog/ip-reputation-101",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-06T12:58:45+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/TlZnwbbO23GVjUWN4jnBE9mLaHmlhbccsIngmw6LzTg.jpeg?width=140&height=73&auto=webp&s=eb16434d41193b9b6d22197ff9b41abaee229b99",
"preview_image": "https://external-preview.redd.it/TlZnwbbO23GVjUWN4jnBE9mLaHmlhbccsIngmw6LzTg.jpeg?auto=webp&s=49e351faa1b1c76b92ebb2ba87ec7d81959fa4f0",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vh3i0j",
"fullname": "t3_1vh3i0j",
"scraped_at": "2026-08-18T09:36:17.072325+00:00",
"title": "TLS and HTTP/2 Fingerprinting (JA3/JA4): The Layer Most Scrapers Miss",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# TLS and HTTP/2 Fingerprinting (JA3/JA4): The Layer Most Scrapers Miss
Your headers are perfect. Your `User-Agent` says Chrome 120, your client hints match, your `Accept-Language` is set. And you still get blocked on the very first request, before any JavaScript runs. The reason is a layer almost every scraper ignores: TLS and HTTP/2 fingerprinting. Before your headers are ever parsed, the way your client negotiates the encrypted connection produces a JA3 or JA4 hash, and if that hash says "Python client" while your header says "Chrome," the contradiction blocks you at the handshake.
This is the layer most tutorials skip. Here is how it works and why it matters more than the headers you have been tuning.
# What gets fingerprinted in a TLS handshake
When any client opens an HTTPS connection, it sends a **ClientHello** that advertises what it supports. That message is far more revealing than people realize. It contains:
* **The cipher suite list, in order.** Which encryption algorithms the client offers, and crucially the exact order. Chrome offers a specific ordered list. OpenSSL defaults, which most Python libraries inherit, offer a different one.
* **TLS extensions, in order.** ALPN, supported groups, signature algorithms, key share, and more. Both which extensions are present and their ordering vary by client.
* **Supported elliptic curves and point formats.** The named groups a client will use for key exchange.
* **The TLS version and supported versions.**
A JA3 hash is an MD5 of a comma-joined string built from these fields (TLS version, cipher list, extension list, curves, point formats). JA4 is the newer, more structured successor that is harder to spoof and includes ALPN and a sorted-versus-unsorted view of extensions. Either way, the result is a compact fingerprint of your TLS stack.
>Your JA3/JA4 is decided by your TLS library, not by the headers you set. You can spoof a User-Agent in one line. You cannot spoof a JA3 hash without changing how the handshake itself is built.
# Why default HTTP clients betray you
Here is the trap. A default Python `requests` or `httpx` call negotiates TLS through the system OpenSSL. Its cipher order, extension set, and curve list produce a JA3 that no browser emits. So a defender sees a request that claims to be Chrome in the `User-Agent` but has the TLS signature of a scripting language. That mismatch is one of the cheapest, most reliable bot signals in existence, and it fires before your code sends anything else.
Your header says: User-Agent: ...Chrome/120...
Your JA3 hash says: (a Python/OpenSSL signature no Chrome produces)
Defender's conclusion: lying client -> block
This is why people report that "adding more headers did nothing." The block was never about headers. It was the handshake.
# HTTP/2 fingerprinting
Even after TLS, the story continues. Modern browsers speak HTTP/2, and the way a client sets up an HTTP/2 connection is itself a fingerprint. Defenders look at:
* **SETTINGS frame values.** The initial header table size, max concurrent streams, initial window size, and the order these settings appear in. Browsers use characteristic values.
* **WINDOW\_UPDATE behavior.** How the client manages flow control.
* **Pseudo-header order.** The order of `:method`, `:authority`, `:scheme`, `:path` in the HEADERS frame. Browsers use a stable order. Many libraries do not, or fall back to HTTP/1.1 entirely, which is itself a tell against a "modern Chrome" claim.
* **Priority and frame ordering.** The overall choreography of frames on the connection.
Combined, the TLS ClientHello and the HTTP/2 setup give a defender a network-layer identity that is very hard to fake from a generic client, and that has to agree with the browser identity you claim on top of it.
# ALPN and the small details that leak
One easy-to-miss field is **ALPN** (Application-Layer Protocol Negotiation), the extension where the client says whether it prefers `h2` (HTTP/2) or `http/1.1`. A client that advertises `http/1.1` only, while claiming to be a current Chrome that always offers `h2`, is another contradiction. GREASE values, the deliberately random extension values Chrome injects to keep the ecosystem flexible, are another. Their presence and pattern are part of the expected Chrome signature, and a client that omits them looks unlike real Chrome.
# How to actually fix it
You have two honest options:
* **Use a client that mimics a real browser's TLS and HTTP/2 stack.** Some libraries are built specifically to reproduce a target browser's JA3/JA4, cipher order, ALPN, and HTTP/2 SETTINGS. The `curl_cffi` family, for example, can present a real Chrome TLS fingerprint rather than the OpenSSL default. When you use one, your JA3 and your `User-Agent` finally tell the same story.
* **Run an actual browser.** A real Chrome produces a real Chrome handshake by definition. This is heavier, but for the strictest targets it is the reliable path because every layer stays coherent for free.
Whichever you pick, the principle is coherence: the TLS layer, the HTTP/2 layer, the headers, and the client-side `navigator` must all agree on one identity. Our writeup on the [Fingerprint API](https://capzy.ai/fingerprints) covers keeping that agreement from the handshake all the way up to canvas and WebGL.
# How defenders collect the hash cheaply
Worth understanding why this layer is so popular with defenders: it is nearly free to collect and very hard to forge. The ClientHello arrives before a single byte of your application request. A reverse proxy or CDN edge computes the JA3/JA4 hash during the handshake it was already performing, attaches it as a header for the origin, and moves on. There is no extra round trip, no JavaScript to inject, no challenge to serve. The defender gets a high-signal fingerprint at zero marginal cost, then simply compares it against a table of known-browser hashes. A hash that matches no real browser, or that contradicts the `User-Agent`, is flagged before your request is ever routed.
That asymmetry is the whole reason this layer wins so often. Spoofing a header is one line of code on your side. Reproducing a real Chrome ClientHello means matching the cipher order, the extension order, the GREASE values, the supported groups, and the ALPN list, all of which are baked into your TLS library rather than exposed as a setting. Most scrapers never touch it because their HTTP client does not expose it, so they lose at a layer they did not know they were playing on.
# A quick self-check
Before blaming your headers or your solver, verify what your client actually sends at the handshake. A public JA3/JA4 echo service will show you the hash your client produces and whether it matches a known browser.
1. Send a request from your scraper to a TLS-fingerprint echo endpoint.
2. Read back the JA3/JA4 hash it observed.
3. Compare it to a real Chrome's known hash for the same version.
4. If they differ, your handshake is the block, not your headers.
If step 4 shows a mismatch, no amount of header tuning will fix it. You need a browser-accurate client or a real browser.
# The IP reputation multiplier
TLS coherence and IP reputation compound. A perfect Chrome JA3 from a burned datacenter IP still fails, and a clean residential IP with a Python JA3 still fails. You need both: a browser-accurate handshake and reputation-clean egress from a [Proxies API](https://capzy.ai/proxies). Get either wrong and you lose points at the network layer before a challenge ever appears.
# Where Capzy fits
Capzy handles the layer most scrapers miss. Our solvers mint tokens from sessions with browser-accurate TLS and HTTP/2 fingerprints, coherent headers, and reputation-clean IPs, so your JA3 and your User-Agent never contradict each other. Browse the [solver catalog](https://capzy.ai/solvers) to see coverage across DataDome, Akamai, PerimeterX, Kasada, and more.
If your headers look perfect but you still get blocked, look at the handshake. [Create an account](https://capzy.ai/auth/register) and send your next request through a session that is coherent from the TLS layer up.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>TLS and HTTP/2 Fingerprinting (JA3/JA4): The Layer Most Scrapers Miss</h1>
<p>Your headers are perfect. Your <code>User-Agent</code> says Chrome 120, your client hints match, your <code>Accept-Language</code> is set. And you still get blocked on the very first request, before any JavaScript runs. The reason is a layer almost every scraper ignores: TLS and HTTP/2 fingerprinting. Before your headers are ever parsed, the way your client negotiates the encrypted connection produces a JA3 or JA4 hash, and if that hash says &quot;Python client&quot; while your header says &quot;Chrome,&quot; the contradiction blocks you at the handshake.</p>
<p>This is the layer most tutorials skip. Here is how it works and why it matters more than the headers you have been tuning.</p>
<h1>What gets fingerprinted in a TLS handshake</h1>
<p>When any client opens an HTTPS connection, it sends a <strong>ClientHello</strong> that advertises what it supports. That message is far more revealing than people realize. It contains:</p>
<ul>
<li><strong>The cipher suite list, in order.</strong> Which encryption algorithms the client offers, and crucially the exact order. Chrome offers a specific ordered list. OpenSSL defaults, which most Python libraries inherit, offer a different one.</li>
<li><strong>TLS extensions, in order.</strong> ALPN, supported groups, signature algorithms, key share, and more. Both which extensions are present and their ordering vary by client.</li>
<li><strong>Supported elliptic curves and point formats.</strong> The named groups a client will use for key exchange.</li>
<li><strong>The TLS version and supported versions.</strong></li>
</ul>
<p>A JA3 hash is an MD5 of a comma-joined string built from these fields (TLS version, cipher list, extension list, curves, point formats). JA4 is the newer, more structured successor that is harder to spoof and includes ALPN and a sorted-versus-unsorted view of extensions. Either way, the result is a compact fingerprint of your TLS stack.</p>
<blockquote>
<p>Your JA3/JA4 is decided by your TLS library, not by the headers you set. You can spoof a User-Agent in one line. You cannot spoof a JA3 hash without changing how the handshake itself is built.</p>
</blockquote>
<h1>Why default HTTP clients betray you</h1>
<p>Here is the trap. A default Python <code>requests</code> or <code>httpx</code> call negotiates TLS through the system OpenSSL. Its cipher order, extension set, and curve list produce a JA3 that no browser emits. So a defender sees a request that claims to be Chrome in the <code>User-Agent</code> but has the TLS signature of a scripting language. That mismatch is one of the cheapest, most reliable bot signals in existence, and it fires before your code sends anything else.</p>
<pre><code>Your header says: User-Agent: ...Chrome/120...
Your JA3 hash says: (a Python/OpenSSL signature no Chrome produces)
Defender&#39;s conclusion: lying client -&gt; block
</code></pre>
<p>This is why people report that &quot;adding more headers did nothing.&quot; The block was never about headers. It was the handshake.</p>
<h1>HTTP/2 fingerprinting</h1>
<p>Even after TLS, the story continues. Modern browsers speak HTTP/2, and the way a client sets up an HTTP/2 connection is itself a fingerprint. Defenders look at:</p>
<ul>
<li><strong>SETTINGS frame values.</strong> The initial header table size, max concurrent streams, initial window size, and the order these settings appear in. Browsers use characteristic values.</li>
<li><strong>WINDOW_UPDATE behavior.</strong> How the client manages flow control.</li>
<li><strong>Pseudo-header order.</strong> The order of <code>:method</code>, <code>:authority</code>, <code>:scheme</code>, <code>:path</code> in the HEADERS frame. Browsers use a stable order. Many libraries do not, or fall back to HTTP/1.1 entirely, which is itself a tell against a &quot;modern Chrome&quot; claim.</li>
<li><strong>Priority and frame ordering.</strong> The overall choreography of frames on the connection.</li>
</ul>
<p>Combined, the TLS ClientHello and the HTTP/2 setup give a defender a network-layer identity that is very hard to fake from a generic client, and that has to agree with the browser identity you claim on top of it.</p>
<h1>ALPN and the small details that leak</h1>
<p>One easy-to-miss field is <strong>ALPN</strong> (Application-Layer Protocol Negotiation), the extension where the client says whether it prefers <code>h2</code> (HTTP/2) or <code>http/1.1</code>. A client that advertises <code>http/1.1</code> only, while claiming to be a current Chrome that always offers <code>h2</code>, is another contradiction. GREASE values, the deliberately random extension values Chrome injects to keep the ecosystem flexible, are another. Their presence and pattern are part of the expected Chrome signature, and a client that omits them looks unlike real Chrome.</p>
<h1>How to actually fix it</h1>
<p>You have two honest options:</p>
<ul>
<li><strong>Use a client that mimics a real browser&#39;s TLS and HTTP/2 stack.</strong> Some libraries are built specifically to reproduce a target browser&#39;s JA3/JA4, cipher order, ALPN, and HTTP/2 SETTINGS. The <code>curl_cffi</code> family, for example, can present a real Chrome TLS fingerprint rather than the OpenSSL default. When you use one, your JA3 and your <code>User-Agent</code> finally tell the same story.</li>
<li><strong>Run an actual browser.</strong> A real Chrome produces a real Chrome handshake by definition. This is heavier, but for the strictest targets it is the reliable path because every layer stays coherent for free.</li>
</ul>
<p>Whichever you pick, the principle is coherence: the TLS layer, the HTTP/2 layer, the headers, and the client-side <code>navigator</code> must all agree on one identity. Our writeup on the <a href="https://capzy.ai/fingerprints">Fingerprint API</a> covers keeping that agreement from the handshake all the way up to canvas and WebGL.</p>
<h1>How defenders collect the hash cheaply</h1>
<p>Worth understanding why this layer is so popular with defenders: it is nearly free to collect and very hard to forge. The ClientHello arrives before a single byte of your application request. A reverse proxy or CDN edge computes the JA3/JA4 hash during the handshake it was already performing, attaches it as a header for the origin, and moves on. There is no extra round trip, no JavaScript to inject, no challenge to serve. The defender gets a high-signal fingerprint at zero marginal cost, then simply compares it against a table of known-browser hashes. A hash that matches no real browser, or that contradicts the <code>User-Agent</code>, is flagged before your request is ever routed.</p>
<p>That asymmetry is the whole reason this layer wins so often. Spoofing a header is one line of code on your side. Reproducing a real Chrome ClientHello means matching the cipher order, the extension order, the GREASE values, the supported groups, and the ALPN list, all of which are baked into your TLS library rather than exposed as a setting. Most scrapers never touch it because their HTTP client does not expose it, so they lose at a layer they did not know they were playing on.</p>
<h1>A quick self-check</h1>
<p>Before blaming your headers or your solver, verify what your client actually sends at the handshake. A public JA3/JA4 echo service will show you the hash your client produces and whether it matches a known browser.</p>
<pre><code>1. Send a request from your scraper to a TLS-fingerprint echo endpoint.
2. Read back the JA3/JA4 hash it observed.
3. Compare it to a real Chrome&#39;s known hash for the same version.
4. If they differ, your handshake is the block, not your headers.
</code></pre>
<p>If step 4 shows a mismatch, no amount of header tuning will fix it. You need a browser-accurate client or a real browser.</p>
<h1>The IP reputation multiplier</h1>
<p>TLS coherence and IP reputation compound. A perfect Chrome JA3 from a burned datacenter IP still fails, and a clean residential IP with a Python JA3 still fails. You need both: a browser-accurate handshake and reputation-clean egress from a <a href="https://capzy.ai/proxies">Proxies API</a>. Get either wrong and you lose points at the network layer before a challenge ever appears.</p>
<h1>Where Capzy fits</h1>
<p>Capzy handles the layer most scrapers miss. Our solvers mint tokens from sessions with browser-accurate TLS and HTTP/2 fingerprints, coherent headers, and reputation-clean IPs, so your JA3 and your User-Agent never contradict each other. Browse the <a href="https://capzy.ai/solvers">solver catalog</a> to see coverage across DataDome, Akamai, PerimeterX, Kasada, and more.</p>
<p>If your headers look perfect but you still get blocked, look at the handshake. <a href="https://capzy.ai/auth/register">Create an account</a> and send your next request through a session that is coherent from the TLS layer up.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/tls-http2-fingerprinting-ja3-ja4",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vh3i0j/tls_and_http2_fingerprinting_ja3ja4_the_layer/",
"url_overridden_by_dest": "https://capzy.ai/blog/tls-http2-fingerprinting-ja3-ja4",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-06T12:56:34+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/g07m7pElin5gWxnSzA1NUqLS1jAkVQSNTN4t262-mHM.jpeg?width=140&height=73&auto=webp&s=9455d3e6c5621e017c23ac0a245c5b9e070d9d3c",
"preview_image": "https://external-preview.redd.it/g07m7pElin5gWxnSzA1NUqLS1jAkVQSNTN4t262-mHM.jpeg?auto=webp&s=825da00db934e16bb7e02754b3a781526e386d32",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vh3he2",
"fullname": "t3_1vh3he2",
"scraped_at": "2026-08-18T09:36:17.072336+00:00",
"title": "How to Bypass CAPTCHAs During Web Scraping Without Getting Blocked",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# How to Bypass CAPTCHAs During Web Scraping Without Getting Blocked
If you want to bypass CAPTCHAs during web scraping without getting blocked, the first thing to accept is that the challenge is rarely the problem on its own. A CAPTCHA is the symptom. The site already flagged your request before it ever painted a puzzle. By the time you see a reCAPTCHA checkbox or a Turnstile widget, the server has looked at your IP reputation, your TLS handshake, your header order, and how fast you clicked through the last three pages. Solving the puzzle is step three of a five-step problem.
So the goal is not "beat the CAPTCHA." The goal is to look like a normal visitor long enough that the site either never challenges you, or challenges you rarely and you clear it cleanly. Everything below is in service of that.
# Why you got blocked in the first place
Most scrapers get flagged for boring reasons. A datacenter IP that a thousand other bots already burned. A `User-Agent` that says Chrome 120 while the TLS fingerprint says Python. Requests firing every 40 milliseconds with no jitter. Cookies that never persist, so every hit looks like a brand-new visitor with no history.
Here is a quick mental checklist before you touch any solver:
* Is your IP residential or mobile, and is it sticky for the session?
* Does your TLS and HTTP/2 fingerprint match the browser you claim to be?
* Are you reusing cookies across a session, or throwing them away every request?
* Is your request timing varied, or is it a metronome?
Fix those and a large share of challenges disappear on their own. That is the cheapest CAPTCHA "bypass" there is: never trigger one.
# The three layers that actually keep data flowing
A working scraper stack has three cooperating pieces, and you want them talking to each other rather than bolted on as afterthoughts.
**Layer one is the proxy.** You need clean egress IPs with good reputation, and for anything session-based you need the IP to stay sticky so your cookies and your source address agree. Capzy's [Proxies API](https://capzy.ai/proxies) gives you rotating and sticky options so a session that starts on one address finishes on it too. Rotating a fresh IP mid-session is a classic self-inflicted block: the site sees a logged-in cookie suddenly arrive from a new country and quietly flags it.
**Layer two is the solver.** When a challenge does appear, you hand it off to something that returns a token. Capzy's [Solver API](https://capzy.ai/solvers) takes the site key and page URL and gives you back a response token you inject into the form. This covers the widget families you will actually hit in the wild, including reCAPTCHA, hCaptcha, and Turnstile.
**Layer three, when you need it, is a real browser.** Some sites gate their content behind JavaScript that only runs correctly in an actual rendering engine, and some anti-bot systems score how the page behaves after load. For those, a headless script is not enough and you want Capzy's [Cloud Browser](https://capzy.ai/browser), a real Chrome you drive over CDP.
>The trap most teams fall into: they buy a solver, wire it in, and still get blocked because their proxies are dirty and their sessions leak. The solver was never the weak link. Fix the identity first, then the solver has an easy job.
# A concrete flow
Say you are scraping a product catalog that throws a Turnstile challenge on the search endpoint. The pattern looks like this. You make your request through a sticky residential IP. If the response comes back clean, great, you parse and move on. If it comes back with a challenge, you extract the site key from the page, send it to the solver, get a token, replay the request with the token attached, and continue on the same IP and cookie jar.
Here is the solver hand-off in plain Python:
import requests
API_KEY = "your_capzy_key"
# 1. Create the solve task with the site key you scraped from the page
task = requests.post(
"https://api.capzy.ai/createTask",
json={
"clientKey": API_KEY,
"task": {
"type": "TurnstileTaskProxyless",
"websiteURL": "https://shop.example.com/search",
"websiteKey": "0x4AAAAAAABkMYinukE8nzYS",
},
},
timeout=30,
).json()
task_id = task["taskId"]
# 2. Poll for the token
import time
while True:
result = requests.post(
"https://api.capzy.ai/getTaskResult",
json={"clientKey": API_KEY, "taskId": task_id},
timeout=30,
).json()
if result.get("status") == "ready":
token = result["solution"]["token"]
break
time.sleep(2)
# 3. Replay the original request WITH the token, on the SAME session
session = requests.Session()
session.proxies = {"https": "http://user:pass@sticky.capzy.ai:8000"}
resp = session.post(
"https://shop.example.com/search",
data={"q": "wireless earbuds", "cf-turnstile-response": token},
)
print(resp.status_code)
The important detail is step three. The token is bound to the site and often to the session context. If you solve on one IP and replay on another, the site may reject a perfectly valid token because the context no longer matches. Keep the solve, the proxy, and the replay on one coherent identity.
One more thing that trips people up: cache tokens carefully or not at all. A response token has a short lifetime, often around two minutes, and many are single-use. Reusing a stale token feels like a free optimization until the site starts silently rejecting them and your solve rate craters for no visible reason. If you are solving in a loop, solve fresh per request unless you have confirmed the token type is safe to reuse for that specific widget and site.
# Rate, timing, and the human-shaped curve
Even with clean IPs and a working solver, you can burn a good setup with bad pacing. A real person does not request 200 pages in 90 seconds. Add jitter to your delays, respect the natural rhythm of the site, and back off when you see the first sign of friction. A single `429` is a warning; three in a row is you being throttled toward a block.
When you do get challenged repeatedly on the same target, that is usually IP reputation talking, not the solver failing. Rotate to a fresh clean address, slow down, and let the previous IP cool off. Solve rate is a product of the whole pipeline, not one component.
# When to let a managed API carry it
Wiring proxies, solving, retries, and session state together is real work, and it breaks in annoying ways at 2 a.m. If you would rather send a URL and get parsed data back, Capzy's [Web Scraper API](https://capzy.ai/web-scraper) bundles the proxy rotation, the challenge handling, and optional JavaScript rendering into one `/scrape` call. You point it at a page, it returns the content, and the CAPTCHA layer is handled for you.
The honest summary: you rarely "bypass" a CAPTCHA by outsmarting the puzzle. You bypass it by not looking like a bot, and by having a clean token ready for the times the site challenges you anyway. Get the proxy identity right, keep sessions coherent, pace like a human, and hand the puzzle to a solver only when one shows up.
Ready to stop fighting blocks? [Create a free Capzy account](https://capzy.ai/auth/register) and route your next scrape through clean proxies with solving built in.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>How to Bypass CAPTCHAs During Web Scraping Without Getting Blocked</h1>
<p>If you want to bypass CAPTCHAs during web scraping without getting blocked, the first thing to accept is that the challenge is rarely the problem on its own. A CAPTCHA is the symptom. The site already flagged your request before it ever painted a puzzle. By the time you see a reCAPTCHA checkbox or a Turnstile widget, the server has looked at your IP reputation, your TLS handshake, your header order, and how fast you clicked through the last three pages. Solving the puzzle is step three of a five-step problem.</p>
<p>So the goal is not &quot;beat the CAPTCHA.&quot; The goal is to look like a normal visitor long enough that the site either never challenges you, or challenges you rarely and you clear it cleanly. Everything below is in service of that.</p>
<h1>Why you got blocked in the first place</h1>
<p>Most scrapers get flagged for boring reasons. A datacenter IP that a thousand other bots already burned. A <code>User-Agent</code> that says Chrome 120 while the TLS fingerprint says Python. Requests firing every 40 milliseconds with no jitter. Cookies that never persist, so every hit looks like a brand-new visitor with no history.</p>
<p>Here is a quick mental checklist before you touch any solver:</p>
<ul>
<li>Is your IP residential or mobile, and is it sticky for the session?</li>
<li>Does your TLS and HTTP/2 fingerprint match the browser you claim to be?</li>
<li>Are you reusing cookies across a session, or throwing them away every request?</li>
<li>Is your request timing varied, or is it a metronome?</li>
</ul>
<p>Fix those and a large share of challenges disappear on their own. That is the cheapest CAPTCHA &quot;bypass&quot; there is: never trigger one.</p>
<h1>The three layers that actually keep data flowing</h1>
<p>A working scraper stack has three cooperating pieces, and you want them talking to each other rather than bolted on as afterthoughts.</p>
<p><strong>Layer one is the proxy.</strong> You need clean egress IPs with good reputation, and for anything session-based you need the IP to stay sticky so your cookies and your source address agree. Capzy&#39;s <a href="https://capzy.ai/proxies">Proxies API</a> gives you rotating and sticky options so a session that starts on one address finishes on it too. Rotating a fresh IP mid-session is a classic self-inflicted block: the site sees a logged-in cookie suddenly arrive from a new country and quietly flags it.</p>
<p><strong>Layer two is the solver.</strong> When a challenge does appear, you hand it off to something that returns a token. Capzy&#39;s <a href="https://capzy.ai/solvers">Solver API</a> takes the site key and page URL and gives you back a response token you inject into the form. This covers the widget families you will actually hit in the wild, including reCAPTCHA, hCaptcha, and Turnstile.</p>
<p><strong>Layer three, when you need it, is a real browser.</strong> Some sites gate their content behind JavaScript that only runs correctly in an actual rendering engine, and some anti-bot systems score how the page behaves after load. For those, a headless script is not enough and you want Capzy&#39;s <a href="https://capzy.ai/browser">Cloud Browser</a>, a real Chrome you drive over CDP.</p>
<blockquote>
<p>The trap most teams fall into: they buy a solver, wire it in, and still get blocked because their proxies are dirty and their sessions leak. The solver was never the weak link. Fix the identity first, then the solver has an easy job.</p>
</blockquote>
<h1>A concrete flow</h1>
<p>Say you are scraping a product catalog that throws a Turnstile challenge on the search endpoint. The pattern looks like this. You make your request through a sticky residential IP. If the response comes back clean, great, you parse and move on. If it comes back with a challenge, you extract the site key from the page, send it to the solver, get a token, replay the request with the token attached, and continue on the same IP and cookie jar.</p>
<p>Here is the solver hand-off in plain Python:</p>
<pre><code>import requests
API_KEY = &quot;your_capzy_key&quot;
# 1. Create the solve task with the site key you scraped from the page
task = requests.post(
&quot;https://api.capzy.ai/createTask&quot;,
json={
&quot;clientKey&quot;: API_KEY,
&quot;task&quot;: {
&quot;type&quot;: &quot;TurnstileTaskProxyless&quot;,
&quot;websiteURL&quot;: &quot;https://shop.example.com/search&quot;,
&quot;websiteKey&quot;: &quot;0x4AAAAAAABkMYinukE8nzYS&quot;,
},
},
timeout=30,
).json()
task_id = task[&quot;taskId&quot;]
# 2. Poll for the token
import time
while True:
result = requests.post(
&quot;https://api.capzy.ai/getTaskResult&quot;,
json={&quot;clientKey&quot;: API_KEY, &quot;taskId&quot;: task_id},
timeout=30,
).json()
if result.get(&quot;status&quot;) == &quot;ready&quot;:
token = result[&quot;solution&quot;][&quot;token&quot;]
break
time.sleep(2)
# 3. Replay the original request WITH the token, on the SAME session
session = requests.Session()
session.proxies = {&quot;https&quot;: &quot;http://user:pass@sticky.capzy.ai:8000&quot;}
resp = session.post(
&quot;https://shop.example.com/search&quot;,
data={&quot;q&quot;: &quot;wireless earbuds&quot;, &quot;cf-turnstile-response&quot;: token},
)
print(resp.status_code)
</code></pre>
<p>The important detail is step three. The token is bound to the site and often to the session context. If you solve on one IP and replay on another, the site may reject a perfectly valid token because the context no longer matches. Keep the solve, the proxy, and the replay on one coherent identity.</p>
<p>One more thing that trips people up: cache tokens carefully or not at all. A response token has a short lifetime, often around two minutes, and many are single-use. Reusing a stale token feels like a free optimization until the site starts silently rejecting them and your solve rate craters for no visible reason. If you are solving in a loop, solve fresh per request unless you have confirmed the token type is safe to reuse for that specific widget and site.</p>
<h1>Rate, timing, and the human-shaped curve</h1>
<p>Even with clean IPs and a working solver, you can burn a good setup with bad pacing. A real person does not request 200 pages in 90 seconds. Add jitter to your delays, respect the natural rhythm of the site, and back off when you see the first sign of friction. A single <code>429</code> is a warning; three in a row is you being throttled toward a block.</p>
<p>When you do get challenged repeatedly on the same target, that is usually IP reputation talking, not the solver failing. Rotate to a fresh clean address, slow down, and let the previous IP cool off. Solve rate is a product of the whole pipeline, not one component.</p>
<h1>When to let a managed API carry it</h1>
<p>Wiring proxies, solving, retries, and session state together is real work, and it breaks in annoying ways at 2 a.m. If you would rather send a URL and get parsed data back, Capzy&#39;s <a href="https://capzy.ai/web-scraper">Web Scraper API</a> bundles the proxy rotation, the challenge handling, and optional JavaScript rendering into one <code>/scrape</code> call. You point it at a page, it returns the content, and the CAPTCHA layer is handled for you.</p>
<p>The honest summary: you rarely &quot;bypass&quot; a CAPTCHA by outsmarting the puzzle. You bypass it by not looking like a bot, and by having a clean token ready for the times the site challenges you anyway. Get the proxy identity right, keep sessions coherent, pace like a human, and hand the puzzle to a solver only when one shows up.</p>
<p>Ready to stop fighting blocks? <a href="https://capzy.ai/auth/register">Create a free Capzy account</a> and route your next scrape through clean proxies with solving built in.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/bypass-captcha-web-scraping-without-getting-blocked",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vh3he2/how_to_bypass_captchas_during_web_scraping/",
"url_overridden_by_dest": "https://capzy.ai/blog/bypass-captcha-web-scraping-without-getting-blocked",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-06T12:55:50+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/wIvWZYPWdcgsO43O4t-PZydYzvUHDVg8BmQMJrZiX_I.jpeg?width=140&height=73&auto=webp&s=a9e00ead37d5d7503f547bf9ded93a78b0270ecd",
"preview_image": "https://external-preview.redd.it/wIvWZYPWdcgsO43O4t-PZydYzvUHDVg8BmQMJrZiX_I.jpeg?auto=webp&s=3711e8ddc367d8b92d2d21ead90b398b7bcb7290",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vh3eeq",
"fullname": "t3_1vh3eeq",
"scraped_at": "2026-08-18T09:36:17.072344+00:00",
"title": "What Is Browser Fingerprinting? A Field Guide for Developers",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# What Is Browser Fingerprinting? A Field Guide for Developers
Open your browser's dev tools, run `navigator.hardwareConcurrency`, and you get back a number. Eight, maybe sixteen. That single number is one thread in a much larger rope. Browser fingerprinting is the practice of pulling on dozens of those threads at once, `navigator` properties, screen geometry, installed fonts, the exact way your GPU renders a curve, and weaving them into an identifier that follows you around without a single cookie ever being set.
If you write automation, scrapers, or test harnesses, browser fingerprinting is the wall you keep bouncing off. You rotate IP addresses, you clear cookies, you swap the user-agent string, and the site still knows it is the same client. This guide walks through what actually gets measured, why it works, and how to stop your automation from lighting up like a flare.
# What browser fingerprinting reads
A fingerprint is not one signal. It is the *combination* of many weak signals into one strong one. Any single value, say a timezone of `America/New_York`, is shared by millions of people. But stack twenty such values together and the intersection narrows to a handful of machines, sometimes exactly one.
Here is the short list of what a modern fingerprinting script collects:
* **Canvas rendering.** The script draws text and shapes to an off-screen `<canvas>`, then reads the pixels back as a hash. Your GPU, driver version, font rasterizer, and anti-aliasing settings all nudge those pixels a fraction. Two machines rarely produce the identical hash.
* **WebGL.** Beyond canvas, WebGL exposes the GPU renderer string directly, think `ANGLE (NVIDIA GeForce RTX 4070 ...)`, plus a rendered-image hash and a long list of supported extensions.
* **Audio context.** An `OfflineAudioContext` processes a waveform and the floating-point output varies subtly by hardware and OS math libraries. Another hash.
* **Fonts.** By measuring the width of test strings, a script infers which fonts are installed. A stock Windows install and a design workstation look very different here.
* **navigator surface.** `hardwareConcurrency` (CPU threads), `deviceMemory` (RAM in GB), `platform`, `languages`, `userAgent`, and the newer `userAgentData` client hints.
* **Screen and display.** Resolution, color depth, `devicePixelRatio`, and available screen area.
* **Timezone and locale.** `Intl.DateTimeFormat().resolvedOptions().timeZone` and the `Accept-Language` ordering.
* **WebRTC.** Left unguarded, WebRTC can leak your real local and public IP addresses even behind a proxy, a classic way automation gets caught.
>The power of fingerprinting is statistical. No one value identifies you. The joint distribution of all of them does, and entropy adds up fast.
# Why swapping the user-agent fails
The most common first attempt at hiding is to change the user-agent header. It is right there, it is a string, and it seems to declare what browser you are. So people set it to a fresh Chrome UA and expect to blend in.
The problem is that the user-agent is one claim among hundreds of measurements, and the measurements do not lie the way the string can. If your UA announces Chrome 149 on Windows, but your WebGL renderer reports a Linux Mesa driver, your `platform` says `Linux x86_64`, and your font list has no Segoe UI, the contradiction is obvious. A fingerprinting service does not even need a blocklist. It just notices that the pieces do not belong to the same machine.
This is why coherence matters more than any single spoof. A believable browser is one where every surface agrees: the UA, the client-hint headers, `userAgentData`, the GPU string, the fonts, the screen, and the timezone all describe one plausible real computer that could exist and be sold.
# The three axes of a coherent fingerprint
When you generate a fingerprint properly, you are enforcing consistency along a few axes at once.
**Internal consistency.** The UA string, the `Sec-CH-UA` client hints, and `navigator.userAgentData` must be derived from the same source of truth. If they are assembled independently, they drift, and drift is detectable.
**Temporal consistency.** Parts have to co-exist in time. A GPU that shipped in 2024 paired with a browser build from 2021 is a red flag, because that pairing never happened on a real desktop. No OS version newer than what the hardware supports, no browser build older than the driver.
**Tier consistency.** Real machines are bought as balanced bundles. A flagship GPU comes with a recent high-core CPU, plenty of RAM, and a high-refresh display. A budget integrated GPU pairs with a modest CPU and a 1080p panel. A fingerprint that bolts a top-tier `deviceMemory` of 32 onto a `hardwareConcurrency` of 2 describes a machine nobody built.
Getting all three right by hand, for thousands of sessions, is where most automation projects quietly give up. That is the gap the [Fingerprint API](https://capzy.ai/fingerprints) fills: it synthesizes complete, internally-consistent fingerprints under these hard constraints, and hands them back as ready-to-inject data.
# Fetching a coherent fingerprint
The whole protocol is a single authenticated GET. You ask for a fresh fingerprint, filtered by platform and country, and parse the JSON.
curl -s "https://api.capzy.ai/fingerprint/generate?key=$CAPZY_KEY&format=chromium&tags=Windows&country=us"
In Python it is just as short:
import os, requests
fp = requests.get(
"https://api.capzy.ai/fingerprint/generate",
params={
"key": os.environ["CAPZY_KEY"],
"format": "chromium",
"tags": "Windows",
"country": "us",
},
timeout=15,
).json()
print(fp["userAgent"]["userAgent"])
print(fp["navigator"]["hardwareConcurrency"], "threads")
print(fp["intl"]["timeZone"])
The `chromium` format is flat and injection-ready: user-agent, matching `platform`, screen geometry, languages, timezone, and the `navigator` overrides you push into a browser context. Because `country=us` drives the languages, timezone, and speech-synthesis voices together, the locale surfaces line up with the IP you route through, which brings us to the next piece.
# Fingerprint, proxy, browser: the full disguise
A coherent fingerprint on its own is necessary but not sufficient. Three things have to agree for automation to read as a real user:
* **The fingerprint** describes a believable machine. That is the [Fingerprint API](https://capzy.ai/fingerprints).
* **The network path** matches the story that fingerprint tells. A US English fingerprint should egress from a US IP, not a datacenter range in another hemisphere. That is the [Proxies API](https://capzy.ai/proxies).
* **The execution environment** actually renders like the browser you claim to be, so canvas, WebGL, and audio hashes come from a real engine rather than a headless shim. That is the [Cloud Browser](https://capzy.ai/browser), a genuine Chrome you drive over CDP.
When you inject a coherent fingerprint into a real cloud Chrome and route it through a proxy whose geography matches the locale, the three layers stop contradicting each other. The canvas hash comes from a real GPU. The timezone matches the IP. The client hints match the UA. There is nothing left sticking out.
Fingerprinting is not magic and it is not unbeatable. It is arithmetic on measurements, and you win by making every measurement tell one consistent story. Start with a fingerprint you can trust.
Ready to try it? [Create a free account](https://capzy.ai/auth/register) and pull your first coherent fingerprint in a single GET.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>What Is Browser Fingerprinting? A Field Guide for Developers</h1>
<p>Open your browser&#39;s dev tools, run <code>navigator.hardwareConcurrency</code>, and you get back a number. Eight, maybe sixteen. That single number is one thread in a much larger rope. Browser fingerprinting is the practice of pulling on dozens of those threads at once, <code>navigator</code> properties, screen geometry, installed fonts, the exact way your GPU renders a curve, and weaving them into an identifier that follows you around without a single cookie ever being set.</p>
<p>If you write automation, scrapers, or test harnesses, browser fingerprinting is the wall you keep bouncing off. You rotate IP addresses, you clear cookies, you swap the user-agent string, and the site still knows it is the same client. This guide walks through what actually gets measured, why it works, and how to stop your automation from lighting up like a flare.</p>
<h1>What browser fingerprinting reads</h1>
<p>A fingerprint is not one signal. It is the <em>combination</em> of many weak signals into one strong one. Any single value, say a timezone of <code>America/New_York</code>, is shared by millions of people. But stack twenty such values together and the intersection narrows to a handful of machines, sometimes exactly one.</p>
<p>Here is the short list of what a modern fingerprinting script collects:</p>
<ul>
<li><strong>Canvas rendering.</strong> The script draws text and shapes to an off-screen <code>&lt;canvas&gt;</code>, then reads the pixels back as a hash. Your GPU, driver version, font rasterizer, and anti-aliasing settings all nudge those pixels a fraction. Two machines rarely produce the identical hash.</li>
<li><strong>WebGL.</strong> Beyond canvas, WebGL exposes the GPU renderer string directly, think <code>ANGLE (NVIDIA GeForce RTX 4070 ...)</code>, plus a rendered-image hash and a long list of supported extensions.</li>
<li><strong>Audio context.</strong> An <code>OfflineAudioContext</code> processes a waveform and the floating-point output varies subtly by hardware and OS math libraries. Another hash.</li>
<li><strong>Fonts.</strong> By measuring the width of test strings, a script infers which fonts are installed. A stock Windows install and a design workstation look very different here.</li>
<li><strong>navigator surface.</strong> <code>hardwareConcurrency</code> (CPU threads), <code>deviceMemory</code> (RAM in GB), <code>platform</code>, <code>languages</code>, <code>userAgent</code>, and the newer <code>userAgentData</code> client hints.</li>
<li><strong>Screen and display.</strong> Resolution, color depth, <code>devicePixelRatio</code>, and available screen area.</li>
<li><strong>Timezone and locale.</strong> <code>Intl.DateTimeFormat().resolvedOptions().timeZone</code> and the <code>Accept-Language</code> ordering.</li>
<li><strong>WebRTC.</strong> Left unguarded, WebRTC can leak your real local and public IP addresses even behind a proxy, a classic way automation gets caught.</li>
</ul>
<blockquote>
<p>The power of fingerprinting is statistical. No one value identifies you. The joint distribution of all of them does, and entropy adds up fast.</p>
</blockquote>
<h1>Why swapping the user-agent fails</h1>
<p>The most common first attempt at hiding is to change the user-agent header. It is right there, it is a string, and it seems to declare what browser you are. So people set it to a fresh Chrome UA and expect to blend in.</p>
<p>The problem is that the user-agent is one claim among hundreds of measurements, and the measurements do not lie the way the string can. If your UA announces Chrome 149 on Windows, but your WebGL renderer reports a Linux Mesa driver, your <code>platform</code> says <code>Linux x86_64</code>, and your font list has no Segoe UI, the contradiction is obvious. A fingerprinting service does not even need a blocklist. It just notices that the pieces do not belong to the same machine.</p>
<p>This is why coherence matters more than any single spoof. A believable browser is one where every surface agrees: the UA, the client-hint headers, <code>userAgentData</code>, the GPU string, the fonts, the screen, and the timezone all describe one plausible real computer that could exist and be sold.</p>
<h1>The three axes of a coherent fingerprint</h1>
<p>When you generate a fingerprint properly, you are enforcing consistency along a few axes at once.</p>
<p><strong>Internal consistency.</strong> The UA string, the <code>Sec-CH-UA</code> client hints, and <code>navigator.userAgentData</code> must be derived from the same source of truth. If they are assembled independently, they drift, and drift is detectable.</p>
<p><strong>Temporal consistency.</strong> Parts have to co-exist in time. A GPU that shipped in 2024 paired with a browser build from 2021 is a red flag, because that pairing never happened on a real desktop. No OS version newer than what the hardware supports, no browser build older than the driver.</p>
<p><strong>Tier consistency.</strong> Real machines are bought as balanced bundles. A flagship GPU comes with a recent high-core CPU, plenty of RAM, and a high-refresh display. A budget integrated GPU pairs with a modest CPU and a 1080p panel. A fingerprint that bolts a top-tier <code>deviceMemory</code> of 32 onto a <code>hardwareConcurrency</code> of 2 describes a machine nobody built.</p>
<p>Getting all three right by hand, for thousands of sessions, is where most automation projects quietly give up. That is the gap the <a href="https://capzy.ai/fingerprints">Fingerprint API</a> fills: it synthesizes complete, internally-consistent fingerprints under these hard constraints, and hands them back as ready-to-inject data.</p>
<h1>Fetching a coherent fingerprint</h1>
<p>The whole protocol is a single authenticated GET. You ask for a fresh fingerprint, filtered by platform and country, and parse the JSON.</p>
<pre><code>curl -s &quot;https://api.capzy.ai/fingerprint/generate?key=$CAPZY_KEY&amp;format=chromium&amp;tags=Windows&amp;country=us&quot;
</code></pre>
<p>In Python it is just as short:</p>
<pre><code>import os, requests
fp = requests.get(
&quot;https://api.capzy.ai/fingerprint/generate&quot;,
params={
&quot;key&quot;: os.environ[&quot;CAPZY_KEY&quot;],
&quot;format&quot;: &quot;chromium&quot;,
&quot;tags&quot;: &quot;Windows&quot;,
&quot;country&quot;: &quot;us&quot;,
},
timeout=15,
).json()
print(fp[&quot;userAgent&quot;][&quot;userAgent&quot;])
print(fp[&quot;navigator&quot;][&quot;hardwareConcurrency&quot;], &quot;threads&quot;)
print(fp[&quot;intl&quot;][&quot;timeZone&quot;])
</code></pre>
<p>The <code>chromium</code> format is flat and injection-ready: user-agent, matching <code>platform</code>, screen geometry, languages, timezone, and the <code>navigator</code> overrides you push into a browser context. Because <code>country=us</code> drives the languages, timezone, and speech-synthesis voices together, the locale surfaces line up with the IP you route through, which brings us to the next piece.</p>
<h1>Fingerprint, proxy, browser: the full disguise</h1>
<p>A coherent fingerprint on its own is necessary but not sufficient. Three things have to agree for automation to read as a real user:</p>
<ul>
<li><strong>The fingerprint</strong> describes a believable machine. That is the <a href="https://capzy.ai/fingerprints">Fingerprint API</a>.</li>
<li><strong>The network path</strong> matches the story that fingerprint tells. A US English fingerprint should egress from a US IP, not a datacenter range in another hemisphere. That is the <a href="https://capzy.ai/proxies">Proxies API</a>.</li>
<li><strong>The execution environment</strong> actually renders like the browser you claim to be, so canvas, WebGL, and audio hashes come from a real engine rather than a headless shim. That is the <a href="https://capzy.ai/browser">Cloud Browser</a>, a genuine Chrome you drive over CDP.</li>
</ul>
<p>When you inject a coherent fingerprint into a real cloud Chrome and route it through a proxy whose geography matches the locale, the three layers stop contradicting each other. The canvas hash comes from a real GPU. The timezone matches the IP. The client hints match the UA. There is nothing left sticking out.</p>
<p>Fingerprinting is not magic and it is not unbeatable. It is arithmetic on measurements, and you win by making every measurement tell one consistent story. Start with a fingerprint you can trust.</p>
<p>Ready to try it? <a href="https://capzy.ai/auth/register">Create a free account</a> and pull your first coherent fingerprint in a single GET.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/what-is-browser-fingerprinting",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vh3eeq/what_is_browser_fingerprinting_a_field_guide_for/",
"url_overridden_by_dest": "https://capzy.ai/blog/what-is-browser-fingerprinting",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-06T12:52:18+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/sQAfiZO_Qe9aChiduf-Z8iv0EIty_0ukMlPBfmL4SPk.jpeg?width=140&height=73&auto=webp&s=a56dfcc3982c26bf6f2b0537f77af2151722b3c0",
"preview_image": "https://external-preview.redd.it/sQAfiZO_Qe9aChiduf-Z8iv0EIty_0ukMlPBfmL4SPk.jpeg?auto=webp&s=f77911156aec186bd80515e38c40a76f5a63b94b",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vgdmv1",
"fullname": "t3_1vgdmv1",
"scraped_at": "2026-08-18T09:36:17.072351+00:00",
"title": "Connecting Playwright and Puppeteer to a Remote Browser over CDP",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# Connecting Playwright and Puppeteer to a Remote Browser over CDP
The nicest thing about a cloud browser is how little your code has to change. If you already drive Chrome with Playwright or Puppeteer, connecting to a remote browser over CDP is a one-line swap: replace the local `launch` with a `connect` to a WebSocket endpoint. This guide shows the exact connect call in each tool, then covers the query parameters that route your traffic through a proxy and attach a persistent profile.
CDP, the Chrome DevTools Protocol, is the wire language your automation tools already speak to Chrome. When Chrome runs remotely, that same protocol just travels over a `wss://` WebSocket instead of a local pipe. Point your client at the endpoint and everything downstream, `goto`, `click`, `screenshot`, works unchanged.
# The endpoint
After you create an account and pick a Browser plan, you generate connection credentials in the [Browser dashboard](https://capzy.ai/dashboard/browser). The dashboard hands you a ready-to-paste endpoint with your username and password inline:
wss://USER:PASS@browser.capzy.ai/cdp
Rotating your password invalidates the old string, so store it somewhere your automation can read it (an environment variable is the usual move). Everything below assumes you have this URL.
# Playwright (Python)
Playwright exposes `connect_over_cdp`. Swap it in for `p.chromium.launch()` and you are driving a remote Chrome 149:
import os
from playwright.sync_api import sync_playwright
CDP = os.environ["CAPZY_CDP"] # wss://USER:PASS@browser.capzy.ai/cdp
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(CDP)
ctx = browser.contexts[0] # the remote browser starts with a context
page = ctx.new_page()
page.goto("https://httpbin.org/ip")
print(page.inner_text("pre"))
browser.close()
One gotcha worth knowing: with `connect_over_cdp` the browser already has a default context, so reach for `browser.contexts[0]` rather than calling `new_context()` and wondering why your first tab is empty.
The async API is the same shape:
import asyncio, os
from playwright.async_api import async_playwright
async def main():
async with async_playwright() as p:
browser = await p.chromium.connect_over_cdp(os.environ["CAPZY_CDP"])
page = await browser.contexts[0].new_page()
await page.goto("https://example.com")
print(await page.title())
await browser.close()
asyncio.run(main())
# Puppeteer (Node.js)
Puppeteer uses `puppeteer.connect` with a `browserWSEndpoint`:
const puppeteer = require("puppeteer");
(async () => {
const browser = await puppeteer.connect({
browserWSEndpoint: process.env.CAPZY_CDP, // wss://USER:PASS@browser.capzy.ai/cdp
});
const page = await browser.newPage();
await page.goto("https://example.com");
console.log(await page.title());
await browser.disconnect();
})();
Note `disconnect()` rather than `close()` here: you are detaching from a remote browser you did not launch, not killing a local process.
# Playwright (Node.js)
const { chromium } = require("playwright");
(async () => {
const browser = await chromium.connectOverCDP(process.env.CAPZY_CDP);
const page = await browser.contexts()[0].newPage();
await page.goto("https://example.com");
console.log(await page.title());
await browser.close();
})();
# Selenium
Selenium can attach to a remote CDP session by setting the `se:cdpUrl` capability:
opts.set_capability("se:cdpUrl", "wss://USER:PASS@browser.capzy.ai/cdp")
One aside before the parameters: everything here connects you to plain remote Chrome, which is exactly what you want for rendering and automation at scale. If instead you need a stealth-hardened browser built to defeat aggressive fingerprinting, that is a different product. [Capium](https://capzy.ai/capium) is Capzy's proprietary stealth browser: you install it with `pip install capium` and drive it locally like any automation browser, licensed per concurrent session. Reach for it when maximum evasion matters more than a remote connection.
# Steering traffic with query parameters
The connect step is only half the story. The other half is telling the browser *how* to behave, and you do that by appending query parameters to the `/cdp` endpoint. All of them are optional; without them the browser uses the defaults you set in the dashboard.
**Route through a proxy.** Append `?proxy=capzy` to send egress through Capzy proxy data, `?proxy=capzy-us` to target a specific country by its two-letter code, or `?proxy=<your-proxy-id>` to use a proxy you saved in the dashboard:
CDP = "wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-de"
browser = p.chromium.connect_over_cdp(CDP)
**Attach a persistent profile.** Append `?profile=<profile-id>` so cookies, local storage, and login state carry over to the next connection. Log in once, and future sessions on that profile skip the login:
CDP = "wss://USER:PASS@browser.capzy.ai/cdp?profile=abc123"
**Combine them** with `&`, the same way any query string works:
CDP = "wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-de&profile=abc123"
>The connection string is the whole control surface. Which country you exit from and which saved session you resume are both just query parameters on the endpoint you already have.
# Verifying the connection actually worked
Before you build a whole pipeline on top, prove two things: that you are driving the remote browser, and that your traffic exits where you expect. The quickest check is to hit an IP echo endpoint and confirm the address is the proxy's, not your own machine's.
from playwright.sync_api import sync_playwright
CDP = "wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-us"
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(CDP)
page = browser.contexts[0].new_page()
page.goto("https://httpbin.org/ip")
print("egress IP:", page.inner_text("pre"))
page.goto("https://httpbin.org/headers")
print(page.inner_text("pre")) # confirm the UA the site actually sees
browser.close()
If the egress IP is a US address and not your workstation's, the proxy parameter took effect. If it is your own IP, re-check the `?proxy=` value and that your plan has proxy data active.
# Common connection snags
A few issues come up often enough to list:
* **Empty first tab with Playwright.** If `new_context()` gives you a blank browser, remember the remote Chrome already has a context. Use `browser.contexts[0]` and call `new_page()` on it.
* **Auth failures on the WebSocket.** The username and password live *inline* in the `wss://USER:PASS@...` URL. If you rotated the password in the dashboard, the old string stops working immediately; regenerate and update your environment variable.
* **Puppeteer hanging on exit.** Use `disconnect()`, not `close()`. You are detaching from a browser you did not launch, and calling `close()` on a remote session behaves differently than you expect.
* **Proxy parameter ignored.** `?proxy=capzy` and `?proxy=capzy-<cc>` require an active plan with proxy data. Without it, the browser falls back to the default egress from your dashboard configuration.
* **Session state not persisting.** A bare connection does not save cookies across runs. Attach `?profile=<id>` so storage and login state survive.
Keeping these five in mind saves most of the first-hour confusion people hit when they move from a local `launch` to a remote `connect`.
# Why this pairs so well with a fingerprint
Connecting is easy; blending in is the real work, and a raw connection is only one layer. The remote Chrome renders canvas, WebGL, and audio from a real graphics stack, which is a strong start. On top of that you still want a coherent machine identity and a matching network path.
That is where the pieces click together. Pull a fingerprint from the [Fingerprint API](https://capzy.ai/fingerprints) whose locale matches the country you selected in `?proxy=capzy-<cc>`, inject its user-agent and `navigator` overrides into the context, and route the whole session through that geography. Now the browser is real, the machine profile is coherent, and the network exit lines up with the timezone and language. Three layers, one consistent story.
For example, a fingerprint generated with `country=de` gives you German languages, `Europe/Berlin` timezone, and German voices; pair it with `?proxy=capzy-de` and the disguise holds together. The [Proxies API](https://capzy.ai/proxies) covers the network side, and the [Solver API](https://capzy.ai/solvers) handles any captcha that still appears along the way.
# A quick checklist
* Swap `launch` for `connect_over_cdp` / `connect`.
* With Playwright, use `browser.contexts[0]`, not a fresh `new_context()`.
* With Puppeteer, `disconnect()` instead of `close()`.
* Add `?proxy=` to control egress geography.
* Add `?profile=` to persist sessions across runs.
* Layer a coherent fingerprint on top so the browser, machine, and network agree.
That is the entire integration. Change one line, add a couple of query parameters, and your existing automation is driving a real remote Chrome.
[Sign up](https://capzy.ai/auth/register), generate your connection credentials, and connect in the next five minutes.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>Connecting Playwright and Puppeteer to a Remote Browser over CDP</h1>
<p>The nicest thing about a cloud browser is how little your code has to change. If you already drive Chrome with Playwright or Puppeteer, connecting to a remote browser over CDP is a one-line swap: replace the local <code>launch</code> with a <code>connect</code> to a WebSocket endpoint. This guide shows the exact connect call in each tool, then covers the query parameters that route your traffic through a proxy and attach a persistent profile.</p>
<p>CDP, the Chrome DevTools Protocol, is the wire language your automation tools already speak to Chrome. When Chrome runs remotely, that same protocol just travels over a <code>wss://</code> WebSocket instead of a local pipe. Point your client at the endpoint and everything downstream, <code>goto</code>, <code>click</code>, <code>screenshot</code>, works unchanged.</p>
<h1>The endpoint</h1>
<p>After you create an account and pick a Browser plan, you generate connection credentials in the <a href="https://capzy.ai/dashboard/browser">Browser dashboard</a>. The dashboard hands you a ready-to-paste endpoint with your username and password inline:</p>
<pre><code>wss://USER:PASS@browser.capzy.ai/cdp
</code></pre>
<p>Rotating your password invalidates the old string, so store it somewhere your automation can read it (an environment variable is the usual move). Everything below assumes you have this URL.</p>
<h1>Playwright (Python)</h1>
<p>Playwright exposes <code>connect_over_cdp</code>. Swap it in for <code>p.chromium.launch()</code> and you are driving a remote Chrome 149:</p>
<pre><code>import os
from playwright.sync_api import sync_playwright
CDP = os.environ[&quot;CAPZY_CDP&quot;] # wss://USER:PASS@browser.capzy.ai/cdp
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(CDP)
ctx = browser.contexts[0] # the remote browser starts with a context
page = ctx.new_page()
page.goto(&quot;https://httpbin.org/ip&quot;)
print(page.inner_text(&quot;pre&quot;))
browser.close()
</code></pre>
<p>One gotcha worth knowing: with <code>connect_over_cdp</code> the browser already has a default context, so reach for <code>browser.contexts[0]</code> rather than calling <code>new_context()</code> and wondering why your first tab is empty.</p>
<p>The async API is the same shape:</p>
<pre><code>import asyncio, os
from playwright.async_api import async_playwright
async def main():
async with async_playwright() as p:
browser = await p.chromium.connect_over_cdp(os.environ[&quot;CAPZY_CDP&quot;])
page = await browser.contexts[0].new_page()
await page.goto(&quot;https://example.com&quot;)
print(await page.title())
await browser.close()
asyncio.run(main())
</code></pre>
<h1>Puppeteer (Node.js)</h1>
<p>Puppeteer uses <code>puppeteer.connect</code> with a <code>browserWSEndpoint</code>:</p>
<pre><code>const puppeteer = require(&quot;puppeteer&quot;);
(async () =&gt; {
const browser = await puppeteer.connect({
browserWSEndpoint: process.env.CAPZY_CDP, // wss://USER:PASS@browser.capzy.ai/cdp
});
const page = await browser.newPage();
await page.goto(&quot;https://example.com&quot;);
console.log(await page.title());
await browser.disconnect();
})();
</code></pre>
<p>Note <code>disconnect()</code> rather than <code>close()</code> here: you are detaching from a remote browser you did not launch, not killing a local process.</p>
<h1>Playwright (Node.js)</h1>
<pre><code>const { chromium } = require(&quot;playwright&quot;);
(async () =&gt; {
const browser = await chromium.connectOverCDP(process.env.CAPZY_CDP);
const page = await browser.contexts()[0].newPage();
await page.goto(&quot;https://example.com&quot;);
console.log(await page.title());
await browser.close();
})();
</code></pre>
<h1>Selenium</h1>
<p>Selenium can attach to a remote CDP session by setting the <code>se:cdpUrl</code> capability:</p>
<pre><code>opts.set_capability(&quot;se:cdpUrl&quot;, &quot;wss://USER:PASS@browser.capzy.ai/cdp&quot;)
</code></pre>
<p>One aside before the parameters: everything here connects you to plain remote Chrome, which is exactly what you want for rendering and automation at scale. If instead you need a stealth-hardened browser built to defeat aggressive fingerprinting, that is a different product. <a href="https://capzy.ai/capium">Capium</a> is Capzy&#39;s proprietary stealth browser: you install it with <code>pip install capium</code> and drive it locally like any automation browser, licensed per concurrent session. Reach for it when maximum evasion matters more than a remote connection.</p>
<h1>Steering traffic with query parameters</h1>
<p>The connect step is only half the story. The other half is telling the browser <em>how</em> to behave, and you do that by appending query parameters to the <code>/cdp</code> endpoint. All of them are optional; without them the browser uses the defaults you set in the dashboard.</p>
<p><strong>Route through a proxy.</strong> Append <code>?proxy=capzy</code> to send egress through Capzy proxy data, <code>?proxy=capzy-us</code> to target a specific country by its two-letter code, or <code>?proxy=&lt;your-proxy-id&gt;</code> to use a proxy you saved in the dashboard:</p>
<pre><code>CDP = &quot;wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-de&quot;
browser = p.chromium.connect_over_cdp(CDP)
</code></pre>
<p><strong>Attach a persistent profile.</strong> Append <code>?profile=&lt;profile-id&gt;</code> so cookies, local storage, and login state carry over to the next connection. Log in once, and future sessions on that profile skip the login:</p>
<pre><code>CDP = &quot;wss://USER:PASS@browser.capzy.ai/cdp?profile=abc123&quot;
</code></pre>
<p><strong>Combine them</strong> with <code>&amp;</code>, the same way any query string works:</p>
<pre><code>CDP = &quot;wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-de&amp;profile=abc123&quot;
</code></pre>
<blockquote>
<p>The connection string is the whole control surface. Which country you exit from and which saved session you resume are both just query parameters on the endpoint you already have.</p>
</blockquote>
<h1>Verifying the connection actually worked</h1>
<p>Before you build a whole pipeline on top, prove two things: that you are driving the remote browser, and that your traffic exits where you expect. The quickest check is to hit an IP echo endpoint and confirm the address is the proxy&#39;s, not your own machine&#39;s.</p>
<pre><code>from playwright.sync_api import sync_playwright
CDP = &quot;wss://USER:PASS@browser.capzy.ai/cdp?proxy=capzy-us&quot;
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(CDP)
page = browser.contexts[0].new_page()
page.goto(&quot;https://httpbin.org/ip&quot;)
print(&quot;egress IP:&quot;, page.inner_text(&quot;pre&quot;))
page.goto(&quot;https://httpbin.org/headers&quot;)
print(page.inner_text(&quot;pre&quot;)) # confirm the UA the site actually sees
browser.close()
</code></pre>
<p>If the egress IP is a US address and not your workstation&#39;s, the proxy parameter took effect. If it is your own IP, re-check the <code>?proxy=</code> value and that your plan has proxy data active.</p>
<h1>Common connection snags</h1>
<p>A few issues come up often enough to list:</p>
<ul>
<li><strong>Empty first tab with Playwright.</strong> If <code>new_context()</code> gives you a blank browser, remember the remote Chrome already has a context. Use <code>browser.contexts[0]</code> and call <code>new_page()</code> on it.</li>
<li><strong>Auth failures on the WebSocket.</strong> The username and password live <em>inline</em> in the <code>wss://USER:PASS@...</code> URL. If you rotated the password in the dashboard, the old string stops working immediately; regenerate and update your environment variable.</li>
<li><strong>Puppeteer hanging on exit.</strong> Use <code>disconnect()</code>, not <code>close()</code>. You are detaching from a browser you did not launch, and calling <code>close()</code> on a remote session behaves differently than you expect.</li>
<li><strong>Proxy parameter ignored.</strong> <code>?proxy=capzy</code> and <code>?proxy=capzy-&lt;cc&gt;</code> require an active plan with proxy data. Without it, the browser falls back to the default egress from your dashboard configuration.</li>
<li><strong>Session state not persisting.</strong> A bare connection does not save cookies across runs. Attach <code>?profile=&lt;id&gt;</code> so storage and login state survive.</li>
</ul>
<p>Keeping these five in mind saves most of the first-hour confusion people hit when they move from a local <code>launch</code> to a remote <code>connect</code>.</p>
<h1>Why this pairs so well with a fingerprint</h1>
<p>Connecting is easy; blending in is the real work, and a raw connection is only one layer. The remote Chrome renders canvas, WebGL, and audio from a real graphics stack, which is a strong start. On top of that you still want a coherent machine identity and a matching network path.</p>
<p>That is where the pieces click together. Pull a fingerprint from the <a href="https://capzy.ai/fingerprints">Fingerprint API</a> whose locale matches the country you selected in <code>?proxy=capzy-&lt;cc&gt;</code>, inject its user-agent and <code>navigator</code> overrides into the context, and route the whole session through that geography. Now the browser is real, the machine profile is coherent, and the network exit lines up with the timezone and language. Three layers, one consistent story.</p>
<p>For example, a fingerprint generated with <code>country=de</code> gives you German languages, <code>Europe/Berlin</code> timezone, and German voices; pair it with <code>?proxy=capzy-de</code> and the disguise holds together. The <a href="https://capzy.ai/proxies">Proxies API</a> covers the network side, and the <a href="https://capzy.ai/solvers">Solver API</a> handles any captcha that still appears along the way.</p>
<h1>A quick checklist</h1>
<ul>
<li>Swap <code>launch</code> for <code>connect_over_cdp</code> / <code>connect</code>.</li>
<li>With Playwright, use <code>browser.contexts[0]</code>, not a fresh <code>new_context()</code>.</li>
<li>With Puppeteer, <code>disconnect()</code> instead of <code>close()</code>.</li>
<li>Add <code>?proxy=</code> to control egress geography.</li>
<li>Add <code>?profile=</code> to persist sessions across runs.</li>
<li>Layer a coherent fingerprint on top so the browser, machine, and network agree.</li>
</ul>
<p>That is the entire integration. Change one line, add a couple of query parameters, and your existing automation is driving a real remote Chrome.</p>
<p><a href="https://capzy.ai/auth/register">Sign up</a>, generate your connection credentials, and connect in the next five minutes.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/connect-playwright-puppeteer-remote-cdp",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vgdmv1/connecting_playwright_and_puppeteer_to_a_remote/",
"url_overridden_by_dest": "https://capzy.ai/blog/connect-playwright-puppeteer-remote-cdp",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-05T17:16:56+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/Fw6D5AIEiqLBk46Vdqe-FDSbP3SIvw1DShUlyz6kQmc.jpeg?width=140&height=73&auto=webp&s=11982c89904c04acfe7734c9db1370cbf321b98c",
"preview_image": "https://external-preview.redd.it/Fw6D5AIEiqLBk46Vdqe-FDSbP3SIvw1DShUlyz6kQmc.jpeg?auto=webp&s=aea637181728295151ea17a395d0b089dc9412b8",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vgdljo",
"fullname": "t3_1vgdljo",
"scraped_at": "2026-08-18T09:36:17.072358+00:00",
"title": "Proxy Types Explained: Datacenter, Residential, ISP, and Mobile",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# Proxy Types Explained: Datacenter, Residential, ISP, and Mobile
Every proxy type solves the same basic problem: it puts a different IP address between you and the site you are talking to. But not all IPs are treated the same way once they arrive. A request from a data center gets scored differently than a request from a phone on a cellular network, and that difference decides whether you get a `200 OK` or a block page. Understanding proxy types is the fastest way to stop wasting bandwidth on the wrong pool.
The four classes worth knowing are datacenter, residential, ISP, and mobile. On the [Proxies API](https://capzy.ai/proxies) these map to four pools you can switch between with a credential change: Datacenter, Residential, Premium Residential, and Mobile. Same endpoint, same account, four reputations to choose from.
# How the proxy types differ
**Datacenter IPs** come from servers in hosting facilities. They are fast, cheap, and effectively unlimited, because a single rack can hold thousands of addresses. The catch is that anyone can see they belong to a hosting provider. A site can look up the ASN behind the IP, notice it is registered to a cloud company rather than a home ISP, and decide to challenge or block it. For targets that do not care where traffic comes from, datacenter is the right call. It is the cheapest per GB and the lowest latency, often 30 to 80 ms of added round trip instead of the 150 ms or more you see on residential paths.
**Residential IPs** are real addresses assigned by home internet providers to households. When a site sees one, it looks like an ordinary visitor on a home connection, because that is exactly what the address belongs to. This is the default for most scraping and anti-bot work. You give up some speed and pay more per GB, but the trust level is in a completely different bracket. If a target runs any kind of IP reputation check, residential is usually where you start.
**ISP proxies** sit between the two. They are datacenter-hosted machines that carry IP addresses registered to a consumer internet provider rather than a hosting company. So you get the speed and stability of a server with an IP that reads as residential on an ASN lookup. On our platform this role is filled by the Premium Residential pool, a cleaner, higher-trust set of addresses aimed at the hardest targets, where an ordinary residential IP that has been hammered all day might already be flagged.
**Mobile IPs** ride on carrier 4G and 5G networks. They carry the highest trust of any type, for one specific reason: carriers use CGNAT, so hundreds or thousands of real phones share a single public IP at any moment. A site that blocks a mobile IP risks blocking a chunk of genuine customers, so most of them tread carefully. That makes mobile the strongest option for mobile-first apps and sites that lean hard on IP reputation. It is also the most expensive per GB, so you save it for the targets that actually need it.
>Rule of thumb: start on the cheapest pool that works, and only climb the trust ladder when a target starts blocking. Paying mobile rates for a site that accepts datacenter traffic is money set on fire.
# How sites tell the types apart
It helps to know what a target is actually looking at when it scores your IP, because that is what makes one type pass where another fails. The first signal is the ASN, the network the address belongs to. A quick lookup tells the site whether the IP is registered to a hosting company, a consumer ISP, or a mobile carrier, and that single fact carries most of the weight. Datacenter ranges are public and well-catalogued, so a site can block entire hosting networks with one rule. Consumer ISP ranges are harder to paint with a broad brush, because banning them risks banning real customers.
Beyond the ASN, sites keep reputation scores on individual addresses. An IP that has sent a flood of automated-looking traffic in the last hour carries a worse score than one that has been quiet, regardless of type. This is why a residential IP that has been hammered all day can still get challenged, and why the Premium Residential pool exists: cleaner addresses that have not been run into the ground. Mobile sits on top because CGNAT makes per-IP reputation almost useless to the site. Punishing one mobile IP punishes a crowd of real phones behind it, so most sites hold their fire.
The practical takeaway is that type and freshness both matter. A fresh datacenter IP can beat a burned residential one on a lenient target, and no type wins if you overload a single address. Spreading load across a pool is as important as picking the right class.
# Picking a proxy in practice
The nice thing about a single endpoint is that switching type is a one-line change. The connection string is always the same host and port; your username selects the account and its pool.
# Datacenter pool, US exit IPs
curl -x http://USERNAME__cr.us:PASSWORD@proxy.capzy.ai:823 https://ipinfo.io/json
In Python the same swap is just a different set of credentials in the proxies dict:
import requests
proxies = {
"http": "http://USERNAME:PASSWORD@proxy.capzy.ai:823",
"https": "http://USERNAME:PASSWORD@proxy.capzy.ai:823",
}
r = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=30)
print(r.json()["ip"]) # a pool IP, not yours
To target a country, append `__cr.<cc>` to the username with a two-letter ISO code, for example `USERNAME__cr.de` for German exits. No other code change is needed.
# Cost versus success rate
The real decision is a trade between price and pass rate. Datacenter might run you a fraction of residential per GB but fail on a protected target 8 times out of 10, so your effective cost per successful request is actually higher. Residential might cost several times more per GB but pass 95 percent of the time, which makes it cheaper per useful result. Run a small test batch of a few hundred requests per pool against your specific target and measure the success rate before you commit a plan to it. The winner is almost never obvious from price alone.
A common pattern is a fallback ladder: try datacenter first, and only escalate a request to residential or mobile when the cheaper pool gets challenged. That keeps your average cost near the floor while still clearing the hard cases. Concretely, you might route 80 percent of a crawl through datacenter, catch the 20 percent that get blocked, and retry only those on residential. If datacenter costs a fifth of residential per GB, that split can cut your effective bill by more than half compared to running everything on residential from the start.
Here is a quick summary of where each type lands:
* **Datacenter** is cheapest, fastest, lowest trust. Use it for lenient targets and high volume.
* **Residential** is the default balance of trust and price. Use it when a target checks IP reputation.
* **Premium Residential** is the ISP-style higher-trust pool. Use it for tough targets that burn ordinary residential IPs.
* **Mobile** is the most expensive and the most trusted. Use it for mobile-first sites and the hardest reputation checks.
If the target throws a captcha rather than a hard IP block, a better IP alone will not finish the job. Pair the proxy with the [Solver API](https://capzy.ai/solvers) so the challenge gets answered and the request completes. And if you would rather not manage pools and challenges yourself, the [Web Scraper API](https://capzy.ai/web-scraper) rolls egress, rendering, and anti-bot handling into a single call.
Proxy type is not a setting you pick once and forget. It is a per-target decision, and the right answer shifts as sites tighten their defenses. Start cheap, measure, and climb only when you have to.
Ready to test all four pools on one account? [Sign up free](https://capzy.ai/auth/register) and route your first request in under a minute.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>Proxy Types Explained: Datacenter, Residential, ISP, and Mobile</h1>
<p>Every proxy type solves the same basic problem: it puts a different IP address between you and the site you are talking to. But not all IPs are treated the same way once they arrive. A request from a data center gets scored differently than a request from a phone on a cellular network, and that difference decides whether you get a <code>200 OK</code> or a block page. Understanding proxy types is the fastest way to stop wasting bandwidth on the wrong pool.</p>
<p>The four classes worth knowing are datacenter, residential, ISP, and mobile. On the <a href="https://capzy.ai/proxies">Proxies API</a> these map to four pools you can switch between with a credential change: Datacenter, Residential, Premium Residential, and Mobile. Same endpoint, same account, four reputations to choose from.</p>
<h1>How the proxy types differ</h1>
<p><strong>Datacenter IPs</strong> come from servers in hosting facilities. They are fast, cheap, and effectively unlimited, because a single rack can hold thousands of addresses. The catch is that anyone can see they belong to a hosting provider. A site can look up the ASN behind the IP, notice it is registered to a cloud company rather than a home ISP, and decide to challenge or block it. For targets that do not care where traffic comes from, datacenter is the right call. It is the cheapest per GB and the lowest latency, often 30 to 80 ms of added round trip instead of the 150 ms or more you see on residential paths.</p>
<p><strong>Residential IPs</strong> are real addresses assigned by home internet providers to households. When a site sees one, it looks like an ordinary visitor on a home connection, because that is exactly what the address belongs to. This is the default for most scraping and anti-bot work. You give up some speed and pay more per GB, but the trust level is in a completely different bracket. If a target runs any kind of IP reputation check, residential is usually where you start.</p>
<p><strong>ISP proxies</strong> sit between the two. They are datacenter-hosted machines that carry IP addresses registered to a consumer internet provider rather than a hosting company. So you get the speed and stability of a server with an IP that reads as residential on an ASN lookup. On our platform this role is filled by the Premium Residential pool, a cleaner, higher-trust set of addresses aimed at the hardest targets, where an ordinary residential IP that has been hammered all day might already be flagged.</p>
<p><strong>Mobile IPs</strong> ride on carrier 4G and 5G networks. They carry the highest trust of any type, for one specific reason: carriers use CGNAT, so hundreds or thousands of real phones share a single public IP at any moment. A site that blocks a mobile IP risks blocking a chunk of genuine customers, so most of them tread carefully. That makes mobile the strongest option for mobile-first apps and sites that lean hard on IP reputation. It is also the most expensive per GB, so you save it for the targets that actually need it.</p>
<blockquote>
<p>Rule of thumb: start on the cheapest pool that works, and only climb the trust ladder when a target starts blocking. Paying mobile rates for a site that accepts datacenter traffic is money set on fire.</p>
</blockquote>
<h1>How sites tell the types apart</h1>
<p>It helps to know what a target is actually looking at when it scores your IP, because that is what makes one type pass where another fails. The first signal is the ASN, the network the address belongs to. A quick lookup tells the site whether the IP is registered to a hosting company, a consumer ISP, or a mobile carrier, and that single fact carries most of the weight. Datacenter ranges are public and well-catalogued, so a site can block entire hosting networks with one rule. Consumer ISP ranges are harder to paint with a broad brush, because banning them risks banning real customers.</p>
<p>Beyond the ASN, sites keep reputation scores on individual addresses. An IP that has sent a flood of automated-looking traffic in the last hour carries a worse score than one that has been quiet, regardless of type. This is why a residential IP that has been hammered all day can still get challenged, and why the Premium Residential pool exists: cleaner addresses that have not been run into the ground. Mobile sits on top because CGNAT makes per-IP reputation almost useless to the site. Punishing one mobile IP punishes a crowd of real phones behind it, so most sites hold their fire.</p>
<p>The practical takeaway is that type and freshness both matter. A fresh datacenter IP can beat a burned residential one on a lenient target, and no type wins if you overload a single address. Spreading load across a pool is as important as picking the right class.</p>
<h1>Picking a proxy in practice</h1>
<p>The nice thing about a single endpoint is that switching type is a one-line change. The connection string is always the same host and port; your username selects the account and its pool.</p>
<pre><code># Datacenter pool, US exit IPs
curl -x http://USERNAME__cr.us:PASSWORD@proxy.capzy.ai:823 https://ipinfo.io/json
</code></pre>
<p>In Python the same swap is just a different set of credentials in the proxies dict:</p>
<pre><code>import requests
proxies = {
&quot;http&quot;: &quot;http://USERNAME:PASSWORD@proxy.capzy.ai:823&quot;,
&quot;https&quot;: &quot;http://USERNAME:PASSWORD@proxy.capzy.ai:823&quot;,
}
r = requests.get(&quot;https://ipinfo.io/json&quot;, proxies=proxies, timeout=30)
print(r.json()[&quot;ip&quot;]) # a pool IP, not yours
</code></pre>
<p>To target a country, append <code>__cr.&lt;cc&gt;</code> to the username with a two-letter ISO code, for example <code>USERNAME__cr.de</code> for German exits. No other code change is needed.</p>
<h1>Cost versus success rate</h1>
<p>The real decision is a trade between price and pass rate. Datacenter might run you a fraction of residential per GB but fail on a protected target 8 times out of 10, so your effective cost per successful request is actually higher. Residential might cost several times more per GB but pass 95 percent of the time, which makes it cheaper per useful result. Run a small test batch of a few hundred requests per pool against your specific target and measure the success rate before you commit a plan to it. The winner is almost never obvious from price alone.</p>
<p>A common pattern is a fallback ladder: try datacenter first, and only escalate a request to residential or mobile when the cheaper pool gets challenged. That keeps your average cost near the floor while still clearing the hard cases. Concretely, you might route 80 percent of a crawl through datacenter, catch the 20 percent that get blocked, and retry only those on residential. If datacenter costs a fifth of residential per GB, that split can cut your effective bill by more than half compared to running everything on residential from the start.</p>
<p>Here is a quick summary of where each type lands:</p>
<ul>
<li><strong>Datacenter</strong> is cheapest, fastest, lowest trust. Use it for lenient targets and high volume.</li>
<li><strong>Residential</strong> is the default balance of trust and price. Use it when a target checks IP reputation.</li>
<li><strong>Premium Residential</strong> is the ISP-style higher-trust pool. Use it for tough targets that burn ordinary residential IPs.</li>
<li><strong>Mobile</strong> is the most expensive and the most trusted. Use it for mobile-first sites and the hardest reputation checks.</li>
</ul>
<p>If the target throws a captcha rather than a hard IP block, a better IP alone will not finish the job. Pair the proxy with the <a href="https://capzy.ai/solvers">Solver API</a> so the challenge gets answered and the request completes. And if you would rather not manage pools and challenges yourself, the <a href="https://capzy.ai/web-scraper">Web Scraper API</a> rolls egress, rendering, and anti-bot handling into a single call.</p>
<p>Proxy type is not a setting you pick once and forget. It is a per-target decision, and the right answer shifts as sites tighten their defenses. Start cheap, measure, and climb only when you have to.</p>
<p>Ready to test all four pools on one account? <a href="https://capzy.ai/auth/register">Sign up free</a> and route your first request in under a minute.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/proxy-types-explained",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vgdljo/proxy_types_explained_datacenter_residential_isp/",
"url_overridden_by_dest": "https://capzy.ai/blog/proxy-types-explained",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-05T17:15:41+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/5n1JsYlqzADWFNswBQdP35EEnlIz4N3yRbJvSCccK04.jpeg?width=140&height=73&auto=webp&s=4293d934298393ff9e6c98cc3bed5f5df74e8c62",
"preview_image": "https://external-preview.redd.it/5n1JsYlqzADWFNswBQdP35EEnlIz4N3yRbJvSCccK04.jpeg?auto=webp&s=8ad98737c547fc2856d20d6bb9e5838cf80692a3",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vgdl7a",
"fullname": "t3_1vgdl7a",
"scraped_at": "2026-08-18T09:36:17.072365+00:00",
"title": "Rotating vs Sticky Proxy Sessions: Which One to Choose",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# Rotating vs Sticky Proxy Sessions: Which One to Choose
The choice between rotating and sticky proxy sessions is one of the most common mistakes in scraping setups, and it costs people either blocks or broken flows. Rotating means a fresh IP on every request. Sticky means you hold the same IP for a window of time. Neither is better in the abstract; they solve opposite problems, and picking the wrong one will quietly wreck your success rate.
On the [Proxies API](https://capzy.ai/proxies) the pool rotates the exit IP on every request by default. To hold an IP, you set a rotation interval in the dashboard, and requests on that account keep the same address until the window elapses. That single setting is the entire difference, so it is worth understanding when each behavior is what you want.
# When rotating sessions win
Rotating is the default for good reason. When you are pulling a lot of independent pages, spreading them across many IPs is exactly what you want. Each request looks like a different visitor, so no single IP racks up a suspicious request count, and you never hit the per-IP rate limits that sites use to throttle scrapers.
Say you are collecting 50,000 product pages. If they all came from one IP, that IP would fire 50,000 requests in an hour and get itself banned in the first few minutes. Spread across a rotating residential pool, each IP might send a handful of requests before the pool moves on, and the traffic blends into normal background noise. This is the pattern for price monitoring, search-result harvesting, catalog crawls, and any job where each page stands on its own with no shared state.
Rotating also gives you natural fault tolerance. If one IP in the pool is slow or gets challenged, the next request simply lands on a different one. You are never stuck behind a single bad address.
There is a throughput benefit too. Because each request can go out on its own IP, you can run many in parallel without concentrating load. Twenty concurrent workers on a rotating pool spread across twenty different addresses instead of stacking twenty requests onto one, which is both faster and far less likely to trip a rate limit. For a large independent crawl, rotation is what lets you push volume without drawing attention.
# When sticky sessions win
Sticky sessions exist for one reason: state that lives across requests. Some flows only make sense if the site sees a consistent visitor from start to finish.
Think about a login. You POST credentials, the server sets a session cookie tied to your address, and the next request needs to come from the same IP or the session looks hijacked and gets dropped. If your proxy rotated between the login and the follow-up call, the second request arrives on a new IP carrying a cookie minted for a different one. Many sites treat that as an attack and kill the session. The same problem hits multi-step carts, checkout flows, paginated results behind a search that stored your query server-side, and anything that walks a wizard across several pages.
>If a flow has a "next step" that depends on what you did in the last step, you almost certainly need a sticky session. If every request is a standalone fetch, rotate.
The trade is that a sticky IP takes on the full request volume of that session. Keep it reasonable. A sticky session is for completing one coherent flow, not for pushing 10,000 requests through a single held IP. That is the fast lane to a block.
Sticky sessions also make debugging saner. When every request in a flow comes from the same address, the site's own logs, cookies, and rate counters all line up with a single visitor, so behavior is predictable and repeatable. With rotation mid-flow, a failure could be the site, the new IP, or the cookie mismatch, and you cannot tell which. Holding the IP removes a whole class of "why did step three fail this time but not last time" confusion.
# How to configure it
Rotating needs no setup; it is on by default. Here is a plain rotating request in Python:
import requests
proxies = {
"http": "http://USERNAME:PASSWORD@proxy.capzy.ai:823",
"https": "http://USERNAME:PASSWORD@proxy.capzy.ai:823",
}
# Each call exits from a different pool IP
for url in urls:
r = requests.get(url, proxies=proxies, timeout=30)
process(r)
For sticky behavior, open the Targeting tab in your [dashboard](https://capzy.ai/dashboard/proxies) and set a rotation interval, configured in minutes. While that window is active, requests on the account keep the same IP, then rotate to a fresh one when it elapses. Size the window to your flow: a login-plus-scrape sequence might need 2 or 3 minutes, while a long multi-page checkout might want 10. Do not set it longer than the flow actually needs, because a held IP that lingers just accumulates request count for no benefit.
A practical pattern for mixed workloads is to run two credential sets. Use a rotating account for the bulk crawl, and a sticky account for the handful of stateful flows like login or checkout. That way each job gets the session behavior it needs without compromising the other.
Sizing the window well takes a little observation. Watch how long your flow actually runs end to end, then set the interval a bit above that so the whole sequence completes on one IP with a small margin. If your login-plus-scrape sequence finishes in 90 seconds, a 3-minute window is comfortable; a 30-minute window just holds the address long after you are done with it, piling on request count and reputation risk for no gain. When a target is aggressive, err toward shorter windows and fewer requests per held IP, because a sticky IP that lingers under load is exactly the pattern anti-bot systems are tuned to catch. The goal is one clean visitor per flow, then rotate away.
# Getting it right the first time
Most block problems that people blame on the pool are actually session mismatches. A rotating session on a login flow reads as suspicious and fails. A sticky session on a bulk crawl piles too many requests onto one IP and gets banned. Match the mode to the job and a lot of "the proxies are bad" complaints simply disappear.
If your target answers with a captcha in the middle of a sticky flow, keep the IP steady and hand the challenge to the [Solver API](https://capzy.ai/solvers) so the session survives. And when you want the session logic handled for you, the [Web Scraper API](https://capzy.ai/web-scraper) accepts a session id and keeps the same IP and cookies together across calls.
Pick rotating for independent fetches, sticky for stateful flows, and size the hold window to the work. [Create an account](https://capzy.ai/auth/register) and try both modes on the same endpoint.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>Rotating vs Sticky Proxy Sessions: Which One to Choose</h1>
<p>The choice between rotating and sticky proxy sessions is one of the most common mistakes in scraping setups, and it costs people either blocks or broken flows. Rotating means a fresh IP on every request. Sticky means you hold the same IP for a window of time. Neither is better in the abstract; they solve opposite problems, and picking the wrong one will quietly wreck your success rate.</p>
<p>On the <a href="https://capzy.ai/proxies">Proxies API</a> the pool rotates the exit IP on every request by default. To hold an IP, you set a rotation interval in the dashboard, and requests on that account keep the same address until the window elapses. That single setting is the entire difference, so it is worth understanding when each behavior is what you want.</p>
<h1>When rotating sessions win</h1>
<p>Rotating is the default for good reason. When you are pulling a lot of independent pages, spreading them across many IPs is exactly what you want. Each request looks like a different visitor, so no single IP racks up a suspicious request count, and you never hit the per-IP rate limits that sites use to throttle scrapers.</p>
<p>Say you are collecting 50,000 product pages. If they all came from one IP, that IP would fire 50,000 requests in an hour and get itself banned in the first few minutes. Spread across a rotating residential pool, each IP might send a handful of requests before the pool moves on, and the traffic blends into normal background noise. This is the pattern for price monitoring, search-result harvesting, catalog crawls, and any job where each page stands on its own with no shared state.</p>
<p>Rotating also gives you natural fault tolerance. If one IP in the pool is slow or gets challenged, the next request simply lands on a different one. You are never stuck behind a single bad address.</p>
<p>There is a throughput benefit too. Because each request can go out on its own IP, you can run many in parallel without concentrating load. Twenty concurrent workers on a rotating pool spread across twenty different addresses instead of stacking twenty requests onto one, which is both faster and far less likely to trip a rate limit. For a large independent crawl, rotation is what lets you push volume without drawing attention.</p>
<h1>When sticky sessions win</h1>
<p>Sticky sessions exist for one reason: state that lives across requests. Some flows only make sense if the site sees a consistent visitor from start to finish.</p>
<p>Think about a login. You POST credentials, the server sets a session cookie tied to your address, and the next request needs to come from the same IP or the session looks hijacked and gets dropped. If your proxy rotated between the login and the follow-up call, the second request arrives on a new IP carrying a cookie minted for a different one. Many sites treat that as an attack and kill the session. The same problem hits multi-step carts, checkout flows, paginated results behind a search that stored your query server-side, and anything that walks a wizard across several pages.</p>
<blockquote>
<p>If a flow has a &quot;next step&quot; that depends on what you did in the last step, you almost certainly need a sticky session. If every request is a standalone fetch, rotate.</p>
</blockquote>
<p>The trade is that a sticky IP takes on the full request volume of that session. Keep it reasonable. A sticky session is for completing one coherent flow, not for pushing 10,000 requests through a single held IP. That is the fast lane to a block.</p>
<p>Sticky sessions also make debugging saner. When every request in a flow comes from the same address, the site&#39;s own logs, cookies, and rate counters all line up with a single visitor, so behavior is predictable and repeatable. With rotation mid-flow, a failure could be the site, the new IP, or the cookie mismatch, and you cannot tell which. Holding the IP removes a whole class of &quot;why did step three fail this time but not last time&quot; confusion.</p>
<h1>How to configure it</h1>
<p>Rotating needs no setup; it is on by default. Here is a plain rotating request in Python:</p>
<pre><code>import requests
proxies = {
&quot;http&quot;: &quot;http://USERNAME:PASSWORD@proxy.capzy.ai:823&quot;,
&quot;https&quot;: &quot;http://USERNAME:PASSWORD@proxy.capzy.ai:823&quot;,
}
# Each call exits from a different pool IP
for url in urls:
r = requests.get(url, proxies=proxies, timeout=30)
process(r)
</code></pre>
<p>For sticky behavior, open the Targeting tab in your <a href="https://capzy.ai/dashboard/proxies">dashboard</a> and set a rotation interval, configured in minutes. While that window is active, requests on the account keep the same IP, then rotate to a fresh one when it elapses. Size the window to your flow: a login-plus-scrape sequence might need 2 or 3 minutes, while a long multi-page checkout might want 10. Do not set it longer than the flow actually needs, because a held IP that lingers just accumulates request count for no benefit.</p>
<p>A practical pattern for mixed workloads is to run two credential sets. Use a rotating account for the bulk crawl, and a sticky account for the handful of stateful flows like login or checkout. That way each job gets the session behavior it needs without compromising the other.</p>
<p>Sizing the window well takes a little observation. Watch how long your flow actually runs end to end, then set the interval a bit above that so the whole sequence completes on one IP with a small margin. If your login-plus-scrape sequence finishes in 90 seconds, a 3-minute window is comfortable; a 30-minute window just holds the address long after you are done with it, piling on request count and reputation risk for no gain. When a target is aggressive, err toward shorter windows and fewer requests per held IP, because a sticky IP that lingers under load is exactly the pattern anti-bot systems are tuned to catch. The goal is one clean visitor per flow, then rotate away.</p>
<h1>Getting it right the first time</h1>
<p>Most block problems that people blame on the pool are actually session mismatches. A rotating session on a login flow reads as suspicious and fails. A sticky session on a bulk crawl piles too many requests onto one IP and gets banned. Match the mode to the job and a lot of &quot;the proxies are bad&quot; complaints simply disappear.</p>
<p>If your target answers with a captcha in the middle of a sticky flow, keep the IP steady and hand the challenge to the <a href="https://capzy.ai/solvers">Solver API</a> so the session survives. And when you want the session logic handled for you, the <a href="https://capzy.ai/web-scraper">Web Scraper API</a> accepts a session id and keeps the same IP and cookies together across calls.</p>
<p>Pick rotating for independent fetches, sticky for stateful flows, and size the hold window to the work. <a href="https://capzy.ai/auth/register">Create an account</a> and try both modes on the same endpoint.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/rotating-vs-sticky-proxy-sessions",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vgdl7a/rotating_vs_sticky_proxy_sessions_which_one_to/",
"url_overridden_by_dest": "https://capzy.ai/blog/rotating-vs-sticky-proxy-sessions",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-05T17:15:19+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/ABT_bMEa2CmIu3B9tpK-VmqY8XCA1HSUi07RYZrUkr0.jpeg?width=140&height=73&auto=webp&s=7eab2778d40df8dd2c786d7890c70d0ffa6dc429",
"preview_image": "https://external-preview.redd.it/ABT_bMEa2CmIu3B9tpK-VmqY8XCA1HSUi07RYZrUkr0.jpeg?auto=webp&s=2e12fdf175ae616b2929495da6d532d8a1bb8214",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
},
{
"id": "1vgdjmc",
"fullname": "t3_1vgdjmc",
"scraped_at": "2026-08-18T09:36:17.072373+00:00",
"title": "Geo-Targeting: Why Location-Accurate Proxies Matter for Data Quality",
"author": "Capzy-ai",
"subreddit": "Capzy",
"selftext": "# Geo-Targeting: Why Location-Accurate Proxies Matter for Data Quality
Geo-targeting with proxies is not a nice-to-have feature for scraping; it is a data-quality requirement. The web you see depends on where the site thinks you are, and a request that exits from the wrong country brings back the wrong data. If you are scraping prices, availability, search rankings, or localized content from a mismatched IP, your dataset is not just incomplete, it is wrong in ways that are hard to notice until they cost you a decision.
The [Proxies API](https://capzy.ai/proxies) builds geo-targeting into the username. Append `__cr.<country>` with a two-letter ISO code and every request exits from that country, no other code change needed. Finer targeting down to state, city, ZIP, and ASN lives in the dashboard. That simplicity hides how much the exit location actually changes what comes back.
# The web is location-specific
Consider a few concrete cases where location changes the data itself, not just the language of the page.
**Pricing.** A product priced at $49.99 for a US visitor might show as €54.99 in Germany or £44.99 in the UK, and not because of a straight currency conversion. Retailers set regional prices deliberately, run country-specific promotions, and display tax-inclusive or tax-exclusive totals depending on the market. Scrape a European store from a US IP and you may get US-facing pricing, a "not available in your region" wall, or a redirect to a different storefront entirely. Your price dataset is now a mix of regions you cannot tell apart.
**Availability.** Stock, shipping options, and even whether a product listing appears at all can depend on the visitor's country. Streaming catalogs are the obvious example, but the same is true for retail SKUs, travel inventory, and any marketplace that gates listings by region. A crawl from one country simply cannot see what a crawl from another can.
**Search and rankings.** Search results are localized hard. The same query returns different results, in a different order, with different local businesses and ads, depending on where the request comes from. If you are tracking rankings or SERP features for a specific market, an IP in the wrong country produces numbers that describe a market you do not care about.
**Content and compliance.** Cookie banners, consent flows, regional legal notices, and localized copy all shift by geography. Sometimes the page structure itself changes, which quietly breaks selectors that worked fine from another exit.
>If your data has a price, a stock level, a ranking, or a "available in your area" element, the exit country is part of the record. Collect it from the wrong place and the value is not comparable to the rest of your set.
# Targeting the right location
The mechanics are a one-token change. Here is the same request pointed at three different countries:
# US exit IPs
curl -x http://USERNAME__cr.us:PASSWORD@proxy.capzy.ai:823 https://example.com
# German exit IPs
curl -x http://USERNAME__cr.de:PASSWORD@proxy.capzy.ai:823 https://example.com
# UK exit IPs
curl -x http://USERNAME__cr.gb:PASSWORD@proxy.capzy.ai:823 https://example.com
In Python, geo-targeting is just which credentials go in the proxies dict:
import requests
def fetch(url, country):
user = f"USERNAME__cr.{country}"
p = f"http://{user}:PASSWORD@proxy.capzy.ai:823"
proxies = {"http": p, "https": p}
return requests.get(url, proxies=proxies, timeout=30)
# Collect the same page from three markets, cleanly separated
for cc in ("us", "de", "gb"):
r = fetch("https://example.com/product/1", cc)
save(cc, r.text)
Tagging every record with the country it was collected from is the habit that keeps a multi-region dataset honest. When you later compare a US price to a German one, you know they came from the right exits, not from whatever IP the pool happened to hand you.
# Accuracy, not just the flag
There is a subtler point than picking the right country: the IP has to genuinely be where it claims. A proxy that says "US" but is actually a datacenter IP registered abroad can still get served the wrong regional content, because the site geolocates the address itself and disagrees with the label. Location-accurate residential and mobile IPs matter here because they resolve to real consumer connections in the target country, so the site's own geolocation lands where you expect. For city-level or ZIP-level precision, the dashboard targeting is where you narrow it, and the more specific you go the more the pool's real coverage in that area matters.
This is also why the pool choice and the geo choice interact. A datacenter IP tagged for a country will pass a simple check but may be treated as out-of-region by a site doing careful geolocation. When location accuracy is load-bearing for your data, the residential and premium residential pools give you addresses that actually belong to the place.
# Verifying you actually landed there
Never assume the exit is where you asked. Before a big geo-targeted run, spend one request confirming it. A quick call through the targeted credentials to an IP-geolocation endpoint tells you the country the world sees for that exit, which is what matters, not the label you attached.
import requests
user = "USERNAME__cr.de"
p = f"http://{user}:PASSWORD@proxy.capzy.ai:823"
proxies = {"http": p, "https": p}
info = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=30).json()
print(info["country"], info.get("city")) # expect DE
If that returns something other than the country you targeted, stop and fix the setup before you collect a dataset built on the wrong exits. A five-second check saves you from a crawl you have to throw away. Run the same check periodically during a long job, too, since it confirms the pool is still serving you addresses in the region you paid to target.
For sub-country precision, the dashboard lets you narrow to state, city, ZIP, and ASN, but accuracy at that level depends on real pool coverage in the area. Targeting a specific city only produces clean data if the pool actually has addresses there; if it does not, you may fall back to country-level exits without noticing. Verify the same way, by checking the reported city on a sample, before trusting city-level records.
One more habit keeps a multi-market dataset trustworthy: hold the geo constant across a comparison. If you are pricing the same product in five countries, collect all five within the same run and the same conditions, rather than the US today and Germany next week. Prices and availability drift over time, so a stale exit mixed with a fresh one introduces differences that look like geography but are really timing. Tag each record with both the country and the timestamp, and your later analysis can tell the two apart.
If a geo-targeted request runs into a captcha, keep the exit country fixed and route the challenge to the [Solver API](https://capzy.ai/solvers) so you do not lose the regional context. And if you want the geo, rendering, and anti-bot handled together, the [Web Scraper API](https://capzy.ai/web-scraper) takes a `proxyCountry` and returns location-correct content in one call.
Location is part of the data, not a setting you can afford to get wrong. Target the country that matters, verify the exit really lands there, and tag every record with where it came from. [Sign up free](https://capzy.ai/auth/register) and pull the same page from three markets to see the difference for yourself.",
"selftext_html": "<!-- SC_OFF --><div class="md"><h1>Geo-Targeting: Why Location-Accurate Proxies Matter for Data Quality</h1>
<p>Geo-targeting with proxies is not a nice-to-have feature for scraping; it is a data-quality requirement. The web you see depends on where the site thinks you are, and a request that exits from the wrong country brings back the wrong data. If you are scraping prices, availability, search rankings, or localized content from a mismatched IP, your dataset is not just incomplete, it is wrong in ways that are hard to notice until they cost you a decision.</p>
<p>The <a href="https://capzy.ai/proxies">Proxies API</a> builds geo-targeting into the username. Append <code>__cr.&lt;country&gt;</code> with a two-letter ISO code and every request exits from that country, no other code change needed. Finer targeting down to state, city, ZIP, and ASN lives in the dashboard. That simplicity hides how much the exit location actually changes what comes back.</p>
<h1>The web is location-specific</h1>
<p>Consider a few concrete cases where location changes the data itself, not just the language of the page.</p>
<p><strong>Pricing.</strong> A product priced at $49.99 for a US visitor might show as €54.99 in Germany or £44.99 in the UK, and not because of a straight currency conversion. Retailers set regional prices deliberately, run country-specific promotions, and display tax-inclusive or tax-exclusive totals depending on the market. Scrape a European store from a US IP and you may get US-facing pricing, a &quot;not available in your region&quot; wall, or a redirect to a different storefront entirely. Your price dataset is now a mix of regions you cannot tell apart.</p>
<p><strong>Availability.</strong> Stock, shipping options, and even whether a product listing appears at all can depend on the visitor&#39;s country. Streaming catalogs are the obvious example, but the same is true for retail SKUs, travel inventory, and any marketplace that gates listings by region. A crawl from one country simply cannot see what a crawl from another can.</p>
<p><strong>Search and rankings.</strong> Search results are localized hard. The same query returns different results, in a different order, with different local businesses and ads, depending on where the request comes from. If you are tracking rankings or SERP features for a specific market, an IP in the wrong country produces numbers that describe a market you do not care about.</p>
<p><strong>Content and compliance.</strong> Cookie banners, consent flows, regional legal notices, and localized copy all shift by geography. Sometimes the page structure itself changes, which quietly breaks selectors that worked fine from another exit.</p>
<blockquote>
<p>If your data has a price, a stock level, a ranking, or a &quot;available in your area&quot; element, the exit country is part of the record. Collect it from the wrong place and the value is not comparable to the rest of your set.</p>
</blockquote>
<h1>Targeting the right location</h1>
<p>The mechanics are a one-token change. Here is the same request pointed at three different countries:</p>
<pre><code># US exit IPs
curl -x http://USERNAME__cr.us:PASSWORD@proxy.capzy.ai:823 https://example.com
# German exit IPs
curl -x http://USERNAME__cr.de:PASSWORD@proxy.capzy.ai:823 https://example.com
# UK exit IPs
curl -x http://USERNAME__cr.gb:PASSWORD@proxy.capzy.ai:823 https://example.com
</code></pre>
<p>In Python, geo-targeting is just which credentials go in the proxies dict:</p>
<pre><code>import requests
def fetch(url, country):
user = f&quot;USERNAME__cr.{country}&quot;
p = f&quot;http://{user}:PASSWORD@proxy.capzy.ai:823&quot;
proxies = {&quot;http&quot;: p, &quot;https&quot;: p}
return requests.get(url, proxies=proxies, timeout=30)
# Collect the same page from three markets, cleanly separated
for cc in (&quot;us&quot;, &quot;de&quot;, &quot;gb&quot;):
r = fetch(&quot;https://example.com/product/1&quot;, cc)
save(cc, r.text)
</code></pre>
<p>Tagging every record with the country it was collected from is the habit that keeps a multi-region dataset honest. When you later compare a US price to a German one, you know they came from the right exits, not from whatever IP the pool happened to hand you.</p>
<h1>Accuracy, not just the flag</h1>
<p>There is a subtler point than picking the right country: the IP has to genuinely be where it claims. A proxy that says &quot;US&quot; but is actually a datacenter IP registered abroad can still get served the wrong regional content, because the site geolocates the address itself and disagrees with the label. Location-accurate residential and mobile IPs matter here because they resolve to real consumer connections in the target country, so the site&#39;s own geolocation lands where you expect. For city-level or ZIP-level precision, the dashboard targeting is where you narrow it, and the more specific you go the more the pool&#39;s real coverage in that area matters.</p>
<p>This is also why the pool choice and the geo choice interact. A datacenter IP tagged for a country will pass a simple check but may be treated as out-of-region by a site doing careful geolocation. When location accuracy is load-bearing for your data, the residential and premium residential pools give you addresses that actually belong to the place.</p>
<h1>Verifying you actually landed there</h1>
<p>Never assume the exit is where you asked. Before a big geo-targeted run, spend one request confirming it. A quick call through the targeted credentials to an IP-geolocation endpoint tells you the country the world sees for that exit, which is what matters, not the label you attached.</p>
<pre><code>import requests
user = &quot;USERNAME__cr.de&quot;
p = f&quot;http://{user}:PASSWORD@proxy.capzy.ai:823&quot;
proxies = {&quot;http&quot;: p, &quot;https&quot;: p}
info = requests.get(&quot;https://ipinfo.io/json&quot;, proxies=proxies, timeout=30).json()
print(info[&quot;country&quot;], info.get(&quot;city&quot;)) # expect DE
</code></pre>
<p>If that returns something other than the country you targeted, stop and fix the setup before you collect a dataset built on the wrong exits. A five-second check saves you from a crawl you have to throw away. Run the same check periodically during a long job, too, since it confirms the pool is still serving you addresses in the region you paid to target.</p>
<p>For sub-country precision, the dashboard lets you narrow to state, city, ZIP, and ASN, but accuracy at that level depends on real pool coverage in the area. Targeting a specific city only produces clean data if the pool actually has addresses there; if it does not, you may fall back to country-level exits without noticing. Verify the same way, by checking the reported city on a sample, before trusting city-level records.</p>
<p>One more habit keeps a multi-market dataset trustworthy: hold the geo constant across a comparison. If you are pricing the same product in five countries, collect all five within the same run and the same conditions, rather than the US today and Germany next week. Prices and availability drift over time, so a stale exit mixed with a fresh one introduces differences that look like geography but are really timing. Tag each record with both the country and the timestamp, and your later analysis can tell the two apart.</p>
<p>If a geo-targeted request runs into a captcha, keep the exit country fixed and route the challenge to the <a href="https://capzy.ai/solvers">Solver API</a> so you do not lose the regional context. And if you want the geo, rendering, and anti-bot handled together, the <a href="https://capzy.ai/web-scraper">Web Scraper API</a> takes a <code>proxyCountry</code> and returns location-correct content in one call.</p>
<p>Location is part of the data, not a setting you can afford to get wrong. Target the country that matters, verify the exit really lands there, and tag every record with where it came from. <a href="https://capzy.ai/auth/register">Sign up free</a> and pull the same page from three markets to see the difference for yourself.</p>
</div><!-- SC_ON -->",
"url": "https://capzy.ai/blog/geo-targeting-proxies-data-quality",
"permalink": "https://www.reddit.com/r/Capzy/comments/1vgdjmc/geotargeting_why_locationaccurate_proxies_matter/",
"url_overridden_by_dest": "https://capzy.ai/blog/geo-targeting-proxies-data-quality",
"score": 1,
"upvote_ratio": 1,
"ups": 1,
"downs": 0,
"num_comments": 0,
"created_at": "2026-08-05T17:13:45+00:00",
"over_18": false,
"is_video": false,
"is_self": false,
"post_hint": "link",
"spoiler": false,
"locked": false,
"archived": false,
"pinned": false,
"is_original_content": false,
"distinguished": null,
"gilded": 0,
"edited": null,
"domain": "capzy.ai",
"flair": null,
"author_flair": null,
"author_premium": false,
"subreddit_subscribers": 5,
"subreddit_id": "t5_jbfcch",
"thumbnail": "https://external-preview.redd.it/jTVKGp5UnIH7k9cfWJc_jAICgYR6J14obdu5ifKlGwQ.jpeg?width=140&height=73&auto=webp&s=2b12ce97a92010630f2e57972f6a435d681141d5",
"preview_image": "https://external-preview.redd.it/jTVKGp5UnIH7k9cfWJc_jAICgYR6J14obdu5ifKlGwQ.jpeg?auto=webp&s=43634a53c702c6181856d1ed9090be747bbc3192",
"preview_width": 1200,
"preview_height": 630,
"video": null,
"gallery": null,
"crosspost_parent": null,
"awards": 0,
"stickied": false,
"num_crossposts": 0
}
]
}
GET
/v1/data/reddit/search/posts1 credit / run VerifiedSearch Posts
Search posts globally or within a subreddit.
Handled for you:CloudflareRate Limit
Parameters
| Name | Type | Req | Description |
|---|---|---|---|
| q | string | yes | Search query. |
| Advanced filters | |||
| subreddit | string | no | Restrict to a subreddit. |
| sort | string | no | relevance | hot | top | new | comments (default: relevance) |
| t | string | no | hour | day | week | month | year | all (default: all) |
| limit | integer | no | 1-100. (default: 25) |
| after | string | no | Pagination cursor. |