Testing ScrAPI at 256 Concurrent Browser Requests

ScrAPI's browser pipeline was recently load-tested from 8 through 256 concurrent requests. Request completion stayed at 100% through 128 concurrent requests and 99.8% at 256, where two requests failed with an HTTP 429 ("Too Many Requests") response. In the same session, the lightweight HTTP pipeline sustained 102.6 requests per second at 100% success.

This is a different exercise from our ecommerce scraping benchmark, which compares ScrAPI against other scraping providers on public targets. This article does not compare ScrAPI to anything else. It measures how ScrAPI itself behaves as load increases: throughput, latency, success rate, and proxy-path behaviour.

Summary

Test	Result
HTTP pipeline, no proxy	102.6 requests/sec, 100% success
Browser pipeline ramp	Tested from 8 to 256 concurrent requests
Browser success at 256 concurrent	99.8% (2 requests failed with HTTP 429)
Browser pipeline, datacenter proxy	100% success at concurrency 16
Browser pipeline, residential proxy	100% success at concurrency 8
HTTP pipeline, residential proxy	100% success at concurrency 16

The tests ran for different durations and at different concurrency levels, so these rows are not directly comparable to each other. Each is explained in its own section below.

Why we ran this test

We wanted a factual answer to a few questions we get asked directly: how many concurrent browser requests can a caller realistically run, what happens to latency as concurrency rises, how much does an HTTP-only request cost compared to a full browser render, and does routing through a proxy change the picture.

Rather than describe the platform in general terms, we ran a fixed test matrix against real targets and are publishing the raw numbers, including the parts that are not flattering (rising latency, two failed requests, and resource filtering trade-offs).

Test methodology

Load was spread across five domains we own and control, assigned to targets in rotation, rather than concentrated on a single site's own rate limiting. Six test cells covered two request pipelines (plain HTTP fetch vs. full browser rendering) crossed with three proxy modes (none, datacenter, residential).

A request counted as successful only if it returned an HTTP status in the 200–299 range and non-empty content. A response with a status code but no content did not count as a success. We calculated latency percentiles (p50/p95/p99) over successful requests only, so a fast rejection could not pull the percentiles down and make a failing level look faster than it was.

The harness retried a request up to twice when the API returned a 503 (no capacity available at that instant), with the retries and any resulting delay counted in that request's latency. We tracked 503s separately from the success rate rather than folding them into it, so a level that only held up because of a retry can be told apart from one that never needed one. The harness discarded, rather than scored, any request still in flight when a level's timer expired. A request cut short by the test harness says nothing about whether it would have succeeded.

The browser ramp (Cell B, below) increased concurrency by doubling it every 120 seconds, starting at 8 and stopping at a pre-configured cap of 256, or earlier if the success rate dropped notably. It did not drop, so the test stopped at its configured cap. 256 was a limit we chose for this test run, not a limit the platform hit. Even at 256 concurrent requests, this run was using only a modest fraction of ScrAPI's available execution capacity, so 256 should not be read as a ceiling on the service.

Browser concurrency results

Cell B ran ScrAPI's browser pipeline with default resource filtering enabled (images, media and fonts are excluded by default when rendering a page), starting at 8 concurrent requests and doubling every 120 seconds up to 256.

Concurrency	Requests completed	Requests/sec	p95 latency	Success rate
8	293	2.4	6,461 ms	100.0%
16	623	5.2	5,512 ms	100.0%
32	1,000	8.3	7,489 ms	100.0%
64	1,135	9.5	16,161 ms	100.0%
128	1,256	10.5	28,576 ms	100.0%
256	1,284	10.7	37,859 ms	99.8%

Two patterns stand out. Throughput rises quickly between 8 and 32 concurrent requests, then its growth slows sharply. Going from 32 to 256 concurrency, an 8x increase, only raised throughput from 8.3 to 10.7 requests/sec, about 29%. At the same time, p95 latency kept climbing at every level, ending roughly 6x higher at 256 than at 8.

By the higher concurrency levels, sending more concurrent requests mostly meant each one waited longer, rather than the platform completing proportionally more of them per second. Request completion, however, remained effectively unchanged: 100% through 128 concurrent requests, 99.8% at 256.

The only two failures in the entire ramp happened at concurrency 256, and both were HTTP 429 ("Too Many Requests") responses. Occasional non-2xx responses like this should be expected in real-world scraping traffic at higher concurrency, regardless of which platform sends the request.

No 503 (capacity unavailable) responses were recorded anywhere in this run, at any concurrency level.

HTTP pipeline performance

Cell A tested ScrAPI's HTTP pipeline (useBrowser not set), which fetches the document directly instead of rendering it in a browser. This cell ran a fixed concurrency of 64 for 5 minutes, rather than a ramp.

• 30,772 completed requests
• 102.6 requests per second
• 1,515 ms p95 latency
• 100.0% success rate

We discarded the 63 requests still in flight when the 5-minute window closed, rather than counting them either way.

The gap between this and the browser pipeline's numbers is the expected cost of full browser rendering: parsing, executing JavaScript and building a page versus fetching a document. If a target does not require JavaScript execution to expose the data you need, the HTTP pipeline is faster and lighter than full browser rendering.

Proxy path tests

Three cells tested routing through ScrAPI's proxy modes. Each ran at a fixed, modest concurrency, so the goal here is confirming that the path works and observing its cost, not measuring a ceiling.

Test	Concurrency	Duration	Requests	Requests/sec	p95 latency	Success
Browser, datacenter proxy	16	5 min	881	2.9	8,661 ms	100.0%
Browser, residential proxy	8	3 min	145	0.8	13,555 ms	100.0%
HTTP, residential proxy	16	2 min	852	7.1	4,740 ms	100.0%

All three tested paths completed every request successfully. We are not claiming these proxy paths are always 100% reliable against arbitrary targets. This shows one successful run against our own test domains, not a general reliability guarantee. Routing an HTTP-only request through a residential proxy is also a supported combination, useful when a target needs a residential IP but not JavaScript rendering.

The effect of resource filtering

ScrAPI excludes images, media and fonts from a rendered page by default, because most extraction workloads need the page's text and structure, not every asset a browser would normally load. Cells D and E compare that default setting against a configuration that fetches full page resources, on the same residential-proxy browser workload.

Configuration	Requests/sec	p95 latency
Default filtering ON	0.8	13,555 ms
Filtering OFF (full page weight)	0.7	16,055 ms

In this test, default resource filtering reduced p95 latency and gave a small increase in throughput.

Capacity under load

Across the entire test run, including the full browser ramp from 8 to 256 concurrent requests, we recorded zero HTTP 503 (capacity unavailable) responses at any concurrency level tested.

What this benchmark does not prove

• 256 was the concurrency limit we chose for this test, not a discovered ceiling.
• These are the results of one test run, on one day, against five domains we control. A more heavily defended target will behave differently.

How this differs from our provider comparison

Our ecommerce scraping benchmark answers a different question: how does ScrAPI compare to other scraping providers on success rate, speed and cost per successful page. This article does not touch that comparison or its dataset. It is entirely about how ScrAPI behaves under its own increasing load.

A note on reliability testing

Throughput numbers are only useful if the page returned is actually usable. A fast response that comes back as a block page or CAPTCHA challenge is not a success in any meaningful sense. Alongside the performance testing described here, we maintain a separate internal test suite that checks a range of bot-detection and challenge scenarios. We are not publishing the techniques it tests for, since that would help defeat them.

Try it yourself

• Try a request in the Playground without writing any code
• Create a free ScrAPI account and get free credits to test with
• Read the API documentation for the full set of request options
• See how ScrAPI compares to other providers