Private Proxies for Scraping: What the Term Still Means

1.Shared, semi-dedicated, private: what you are buying

A shared proxy puts an unknown number of customers behind one IP. You inherit whatever every other user did to it, including bans you had no part in. A semi-dedicated proxy caps that number, commonly at three or five. A private or dedicated proxy assigns the IP to you alone for the rental period.

Those tiers describe one variable: how many people share the address. They say nothing about where the address lives, which is the variable that decides your block rate. A private IP inside a well-known hosting provider's range carries that range's reputation regardless of exclusivity, and reputation is assigned to the block, not the address.

This is why buying up the tier ladder so often produces no improvement. Teams move from shared to private, keep getting 403s, and conclude proxies do not work. The tier was never the problem — the network the IPs sit in was.

• Shared — unknown co-tenants, inherited bans, cheapest per IP
• Semi-dedicated — a capped number of co-tenants, usually three to five
• Private / dedicated — yours alone for the rental term
• None of the three describes the ASN, which is what gets scored

2.Why the ScrapeBox vocabulary persists

The terminology comes from a period when the dominant proxy buyer was running SEO tooling — ScrapeBox and its contemporaries — against search engines and comment forms. In that context the threat model genuinely was per-IP rate limiting, and buying exclusivity genuinely was the fix.

Proxy vendors kept the vocabulary because it converts. It maps onto a clean price ladder and it is what buyers still search for. The result is a market where the tier names describe a 2014 threat model and the actual product differentiation lives in fields further down the page: network type, rotation policy, and whether sessions can be pinned.

When you evaluate a vendor, read past the tier name to those fields. A vendor that advertises private proxies but will not tell you the network type is selling you exclusivity in a datacenter range, and on protected targets that combination underperforms a shared residential pool at a fraction of the effort.

3.ASN reputation replaced exclusivity

Modern anti-bot scoring starts before your HTTP request is parsed. The connection's origin network is classified — hosting, residential, mobile, corporate — and that classification feeds the score alongside the TLS fingerprint. A request from a hosting ASN starts from a worse position than one from a consumer ISP, whatever the IP's rental status.

Residential and mobile addresses score better because they belong to networks that overwhelmingly carry real users. Blocking them outright costs the site real customers, so operators are structurally reluctant to do it. Hosting ranges carry almost no legitimate consumer traffic, so blocking them is nearly free — which is why strict configurations do exactly that.

The practical reframing: stop buying exclusivity and start buying network type. Then, within residential, the thing worth paying attention to is rotation policy, because that is what determines whether a multi-request job holds together.

4.When private datacenter is still the right buy

Residential egress is not automatically correct. It costs more per gigabyte and adds latency, and plenty of workloads do not need it. If your targets are unprotected — public APIs, open data portals, documentation sites, your own properties — a private datacenter IP is faster and substantially cheaper, and nothing about the target is scoring your ASN.

The failure mode worth avoiding is routing an entire mixed workload through residential because part of it is protected. On a list where most URLs are ordinary pages and a minority sit behind a WAF, paying residential rates on everything is a large and invisible overspend.

Split the list. Send unprotected targets through cheap egress and reserve residential for the URLs that actually need it. If you would rather not maintain that split yourself, an unlock API that routes per request handles the same decision without you classifying URLs by hand.

5.Rotation versus sticky sessions

Rotating on every request is the default most people reach for, and it is wrong for anything stateful. Anti-bot cookies — Cloudflare's cf_clearance among them — are validated against the IP that earned them. Rotate mid-session and the next request re-challenges, so you pay for a fresh solve on every page.

Sticky is correct for a logical unit of work: a paginated category walk, a product page plus its dependent XHR calls, an authenticated flow. Hold one IP for the whole unit, then release it. Rotation is correct between units, where continuity carries no value and IP diversity does.

Getting this backwards is one of the most expensive mistakes in scraping, because it is invisible. The job completes, the success rate looks acceptable, and every page quietly costs a browser solve that a sticky session would have avoided entirely.

6.Per-port rental versus per-gigabyte

Private proxies are traditionally sold per port per month: a fixed rental for a fixed number of IPs. The cost is predictable and the utilisation is your problem. Idle ports bill the same as saturated ones, so the model rewards continuous high-volume use and punishes bursty workloads.

Residential is normally sold per gigabyte. That model tracks actual use, which suits bursty and seasonal work, but it makes bandwidth discipline a cost lever rather than a nicety. Blocking images, media, and fonts on a browser-rendered page can cut transfer by a large multiple, and on per-GB billing that reduction lands directly on the invoice.

OmniScrape residential egress is $2.00 per GB with no port rental and no monthly minimum. For teams already running their own crawler that only need clean egress, the proxy product is available directly over HTTP, HTTPS, or SOCKS5. Direct proxy credentials require Pay-As-You-Go or an active plan — trial accounts use Web Unlocker or BaaS instead, and get a PROXY_LOCKED error on the proxy endpoints.

12345678910curl -X POST https://api.omniscrape.io/v1/scrape \
  -H "Content-Type: application/json" \
  -H "X-API-Key: ${OMNISCRAPE_KEY}" \
  -d '{
    "url": "https://protected-retailer.com/category/audio?page=3",
    "mode": "auto",
    "proxy": "residential:gb:sticky",
    "session_id": "audio-category-walk",
    "output_format": "html"
  }'

7.A short decision procedure

Test the target before buying anything. Fetch it from an ordinary server-side client and read what comes back. A clean 200 with the markup you expect means the target is not scoring you, and cheap egress is sufficient. A challenge page, a 403, or a suspiciously small body means the network type matters and you should test residential next.

If residential clears it, the remaining question is only rotation policy, and that follows from the shape of the job rather than from the target. Stateful units get sticky, independent units get rotation.

If residential does not clear it, the blocker is not egress at all — it is JavaScript execution or a fingerprint check, and no proxy tier resolves either. At that point you need a browser in the path, which is a different purchase from a proxy.

• Plain client returns real markup — use cheap egress, no residential needed
• Plain client blocked, residential clears it — buy network type, then pick rotation policy
• Residential still blocked — the blocker is JS or fingerprinting; a proxy tier cannot fix it