1.What you can and cannot collect
Draw a hard line before you write code. Public, non-personal data on business Pages and the Ad Library is the defensible surface. Private profiles, friends lists, private groups, and anything behind authentication are off-limits. scraping them violates Meta's terms and, depending on jurisdiction, data protection law.
The most valuable and lowest-risk Facebook source is the Ad Library, which Meta publishes deliberately for transparency. It exposes active and past ads by advertiser, including creative, run dates, and (for political/issue ads) spend ranges. It is public by design, which makes it the cleanest target for competitor and market research.
- Public Page metadata: name, category, follower/like counts, about, verification badge
- Public post content on Pages: text, timestamp, reaction and comment counts, media URLs
- Facebook Ad Library: advertiser, ad creative, status, start date, platforms, spend ranges (political ads)
- Off-limits: private profiles, friends, private groups, messages, any logged-in personal data
2.Stable public URLs to target
Prefer the mobile and public entry points. they are lighter and less JavaScript-heavy than the main app. The Ad Library has a public search UI and an underlying data surface keyed by page ID and country. Page IDs are more stable than vanity slugs, which can change.
- Public Page: https://www.facebook.com/<page-name>
- Mobile Page (lighter DOM): https://m.facebook.com/<page-name>
- Ad Library search: https://www.facebook.com/ads/library/?active_status=all&country=US&q=<brand>
- Ad Library by page: https://www.facebook.com/ads/library/?view_all_page_id=<page_id>
3.Facebook needs JavaScript rendering
Facebook ships an almost-empty HTML shell and builds the page with React after load. A plain HTTP fetch returns no usable content. You need a real browser that executes JavaScript and waits for the content to appear. With OmniScrape, set mode to js_rendering and provide a js_wait_selector for an element that only exists once the real content has rendered.
Because class names are randomized and change often, anchor your selectors on stable attributes. role, aria-label, data-testid where present, or text content. rather than on generated class names that break within days.
123456789101112131415161718import os, requests
resp = requests.post(
"https://api.omniscrape.io/v1/scrape",
headers={"X-API-Key": os.environ["OMNISCRAPE_KEY"]},
json={
"url": "https://www.facebook.com/ads/library/?active_status=all&country=US&q=nike",
"mode": "js_rendering",
"output_format": "html",
"js_wait_selector": "[role='main']",
"js_wait_timeout": 12000,
"proxy": "residential:us",
},
timeout=120,
)
body = resp.json()
html = body["data"]["content"] if body["success"] else None
print("rendered bytes:", len(html) if html else "FAILED")
4.Scraping the Ad Library
The Ad Library is the highest-value Facebook target for competitor research. Each ad card exposes the advertiser, creative (image or video thumbnail), status (active/inactive), start date, and the platforms it runs on (Facebook, Instagram, Messenger, Audience Network). For political and issue ads you also get impression and spend ranges.
Results are paginated by infinite scroll, so use Browser-as-a-Service when you need to load beyond the first batch. drive scroll actions and collect cards as they appear. For a first-page snapshot, js_rendering with a wait selector is enough.
- Advertiser name and page link
- Ad creative type (image / video / carousel) and media URL
- Status and start date; "active since" duration
- Platforms the ad runs on
- Spend and impression ranges (political / issue ads only)
5.Infinite scroll with Browser-as-a-Service
For deep Ad Library or Page feed collection you must scroll to trigger lazy loading. Connect Playwright to OmniScrape's hosted browser over WebSocket, scroll in a loop, and extract cards after each batch. OmniScrape hosts the browser pool and applies residential IPs, so you focus on the interaction logic.
12345678910111213141516171819import os, asyncio
from playwright.async_api import async_playwright
WS = f"wss://browser.omniscrape.io?apikey={os.environ['OMNISCRAPE_KEY']}&proxy_country=us"
async def main():
async with async_playwright() as p:
browser = await p.chromium.connect_over_cdp(WS)
page = await browser.new_page()
await page.goto("https://www.facebook.com/ads/library/?active_status=all&country=US&q=nike",
wait_until="networkidle")
for _ in range(10):
await page.mouse.wheel(0, 20000)
await page.wait_for_timeout(2000)
cards = await page.query_selector_all("[role='main'] a[href*='/ads/library/']")
print("collected cards:", len(cards))
await browser.close()
asyncio.run(main())
6.Rate limits and detection
Meta rate-limits aggressively and scores sessions on IP reputation, request cadence, and browser fingerprint. Hammering from a single datacenter IP gets you a checkpoint or a temporary block quickly. Route through residential IPs, keep request cadence human, and reuse a session for a coherent flow rather than firing bursts.
OmniScrape handles the fingerprint and IP rotation on its side. Your job is to keep volume reasonable and to back off on any block signal instead of retrying in a tight loop. See web scraping without getting blocked for the general playbook.
7.When to use the official Graph API instead
For data on Pages you own or manage, the official Facebook Graph API is the correct, supported path. it gives you insights, post metrics, and ad performance without scraping. Reserve scraping for public data you cannot get through the API, such as competitor Pages and the Ad Library, and always within Meta's terms and applicable law.
8.Compliance and ethics
Facebook data frequently includes personal data, which brings GDPR, CCPA, and similar regimes into play. Collect only public, non-personal business data; do not build profiles of individuals; honor deletion and do not store personal data you do not need. Review Meta's Terms of Service and the platform's automated-access rules. OmniScrape provides technical infrastructure. lawful, ethical use is your responsibility.
Frequently asked questions
Is it legal to scrape Facebook?
Scraping public, non-personal data such as the Ad Library is generally lower-risk, but Facebook's Terms of Service restrict automated access, and personal data triggers privacy laws like GDPR and CCPA. Never scrape private profiles or logged-in content. Consult the ToS and, for anything involving personal data, legal counsel.
Can I scrape Facebook without logging in?
You can access public Pages and the Ad Library without authentication, and that is the only surface you should scrape. Do not attempt to bypass the login wall to reach private content. it violates Meta's terms and likely the law. Use js_rendering to load the public content, which is built with JavaScript.
Why does my Facebook scraper return an empty page?
Facebook renders content with React after the initial HTML loads, so a plain HTTP fetch returns an empty shell. Use mode:js_rendering with a js_wait_selector like [role='main'] so the API returns the fully rendered HTML. See scraping JavaScript-rendered pages.
How do I scrape the Facebook Ad Library?
Target the public Ad Library URL with a country and search query or page ID, render it with js_rendering, and parse the ad cards. For results beyond the first batch, use Browser-as-a-Service to scroll and collect cards as they lazy-load.
Should I use the Graph API instead of scraping?
Yes, for any Page or ad account you own or manage. The official Graph API is supported and gives clean metrics without scraping. Reserve scraping for public competitor data and the Ad Library that the API does not expose.
Related guides