HTTP vs browser scraping: which mode should you use?
Choose between HTTP, Automatic, and Browser rendering in FetchRelay. Learn how to diagnose missing content, redirects, bot challenges, and partial results.
Start with the page you actually need
HTTP mode downloads the server’s response without running page JavaScript. Browser mode opens a page in Chromium and can see content created by scripts. Neither mode guarantees that a remote website will provide the content you intended. Start with one representative URL and compare the output with the source before choosing a default for a whole collection.
| Page or symptom | First choice | What to check |
|---|---|---|
| Article or documentation in the initial HTML | HTTP | Headings, paragraphs, and source links |
| Product page with server-rendered fields | HTTP | Title, price, currency, and availability |
| Visible content only appears after JavaScript | Browser | The actual content, not a loading shell |
| Unsure whether JavaScript is required | Automatic | Which engine ran and any quality warnings |
| Login wall, CAPTCHA, or denied access | Neither is a guarantee | Whether you have a supported, permitted source |
What Automatic mode does
Automatic starts with an HTTP request and can use the browser when the first response suggests missing dynamic content or an access problem. It is a fallback strategy, not a promise to bypass protection. Inspect diagnostics.engine to understand which path produced the output and diagnostics.warnings for qualifications that need your attention.
Some sites send a real HTTP 200 response containing an error page, a consent screen, or an empty storefront. FetchRelay rejects recognized failures and warns about some partial results. Your application still needs task-specific checks: a page can contain perfectly readable text that is the wrong product, region, or destination.
Compare the modes on one target
- Open the workbench and enter the target URL.
- Select HTTP, choose Markdown, and run the extraction.
- Check the title, first meaningful paragraph, source URL, and any required fields.
- If the content needs JavaScript, rerun with Browser. This is a separate request and a successful extraction uses another credit.
- Keep the mode that produces the information your task needs. Do not pick a mode from elapsed time alone.
For a programmatic request, change only the mode field between http, browser, and auto. This keeps the comparison understandable. See the Python example or Node.js example.
Browser rendering is not proxy routing
A browser changes how the page executes. A proxy changes the network path seen by the destination. A page that works in your own browser can still reject a browser running in a cloud environment. FetchRelay’s current offering does not include managed residential proxies, CAPTCHA solving, login-wall access, or saved authenticated browser profiles. Buying a higher plan does not change those limitations.
Set a quality gate before you scale
Require a recognizable title, a plausible content length, the expected destination, and the fields your task depends on. Treat missing prices as missing data, not zero. Record a collection timestamp and source URL. Review warnings, and keep failed or partial pages separate from clean records in a downstream dataset.
Start a small batch only after the single-page checks pass. Large jobs can return partial results. Failed extraction is not charged, while successful extracted pages consume credits. The pricing page explains reservations and settlement.