Extract product prices from a list of URLs

Start with a catalog you can inspect

This tutorial extracts a title, displayed price and availability from a short list of product URLs. It uses Books to Scrape, a public practice catalog, so the selectors and expected currency are explicit. It does not assume that all shops share the same layout.

To inspect a page first without a key, try Website to text. For named fields, use the runnable recipe below with your own FetchRelay API key.

Run the recipe

Install Node.js 22 or later, create a key in Account, and set FETCHRELAY_API_KEY in your terminal environment. Never put the key in a public repository or frontend app.

git clone https://github.com/JAIPilot/fetchrelay-integrations.git
cd fetchrelay-integrations/examples/web-data-recipes
node product-prices.mjs product-urls.json > prices.json

The supplied product-urls.json contains one product:

["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"]

You can add up to ten pages from the same catalog. Requests run sequentially, and duplicate URLs are processed once. Each successful page consumes a FetchRelay credit.

How the fields are extracted

The script sends the following extraction options to the hosted API:

{
  "formats": ["text"],
  "extract": {
    "title": "h1",
    "price": ".price_color",
    "availability": ".availability"
  }
}

Named values arrive in data.extract. The program validates that the title is nonempty and the price matches the catalog's pound-denominated decimal format. A successful output row includes priceText, currency: "GBP", availability, source URL and collection time. JSON output preserves the displayed value for inspection; it does not turn missing prices into zero or infer a currency from an unrelated symbol.

Adapt it to a real store

Inspect a representative product's HTML and choose selectors for the correct variant and visible price. Change the selectors and currency validation together. Check whether the displayed number includes tax, requires a subscription, applies only to a sale, or changes with geography. A price range is not a single price, and an old crossed-out price is not necessarily the current offer.

For pages that publish structured product data, FetchRelay also has a product extraction workflow. Compare structured metadata with the actual visible offer before relying on it. Neither selectors nor structured metadata guarantee universal extraction.

Keep errors visible

A failed row has success: false and a safe error message. If an account or allowance error occurs, processing stops; remaining input URLs are unattempted. Any failed row makes the process exit nonzero, even though the JSON file can still contain successful rows. Read that exit status in automation rather than assuming that a file means the batch succeeded.

The recipe does not retry automatically, follow API redirects or accept another API host. Warnings and malformed prices fail validation. Unit tests cover missing prices and account-error stopping; the release smoke check extracts the real Books to Scrape fixture.

Get the complete code. Once one layout works, use bounded batch jobs for larger collections.