Skip to content
FetchRelay

Website to Markdown

Give your knowledge base useful source material.

Turn a public documentation page into Markdown you can inspect, store, and pass to your own search or retrieval pipeline. Keep the source URL beside the content so you can trace it back.

Useful for: Developers building documentation search, internal knowledge bases, or retrieval-augmented applications.

Try website to MarkdownWorkbench instructions
The result you are aiming for
{
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples…",
  "metadata": { "sourceURL": "https://example.com/" }
}

Illustrative excerpt of the data object

From the first request to a repeatable workflow

  1. 01

    Start with one public page

    Open the workbench, choose Scrape, and enter a URL. Select Markdown and Links. Automatic rendering starts with HTTP and can open a browser when a page needs JavaScript.

  2. 02

    Check what was actually extracted

    Compare the title, headings, code examples, and important paragraphs with the source. Main-content extraction removes common navigation and layout noise, but every website is different. Inspect the result before indexing it.

  3. 03

    Move to a bounded crawl

    Choose Crawl for a documentation section, set a small page limit, and use an include-path filter such as /docs/**. Sign in and enable background mode when you need a saved job that can resume across worker runs.

  4. 04

    Connect your own pipeline

    Download JSON or JSONL, or read the saved result API. Store source URLs and content hashes with each document. Chunking, embeddings, vector storage, and answer generation belong to your application; FetchRelay supplies the extracted source content.

Make the API request

Create a key in Account and set it as FETCHRELAY_API_KEY in your environment. Keep it out of client-side code and source control.

curl --fail-with-body https://fetchrelay.com/api/scrape \
  -H "Authorization: Bearer $FETCHRELAY_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com",
  "formats": [
    "markdown",
    "links"
  ],
  "onlyMainContent": true
}'

This is a live request and uses account credits. See the OpenAPI reference for the complete response schema.

Check before you ship

  • The result contains the actual content, not a login screen or error message.
  • Important headings, links, and code snippets survived extraction.
  • Each stored document retains its source URL for attribution and refreshes.

Know the limits

FetchRelay does not generate embeddings or answers. PDF support reads embedded text; scanned documents need a separate OCR step. Login walls and bot challenges can prevent extraction. Collect and reuse only material you have permission to use.

Common questions

Can I crawl an entire documentation website?

Use a signed-in background crawl with a page limit and path filters. The current maximum is 1,000 pages per job with a 20 MB saved-dataset budget. These are limits, not a guarantee that every site or large job will complete.

Does this replace a vector database?

No. FetchRelay extracts source content. Your application decides how to split, index, store, and retrieve that content.

What happens when a page fails?

Failed extraction is not charged page credits. Crawl and batch jobs keep successful records and release unused reservations when settled.

Try it on a page you actually need.

Inspect the output before committing to an integration. Create an account for 1,000 free monthly credits, saved jobs, monitors, and API keys. No credit card required.

Open the workbench See pricing