SDKs

JavaScript / TypeScript

The official scrapeflow package for Node.js and the browser (server-side). Fully typed, tree-shakeable, zero required config beyond a key.

Install

bash

npm install scrapeflow

Initialize

client.ts

import ScrapeFlow from "scrapeflow";

export const client = new ScrapeFlow({
  apiKey: process.env.SCRAPEFLOW_API_KEY!,
  // baseUrl defaults to https://api.scrapeflow.dev
  maxRetries: 3,        // retries 429/5xx with backoff
  timeout: 60_000,      // ms
});

Usage

// Scrape
const page = await client.web.scrape({
  url: "https://linear.app",
  formats: { markdown: true, json: true },
  jsonSchema: { title: "string", pricingTiers: "string[]" },
});

// Search
const hits = await client.web.search({ query: "vector databases", limit: 10 });

// Answers
const research = await client.web.answers({ query: "Who founded Vercel?" });

// Brand
const brand = await client.brand.get({ domain: "notion.com" });

// Async: crawl + batches
const crawl = await client.web.crawl({ url: "https://docs.stripe.com", limit: 500 });
const result = await client.jobs.wait(crawl.job_id);

Error handling

import { ScrapeFlowError, RateLimitError } from "scrapeflow";

try {
  await client.web.scrape({ url });
} catch (err) {
  if (err instanceof RateLimitError) {
    // retry after err.retryAfter seconds
  } else if (err instanceof ScrapeFlowError) {
    console.error(err.status, err.code, err.message);
  }
}

ℹ

The SDK retries 429 and 5xx automatically with exponential backoff (configurable via maxRetries). Failed requests never consume credits.

Batch helpers

const batch = await client.batches.create({ urls, formats: { markdown: true } });

// Await completion, or stream results as they land:
for await (const item of batch.stream()) {
  console.log(item.url, item.data.markdown);
}