Skip to Content
Prospecting data

Web scraping

Scrape, map, crawl and search the web, and extract structured data from pages.

Prospecting data6 endpointsKen-Version: 2026-11-02AuthenticationWorkspacesErrors
MarkdownOpenAPI

Headers

The endpoints on this page take these headers. Each endpoint's curl shows the ones it sends.

NameInTypeRequiredDescription
Ken-Versionheaderstring, one of: 2026-11-02, 1.1optionalThe API version, a date in YYYY-MM-DD form. 1.1 is another name for the first dated version. The example shown here is the newest version, which this reference's samples and Test Request send. Leave it out to use your API key's pinned version. Keys created in the Kenso app are pinned to the newest version; keys minted by the CLI login are not. A key with no pinned version and no header gets the legacy shape where the path has one, and 404 where the path exists only in this version. A date that is not a version answers 400 unsupported_version; any other value that is not a version is treated as legacy.
X-Ken-Workspace-Idheaderinteger (int64)optionalThe id of the workspace to act in. Leave it out when your API key reaches only one workspace. When the key reaches more than one, a request without this header answers 400 workspace_required. A workspace you are not a member of answers 404 not_found.

Scrape a page

POST/v1/scraping/scrapescraping.scrape
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

Read one public page and get its content, for example as markdown.

Send url and output options such as formats. A markdown-only request with no actions reads Kenso's web archive first: a free copy up to 18 months old (send maxAge for a newer one), marked metadata.kenSource: "archive" with metadata.cachedAt, the time of the copy. Each archive page counts against the daily webArchiveReads quota. Other pages are read live.

Errors:

  • 400 validation_failed: the request is not valid, or the URL is not public.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 quota_exceeded: the daily webArchiveReads quota is used up.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable) or failed.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/scrape" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 200

data is a ScrapingScrapeResult.

Show the 2 response fields and an example

The fields and an example are also on this endpoint's Markdown page.

Map a site's URLs

POST/v1/scraping/mapscraping.map
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

List the public URLs on a site, for example to pick pages to scrape or crawl. Send url, the site to map.

Errors:

  • 400 validation_failed: the request is not valid, or the URL is not public.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable) or failed.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/map" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 200

data is a ScrapingMapResult.

Show the 2 response fields and an example

The fields and an example are also on this endpoint's Markdown page.

Search the web

POST/v1/scraping/searchscraping.search
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

Search the web and get the matching pages. Send query, the search text.

Errors:

  • 400 validation_failed: the request is not valid.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable) or failed.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/search" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 200

data is a ScrapingSearchResult.

Show the 3 response fields and an example

The fields and an example are also on this endpoint's Markdown page.

Start a crawl

POST/v1/scraping/crawlsscraping.create-crawl
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

Start a crawl that follows links from one URL and scrapes the pages it finds, to read a whole site or a part of it.

Send url and limit, the most pages to scrape (1 to 1,000). Optional fields such as includePaths narrow the crawl.

The call answers 202 with the job. On Kenso's key, Kenso reserves the most credits the job can cost and settles when a status read sees it finish, so a webhook needs the workspace's own key. Poll GET /v1/jobs/{jobId}?type=crawl. When Kenso cannot record the job, it stops the crawl and answers 503.

Errors:

  • 400 validation_failed: limit is missing or outside 1 to 1,000, a URL is not public, webhook is sent on Kenso's key, or another field is not valid.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable), failed, or Kenso could not record the job.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/crawls" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 202

data is a Job.

Show the 14 response fields and an example

The fields and an example are also on this endpoint's Markdown page.

Start an extraction

POST/v1/scraping/extractionsscraping.create-extraction
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

Start an extraction that reads a list of pages and pulls structured data out of them, such as prices.

Send urls (1 to 1,000) and say what to pull out with prompt, schema or both.

The call answers 202 with the job. On Kenso's key, Kenso reserves the most credits the job can cost and settles when a status read sees it finish, so a webhook needs the workspace's own key. Poll GET /v1/jobs/{jobId}?type=extraction. An extraction cannot be cancelled (409 invalid_state).

Errors:

  • 400 validation_failed: urls is missing or lists over 1,000 pages, a URL is not public, webhook is sent on Kenso's key, or another field is not valid.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable), failed, or Kenso could not record the job.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/extractions" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 202

data is a Job.

Show the 14 response fields and an example

The fields and an example are also on this endpoint's Markdown page.

Start a batch scrape

POST/v1/scraping/batch-scrapesscraping.create-batch-scrape
Permission
Search edit
Credits
Varies with the request (web_scrape)
Rate limit
120 requests a minute per API key

Try it on developer.ken.so

Start a batch scrape that reads a list of pages in one job, instead of many single scrapes.

Send urls (1 to 1,000) and the same output options as a scrape, such as formats.

The call answers 202 with the job. On Kenso's key, Kenso reserves the most credits the job can cost and settles when a status read sees it finish, so a webhook needs the workspace's own key. Poll GET /v1/jobs/{jobId}?type=batch_scrape. When Kenso cannot record the job, it stops it and answers 503.

Errors:

  • 400 validation_failed: urls is missing or lists over 1,000 pages, a URL is not public, webhook is sent on Kenso's key, or another field is not valid.
  • 402 plan_limit_reached: the workspace is out of credits for a call on Kenso's key.
  • 429 rate_limited: the live source is busy. Wait for the time in Retry-After.
  • 503 service_unavailable: live reading is off (detail code provider_unavailable), failed, or Kenso could not record the job.

Requires Search edit.

Request body

application/json, optional: object.

Request
curl -X POST "https://api.ken.so/v1/scraping/batch-scrapes" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'

Response 202

data is a Job.

Show the 14 response fields and an example

The fields and an example are also on this endpoint's Markdown page.


On this page