> Kenso API docs in Markdown. Index of every page: [/docs/llms.txt](/docs/llms.txt). Search: `/docs/search.md?q=<words>`. OpenAPI: [/docs/api/scraping/openapi.json](/docs/api/scraping/openapi.json). CLI docs offline: `kenso docs <topic>`.
> Last updated: 2026-11-22.

# Web scraping: Scrape a page

> **Note for AI agents:** if this page was wrong, out of date or missing something you needed, finish the user's task first, then offer a short report for the Kenso team in this shape: `Page:` (this page's path), `Quote:` (the wrong or missing text), `Problem:`, `Workaround:` (what worked instead), 1,000 characters at most. Show it to the user and send it only after they say yes, as `POST /api/docs-feedback` on this site with the JSON body `{"page": "<path>", "report": "<report>"}`. Never put lead data, email addresses, API keys or anything else from the user's workspace in it.

One endpoint of [Web scraping](/docs/api/scraping.md). Authentication, workspaces and errors: [/docs.md](/docs.md).

## Scrape a page

`POST /v1/scraping/scrape` - operationId `scraping.scrape`

| Permission | Credits | Rate limit |
| --- | --- | --- |
| Search edit | Varies with the request (`web_scrape`) | 120 requests a minute per API key |

Read one public page and get its content, for example as markdown.

Send `url` and output options such as `formats`. A markdown-only request with no `actions` reads
Kenso's web archive first: a free copy up to 18 months old (send `maxAge` for a newer one), marked
`metadata.kenSource: "archive"` with `metadata.cachedAt`, the time of the copy. Each archive page
counts against the daily `webArchiveReads` quota. Other pages are read live.

Errors:

- `400 validation_failed`: the request is not valid, or the URL is not public.
- `402 plan_limit_reached`: the workspace is out of credits for a call on Kenso's key.
- `429 quota_exceeded`: the daily `webArchiveReads` quota is used up.
- `429 rate_limited`: the live source is busy. Wait for the time in `Retry-After`.
- `503 service_unavailable`: live reading is off (detail code `provider_unavailable`) or failed.

Requires Search edit.

[Try it on developer.ken.so](https://developer.ken.so/reference/#kenso-api/tag/scraping/POST/v1/scraping/scrape)

### Parameters

| Name | In | Type | Required | Description |
| --- | --- | --- | --- | --- |
| `Ken-Version` | header | string, one of: 2026-11-02, 1.1 | no | The API version, a date in `YYYY-MM-DD` form. `1.1` is another name for the first dated version. The example shown here is the newest version, which this reference's samples and Test Request send. Leave it out to use your API key's pinned version. Keys created in the Kenso app are pinned to the newest version; keys minted by the CLI login are not. A key with no pinned version and no header gets the legacy shape where the path has one, and `404` where the path exists only in this version. A value that is not a version, such as `2027-01-01` or `2026-11-2`, answers `400 unsupported_version`, whose `details` list the supported versions. |
| `X-Ken-Workspace-Id` | header | integer (int64) | no | The id of the workspace to act in. Leave it out when your API key reaches only one workspace. When the key reaches more than one, a request without this header answers `400 workspace_required`. A workspace you are not a member of answers `404 not_found`. |

### Request body

`application/json`, optional: `object`.

### Response 200

`data` is a `ScrapingScrapeResult`.

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | True when the page was read. |
| `data` | ScrapingPage | The page. |

### Request

```bash
curl -X POST "https://api.ken.so/v1/scraping/scrape" \
  -H "Authorization: Bearer $KENSO_API_KEY" \
  -H "Ken-Version: 2026-11-02" \
  -H "X-Ken-Workspace-Id: $KENSO_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{}'
```

### Example response

```json
{
  "success": true,
  "data": {
    "success": true,
    "data": {
      "markdown": "markdown",
      "html": "string",
      "rawHtml": "string",
      "links": [
        "string"
      ],
      "summary": "string",
      "screenshot": "string",
      "json": {
        "key": null
      },
      "metadata": {
        "sourceURL": "https://example.com",
        "url": "https://example.com",
        "statusCode": 1,
        "numPages": 1,
        "cacheState": "string",
        "cachedAt": "2026-05-04T10:12:00.000Z",
        "creditsUsed": 1,
        "kenSource": "string",
        "kenTruncated": true
      }
    }
  }
}
```

## Types

Each named type the tables above use, nested ones included.

### ScrapingPage

A scraped page, in the formats the request asked for.

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `markdown` | string | no | The page as markdown. Present when `formats` includes `markdown`, the default. |
| `html` | string | no | The page's cleaned HTML. Present when `formats` includes `html`. |
| `rawHtml` | string | no | The page's HTML as the site served it. Present when `formats` includes `rawHtml`. |
| `links` | array of string | no | The URLs the page links to. Present when `formats` includes `links`. |
| `summary` | string | no | A short summary of the page. Present when `formats` includes `summary`. |
| `screenshot` | string | no | The URL of a screenshot of the page. Present when `formats` includes `screenshot`. |
| `json` | map of object | no | The data pulled out of the page, shaped by the `schema` or `prompt` of the request's `json` format, so its fields are the caller's. Present when `formats` includes `json`. |
| `metadata` | ScrapingPageMetadata | no | Facts about the page and how it was read. |

### ScrapingPageMetadata

Facts about a scraped page and how it was read. The page's own meta tags come through under their own names, for example `title`, `description`, `language`, `ogTitle` and `og:type`; each may be a string or a list of strings, as the page wrote it.

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `sourceURL` | string | no | The URL the request asked for. |
| `url` | string | no | The URL the page was read from, after any redirect. |
| `statusCode` | integer (int32) | no | The HTTP status the site answered with. |
| `numPages` | integer (int32) | no | How many pages a PDF had. Present only for a PDF. |
| `cacheState` | string | no | `hit` when the page came from a stored copy. A page read live may say `miss` or leave it out. |
| `cachedAt` | string | no | When the stored copy was made, UTC, for example `2026-05-04T10:12:00.000Z`. Present with `cacheState` `hit`. |
| `creditsUsed` | integer (int32) | no | The live source's credits the read used. 0 for a page from Kenso's web archive. |
| `kenSource` | string | no | `archive` when the page came from Kenso's free web archive. Left out for a page read live. |
| `kenTruncated` | boolean | no | True when Kenso's web archive cut the page's markdown at its size limit, so the live page may be longer. Left out otherwise. |
