Pagination

Handling large datasets efficiently.

How Pagination Works#

All list endpoints return paginated results. By default, we return 50 items per page with a maximum of 100.

...

of 1,250 total items

Parameters#

Parameter	Type	Description
`page`	number	Page number to retrieve (default: 1)
`limit`	number	Items per page (default: 50, max: 100)

Response Metadata#

Every paginated response includes a meta object:

Response Structure

{
  "data": [...],
  "meta": {
    "total": 1250,      // Total items
    "page": 1,          // Current page
    "per_page": 50,     // Items per page
    "total_pages": 25   // Total pages
  }
}

Iterating All Pages#

Here's how to fetch all items across multiple pages:

fetchAllItems.js

async function fetchAllItems(projectSlug, apiKey) {
  let allItems = [];
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(
      `https://dataclean.to/v1/${projectSlug}/items?page=${page}&limit=100`,
      { headers: { 'X-API-Key': apiKey } }
    );
    const data = await response.json();

    allItems = allItems.concat(data.data);
    hasMore = page < data.meta.total_pages;
    page++;
  }

  return allItems;
}

Rate Limits

Be mindful of rate limits when iterating through many pages. Add delays between requests if you're processing large datasets.

Rate Limits by Plan#

Plan	Requests/Month	Requests/Minute
Free	1,000	10
Starter	50,000	60
Pro	500,000	300
Agency	5,000,000	1,000

Optimizing Requests

Use limit=100 when fetching all data to minimize the number of API calls needed.

API Endpoints Back to Docs