How to Paginate API Results: Best Practices

📝 Blog5 min read

How to Paginate API Results: Best Practices

Pagination is essential for APIs that return large datasets. Without it, a single request can return thousands of records, causing slow responses, high memory usage, and poor user experience. This guide covers how to paginate API results using modern best practices, comparing offset and cursor-based approaches, and showing real-world implementation patterns.

Why Pagination Matters

Problem Without PaginationImpact
Large response payloadsSlow network transfer, high latency
Memory spikes on serverOut-of-memory errors under load
Database timeoutsFull table scans block other queries
Poor mobile experienceLong loading times, app crashes
Unbounded costsCloud egress and compute bills surge

A well-designed pagination strategy keeps APIs fast, predictable, and scalable.

Offset vs Cursor Pagination

The two dominant pagination patterns are offset-based and cursor-based. Each has distinct tradeoffs.

AspectOffset PaginationCursor Pagination
MechanismSkip N records, take MUse opaque cursor from last item
ConsistencyCan miss or duplicate rowsStable, consistent ordering
Database costOFFSET scans all skipped rowsIndex seek on cursor column
Deep page performanceSlows dramatically at high offsetsConstant time per page
ComplexitySimple, widely understoodRequires cursor encoding
Use caseAdmin panels, static catalogsFeeds, real-time apps

Rule of thumb: Use offset pagination for small, mostly static datasets. Use cursor pagination for large, frequently updated datasets or infinite scroll.

REST API Pagination Patterns

Offset Pagination with Limit

The simplest approach passes offset and limit query parameters.

GET /api/orders?offset=40&limit=20

Pros: Easy to implement, familiar to clients, supports random page access. Cons: Database must scan and discard skipped rows, rows can shift between requests.

Cursor Pagination with bookmarks

Return a next_cursor opaque token in the response. Clients send it back to fetch the next page.

GET /api/feed?cursor=eyJpZCI6MTIzfQ&limit=20

Pros: Constant query cost, no duplicates or misses, ideal for real-time feeds. Cons: No random page access, cursor usually encodes internal state.

Keyset Pagination

Use explicit column values instead of an opaque cursor.

GET /api/orders?last_id=1024&limit=20

Pros: Transparent, cacheable, index-friendly. Cons: Requires a stable unique ordering column.

GraphQL Pagination Patterns

Relay-Style Cursor Connections

GraphQL community standards recommend Relay-style cursor connections.

query {
  products(first: 20, after: "opaqueCursor") {
    edges { node { id name } cursor }
    pageInfo { endCursor hasNextPage }
  }
}

This pattern bundles edges, cursors, and navigation metadata into a single response.

Offset-Based in GraphQL

Simpler but less common in production GraphQL APIs.

query {
  users(offset: 40, limit: 20) {
    id name
  }
}

Modern Best Practices for Pagination

Best PracticeWhy It MattersImplementation
Always set a max page sizePrevent abuse and runaway queriesCap `limit` at 100-1000
Return total count carefullyCounting large tables is expensiveUse estimated counts or omit totals
Include navigation linksClients should not construct URLs`next`, `prev`, `first`, `last` in headers or body
Sort deterministicallyOrder matters for cursor stabilityAlways sort by unique column(s)
Document cursor expiryCursors may invalidate over timeDefine TTL and refresh strategy
Cache paginated responsesReduce database loadCache at CDN or app layer
Support both formatsDifferent consumers need different patternsOffer offset and cursor endpoints

Response Envelope Example

A modern response envelope gives clients everything they need to navigate.

{
  "data": [...],
  "pagination": {
    "limit": 20,
    "offset": 40,
    "total": 542,
    "next_cursor": "eyJpZCI6NDB9",
    "prev_cursor": "eyJpZCI6MjB9"
  }
}

For cursor pagination, omit total if counting is expensive, and rely on hasNextPage.

Database Query Patterns

Efficient pagination requires the right query shape.

PatternQuery ExamplePerformance
OffsetSELECT * FROM posts ORDER BY id LIMIT 20 OFFSET 40Slows at high offset
KeysetSELECT * FROM posts WHERE id > 1024 ORDER BY id LIMIT 20Constant time
Cursor seekSELECT * FROM posts WHERE created_at > '...' ORDER BY id LIMIT 20Constant time

Add covering indexes on ordering columns. Avoid SELECT * in paginated endpoints; project only needed fields.

Infinite Scroll vs Pagination UI

Choose the right UX pattern for your product.

UI PatternBest ForSEO Impact
Traditional paginationBlogs, documentation, search resultsHigh: each page is crawlable
Infinite scrollSocial feeds, image galleriesLow: requires special handling
Load more buttonE-commerce, moderate feedsMedium: partial crawlability

For SEO-critical content, traditional pagination with rel="next" and rel="prev" links ensures search engines discover all pages.

Common Pagination Mistakes

MistakeConsequenceFix
No max page sizeAbuse, DB overloadEnforce hard limit on `limit`
Counting total on every requestSlow queries, timeoutsCache counts, use估算 counts
Inconsistent sortingDuplicate or missing itemsAlways sort by unique column first
Exposing internal IDs in cursorsSecurity risk, couplingSign or encode cursors
Ignoring timezone in date cursorsWrong results for global usersStore timestamps in UTC

Conclusion

Effective pagination balances client experience, database performance, and API security. Start with cursor pagination for new APIs handling large or dynamic datasets. Add offset pagination only when random page access is a hard requirement. Always document limits, enforce maximum page sizes, and monitor query performance in production.

Frequently Asked Questions

Cursor-based pagination is best for large or frequently updated datasets because it provides consistent results and constant query performance. Offset pagination works well for small, mostly static datasets where random page access is needed.

Use Relay-style cursor connections with first/after and last/before arguments. Return edges, nodes, and pageInfo containing endCursor and hasNextPage. This is the de facto standard for production GraphQL APIs.

Offset pagination uses LIMIT/OFFSET to skip rows, which becomes slow at high offsets and can miss or duplicate rows during updates. Cursor pagination uses a bookmark from the last row, offering constant-time queries and stable ordering.

Avoid returning exact totals for large datasets because COUNT(*) is expensive. Use estimated counts, omit totals, or cache them separately. Only return totals for small, static collections.

Keyset pagination uses explicit column values (e.g., last_id or created_at) instead of an opaque cursor. It is simpler than cursor pagination, more cacheable, and still provides constant-time database queries.

Use cursor pagination with a next_cursor token. Load one page at a time as the user scrolls. Track scroll position for SEO with History.pushState or server-rendered pagination fallback. Avoid offset pagination because deep pages become slow.

Advertisement

Related Articles