Pagination in REST API Design: Cursor vs Offset at Scale

Why Pagination Matters in REST API Design

When I was building the backend for AudioBook AI — a platform that now serves 50K+ users — I made a rookie mistake. I loaded 10,000 records into a single API response because I hadn't thought about pagination. The first user complaint came within hours: the app crashed, network requests timed out, and the database melted.

That's when I realized: pagination isn't optional in REST API design. It's fundamental. Whether you're using Node.js backend or Laravel, how you paginate directly impacts your API performance, user experience, and infrastructure costs.

There are two main strategies: offset pagination and cursor-based pagination. Both work. But one scales, and the other doesn't. Let me show you the difference from 8+ years of production experience.

Offset Pagination: The Classic Approach

Offset pagination is what most developers learn first. You ask for a page number and a limit:

GET /api/v1/audiobooks?page=2&limit=20

The API then returns:

• Items 21–40 (skipping the first 20)
• Total count
• Current page

Why it's intuitive: Users understand "page 2." It's familiar. You can jump directly to any page.

Why it breaks at scale: To get page 1000, the database must scan and skip the first 19,999 rows. With millions of records, this becomes prohibitively expensive. I learned this the hard way when our EmpSuite ERP platform's report endpoint started taking 30+ seconds.

Cursor-Based Pagination: The Modern Standard

Cursor-based pagination doesn't skip rows. Instead, it remembers where you left off and starts from that point. You get a cursor (typically an encoded string or ID) instead of a page number:

GET /api/v1/audiobooks?limit=20&cursor=eyJpZCI6IDEwMjN9

The response includes:

• 20 items
• A next_cursor for fetching the next batch
• No total count (you don't need it for forward-only navigation)

Why it's superior at scale: The database seeks directly to where the cursor points. No scanning. No skipping. O(1) lookups instead of O(n).

This is what Stripe, GitHub, and Facebook use. At CodeBrew Labs, when we migrated Nova Cabs' ride history API from offset to cursor-based pagination, response times dropped from 2.3s to 140ms.

Implementing Cursor Pagination in Node.js

Here's how I implement cursor-based pagination in a Node.js backend with Express and a relational database:

const express = require('express');
const mysql = require('mysql2/promise');
const Buffer = require('buffer').Buffer;

const app = express();

// Encode cursor: base64(JSON)
const encodeCursor = (id) => {
  return Buffer.from(JSON.stringify({ id })).toString('base64');
};

// Decode cursor
const decodeCursor = (cursor) => {
  try {
    const decoded = Buffer.from(cursor, 'base64').toString('utf-8');
    return JSON.parse(decoded);
  } catch (e) {
    return null;
  }
};

app.get('/api/v1/audiobooks', async (req, res) => {
  const { limit = 20, cursor } = req.query;
  const numLimit = Math.min(parseInt(limit), 100); // Cap at 100

  try {
    const connection = await mysql.createConnection({
      host: process.env.DB_HOST,
      user: process.env.DB_USER,
      password: process.env.DB_PASSWORD,
      database: process.env.DB_NAME,
    });

    let whereClause = '';
    const params = [];

    // If cursor exists, start after that ID
    if (cursor) {
      const decodedCursor = decodeCursor(cursor);
      if (!decodedCursor) {
        return res.status(400).json({ error: 'Invalid cursor' });
      }
      whereClause = 'WHERE id > ?';
      params.push(decodedCursor.id);
    }

    // Fetch limit + 1 to check if there's a next page
    const query = `
      SELECT id, title, author, created_at
      FROM audiobooks
      ${whereClause}
      ORDER BY id ASC
      LIMIT ?
    `;

    params.push(numLimit + 1);
    const [rows] = await connection.execute(query, params);

    const hasMore = rows.length > numLimit;
    const items = rows.slice(0, numLimit);

    const response = {
      data: items,
      pagination: {
        limit: numLimit,
        next_cursor: hasMore ? encodeCursor(items[items.length - 1].id) : null,
      },
    };

    await connection.end();
    return res.json(response);
  } catch (err) {
    console.error(err);
    return res.status(500).json({ error: 'Internal server error' });
  }
});

app.listen(3000);

Key points in this Node.js implementation:

• We encode the cursor as base64 (safe for URLs)
• We fetch limit + 1 to detect if more pages exist
• We use an indexed ORDER BY id for fast seeks
• No expensive COUNT(*) query — we just check if the next record exists

Building Cursor Pagination in Laravel

Laravel has built-in cursor pagination support, which I use heavily. Here's how:

<?php

namespace App\Http\Controllers;

use App\Models\Audiobook;
use Illuminate\Pagination\Cursor;
use Illuminate\Pagination\CursorPaginator;

class AudiobookController extends Controller
{
    public function index()
    {
        // Laravel handles encoding/decoding automatically
        $audiobooks = Audiobook::query()
            ->orderBy('id', 'asc')
            ->cursorPaginate(
                perPage: request('limit', 20)
            );

        return response()->json([
            'data' => $audiobooks->items(),
            'pagination' => [
                'limit' => $audiobooks->perPage(),
                'next_cursor' => $audiobooks->nextCursor(),
                'prev_cursor' => $audiobooks->previousCursor(),
            ],
        ]);
    }
}

// routes/api.php
Route::get('/audiobooks', [AudiobookController::class, 'index']);

Why Laravel's cursor pagination is elegant:

• cursorPaginate() handles cursor encoding/decoding internally
• Automatic support for nextCursor() and previousCursor()
• Works with Eloquent relationships seamlessly
• Built-in request binding — no manual parsing

If you need more control, you can build custom cursor logic like I did for EmpSuite's complex filtering. But for most REST API design scenarios, this is production-ready out of the box.

API Performance: When to Choose Each Strategy

Here's my decision matrix based on 8 years of production systems:

Use Offset Pagination When:

• Dataset is small (<100K records)
• Random access is critical (e.g., "jump to page 500")
• Users need a total count (e.g., search results: "Showing 1–20 of 4,532")
• You're building internal tools where performance isn't user-facing

Use Cursor Pagination When:

• Dataset is massive (1M+ records)
• Sequential browsing is the norm (feeds, timelines, logs)
• Data changes frequently (cursor prevents duplicates/gaps)
• You're optimizing for mobile (infinite scroll, lower bandwidth)
• Your business depends on response time (e-commerce, real-time systems)

At AudioBook AI, switching to cursor pagination reduced our server costs by 25% because we eliminated expensive COUNT(*) queries and slow offset scans. That's real money saved.

Common Pitfalls & How to Avoid Them

Pitfall 1: Unordered Results Break Cursors

Your cursor strategy fails if results aren't consistently ordered. Always use a stable, indexed column (usually id or created_at). If you need secondary sort, include it in the cursor:

// Good: primary + secondary sort
const encodeCursor = (id, createdAt) => {
  return Buffer.from(
    JSON.stringify({ id, created_at: createdAt })
  ).toString('base64');
};

Pitfall 2: Forgetting Index on Cursor Column

If your cursor column (id, created_at) isn't indexed, the database still scans. In your migration:

Schema::create('audiobooks', function (Blueprint $table) {
    $table->id();
    $table->string('title');
    $table->index('id'); // Explicit index for cursor
    $table->index('created_at');
    $table->timestamps();
});

Pitfall 3: Exposing Internal IDs in Cursors

If you encode raw IDs, users can infer your scale ("we're at ID 5M?"). Use opaque tokens instead. This is where encoding as base64 helps — it hides the underlying structure.

Pitfall 4: Not Validating Cursor Format

Always validate and sanitize cursors. I've seen APIs crash because a malformed cursor broke JSON parsing. The example code above handles this with try-catch.

Key Takeaways

• Cursor-based pagination is the modern standard for REST API design. It scales linearly regardless of dataset size, while offset pagination degrades exponentially.
• Use offset pagination only for small datasets (<100K records) or when random page access is critical. For most production REST APIs, cursor pagination is the right choice.
• Always index your cursor column. An indexed id or created_at turns an O(n) scan into an O(1) seek. This is the difference between 2.3s and 140ms responses.
• Encode cursors as opaque tokens. Base64-encoding JSON provides security through obscurity and prevents users from guessing internal IDs.
• Test pagination with realistic data volumes. Build cursor pagination into your Node.js backend and Laravel REST APIs from the start. It's not an afterthought — it's foundational to API performance.