Filtering, Sorting & Pagination

Imagine you build an e-commerce API and create a GET /api/products endpoint that returns every product in the database. During development with 20 test products, everything works fine. Then you deploy to production with 100,000 products. The response is a massive JSON payload — megabytes of data. The request takes 10 seconds. The client app crashes trying to render 100,000 items. Your database server is under heavy load.

This is why every production API needs advanced query capabilities: filtering, sorting, pagination, and field selection. These features let clients request exactly the data they need — nothing more, nothing less.

Filtering — "Show me only electronics under $200." Without filtering, the client must download all 100,000 products and filter them in the browser. With filtering, the server returns only the 47 products that match, saving bandwidth and processing time.

Sorting — "Show me the newest products first" or "Show me the cheapest products first." Users expect to sort search results. The server should handle this because sorting 100,000 items on the client is slow and wasteful when the server can do it efficiently using database indexes.

Pagination — "Show me products 21-40" (page 2 with 20 items per page). This is the most critical feature. Instead of sending all 100,000 products, send them 20 at a time. The user sees a fast response and can navigate through pages. Most users never go past page 3 anyway.

Field selection — "I only need the name and price, not the full description and reviews." This reduces the payload size significantly. If each product has a 5KB description but the list view only shows the title and price, field selection can reduce the response size by 90%.

All of these features are controlled via query parameters in the URL: GET /api/products?category=electronics&price[lte]=200&sort=-createdAt&page=2&limit=20&fields=name,price. This is the REST convention, and it is what frontend developers and API consumers expect.

Filtering allows clients to narrow down results by specifying conditions in query parameters. The server translates these parameters into database queries.

Simple equality filters are the most straightforward:

This translates to the Mongoose query: Product.find({ category: 'electronics', brand: 'Apple' }).

In Express, query parameters are available on req.query:

const { category, brand } = req.query;
const filter = {};
if (category) filter.category = category;
if (brand) filter.brand = brand;
const products = await Product.find(filter);

Comparison operators let clients filter by ranges — prices between $50 and $200, ratings above 4 stars:

Express parses bracket notation into nested objects: req.query = { price: { gte: '50', lte: '200' }, rating: { gt: '4' } }. You need to convert these to MongoDB operators by adding the $ prefix:

let queryStr = JSON.stringify(req.query);
queryStr = queryStr.replace(/\b(gte|gt|lte|lt)\b/g, match => `$${match}`);
const filter = JSON.parse(queryStr);
// Result: { price: { $gte: '50', $lte: '200' }, rating: { $gt: '4' } }

Search/regex filters let clients search by partial text:

You can convert this to a case-insensitive regex: { name: { $regex: 'phone', $options: 'i' } }. This matches "iPhone", "Phone Case", "Smartphone", etc.

Important: Before applying query parameters as filters, you must strip out non-filter parameters. Parameters like sort, page, limit, and fields are not filters — they control how results are presented, not which results are returned. Remove them before building the filter object:

const excludeFields = ['sort', 'page', 'limit', 'fields'];
excludeFields.forEach(field => delete queryObj[field]);

Sorting controls the order in which results are returned. It is specified via the sort query parameter.

Ascending sort (A to Z, lowest to highest):

Translates to: Product.find().sort('price') or Product.find().sort({ price: 1 }).

Descending sort (Z to A, highest to lowest) — prefix with a minus sign:

Translates to: Product.find().sort('-price') or Product.find().sort({ price: -1 }). The minus sign convention is standard across most REST APIs.

Multiple sort fields — separate with commas for tiebreakers:

This sorts by highest rating first. For products with the same rating, it sorts by lowest price. Translates to: Product.find().sort('-rating price') (Mongoose uses spaces, not commas).

Implementation in Express:

if (req.query.sort) {
  // Convert comma-separated to space-separated for Mongoose
  const sortBy = req.query.sort.split(',').join(' ');
  query = query.sort(sortBy);
} else {
  // Default sort: newest first
  query = query.sort('-createdAt');
}

Always provide a default sort order. Without a default, the order of results is unpredictable and may change between queries. Sorting by -createdAt (newest first) is the most common default for content-based APIs. For search results, sorting by relevance is typical.

Performance note: Sorting requires the database to examine all matching documents and order them. For large datasets, ensure you have database indexes on fields you frequently sort by. Without an index, sorting 100,000 documents requires reading them all into memory and sorting them — slow and memory-intensive. With an index, MongoDB can read the documents in sorted order directly — fast and efficient.

Pagination splits results into pages so clients receive manageable chunks of data. This is arguably the most important API feature for performance.

The query parameters:

• page — Which page to return (1-indexed, meaning page 1 is the first page)
• limit — How many items per page

The math:

const page = parseInt(req.query.page) || 1;   // Default: page 1
const limit = parseInt(req.query.limit) || 20; // Default: 20 items
const skip = (page - 1) * limit;               // How many items to skip

// Page 1: skip 0, take 20  (items 1-20)
// Page 2: skip 20, take 20 (items 21-40)
// Page 3: skip 40, take 20 (items 41-60)

const results = await Product.find(filter).skip(skip).limit(limit);

Pagination metadata — clients need to know how many total pages exist, whether there is a next/previous page, and how many total results match the filter. Return this alongside the data:

const total = await Product.countDocuments(filter);
const totalPages = Math.ceil(total / limit);

res.json({
  data: results,
  pagination: {
    page,
    limit,
    total,
    totalPages,
    hasNextPage: page < totalPages,
    hasPreviousPage: page > 1,
  },
});

Safety limits — Always enforce a maximum limit to prevent abuse. A client requesting ?limit=1000000 would defeat the purpose of pagination:

const limit = Math.min(parseInt(req.query.limit) || 20, 100);

This caps the limit at 100, regardless of what the client requests.

Cursor-based pagination is an alternative used by APIs with frequently changing data (social media feeds, real-time logs). Instead of page numbers, the server returns a cursor (usually the last item's ID), and the client requests the next batch after that cursor: ?after=abc123&limit=20. This avoids the problem where inserting new items shifts all page offsets. Twitter, Facebook, and Slack APIs all use cursor-based pagination.