Pagination: rel=next/prev or canonical

Issue No: 13

Category: page seo basics

Issue type: Warning

Priority: IMPORTANT

Description

Verify that paginated pages expose clear pagination signals and canonical handling so crawlers can understand sequence relationships and avoid indexing ambiguity.

How do we capture it

1. Fetch page URL with HTTP GET and record final URL and response headers.
2. Parse raw HTML and extract <link> tags from head and <a> tags from body navigation regions.
3. Detect pagination context using URL and HTML evidence:
   • Query patterns (page, paged, offset, start, cursor parameters).
   • Path patterns (/page/2/, /p/2/, /page-2/).
   • Presence of pagination anchors (next/prev labels or numbered pager links).
4. Extract canonical URL from <link rel="canonical">.
5. Extract <link rel="next"> and <link rel="prev"> href values when present.
6. Normalize all extracted URLs to absolute URLs for comparison and deduplicate duplicates.
7. Mark page as paginated only when evidence is strong (pattern plus navigation or rel links).
8. Trigger warning when paginated page has neither canonical nor rel next/prev signals.
9. Constraint: extraction relies only on raw HTTP response and raw HTML. No JavaScript-rendered pagination is evaluated.

What to store

Field	Type	Comment
page_url TEXT	TEXT	Original crawled page URL.
final_url TEXT	TEXT	Final URL after redirects.
is_html_response BOOLEAN	BOOLEAN	Whether response is HTML and eligible for this check.
is_paginated_page BOOLEAN	BOOLEAN	Whether page is classified as paginated.
pagination_detection_signals JSONB	JSONB	Evidence used to classify pagination.
canonical_url TEXT	TEXT	Canonical URL extracted from head.
has_canonical_tag BOOLEAN	BOOLEAN	Whether canonical link exists.
rel_next_url TEXT	TEXT	URL from rel next link tag if present.
rel_prev_url TEXT	TEXT	URL from rel prev link tag if present.
has_rel_next BOOLEAN	BOOLEAN	Whether rel next exists.
has_rel_prev BOOLEAN	BOOLEAN	Whether rel prev exists.
pagination_capture_error TEXT	TEXT	Parser errors or malformed metadata details.

Condition for trigger

This audit triggers when pagination metadata is missing or incomplete on paginated pages.

• Trigger Rule 1: Paginated Page Missing Canonical

  • Target Field: is_paginated_page
  • Evaluation Logic: is_paginated_page = TRUE AND has_canonical_tag = FALSE
  • Severity: WARNING
  • Diagnostic Message: "Paginated page does not include a canonical tag."

• Trigger Rule 2: Paginated Page Missing rel next and rel prev

  • Target Field: has_rel_next
  • Evaluation Logic: is_paginated_page = TRUE AND has_rel_next = FALSE AND has_rel_prev = FALSE
  • Severity: WARNING
  • Diagnostic Message: "Paginated page has no rel=next or rel=prev signals."

• Trigger Rule 3: No Pagination Signal At All

  • Target Field: pagination_detection_signals
  • Evaluation Logic: is_paginated_page = TRUE AND has_canonical_tag = FALSE AND has_rel_next = FALSE AND has_rel_prev = FALSE
  • Severity: WARNING
  • Diagnostic Message: "Paginated URL lacks canonical and pagination relationship metadata."

Sources

• Google Search Central Canonicalization: https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls
• Google Search Blog Pagination Guidance: https://developers.google.com/search/blog/2011/09/pagination-with-relnext-and-relprev
• WHATWG HTML Link Types: https://html.spec.whatwg.org/multipage/links.html

Long description

Pagination creates multiple URLs that often share template and topic context, which can lead to indexing uncertainty if relationship signals are weak. Canonical and pagination link metadata provide structural hints that help crawlers cluster sequence pages appropriately. Even where rel next/prev is not a direct ranking mechanism, clear internal pagination markup improves crawl path clarity and diagnostic consistency.

Common failure scenarios checked by the crawler:

• Paginated URL detected but no canonical link exists.
• Paginated URL has neither rel next nor rel prev metadata.
• Pagination markers exist in URL but no corresponding navigation signals exist in HTML.
• Extracted pagination URLs are malformed or unresolved.

How to Fix

For paginated templates, emit consistent canonical tags and include robust pagination navigation metadata. Ensure pager links are crawlable and URLs are valid absolute paths after template rendering. Keep pagination URL patterns stable across the site so crawlers can reliably detect sequence pages.