Canonical tags on all pages
Issue No: 6
Category: page seo basics
Issue type: Issue
Priority: CRITICAL
Description
Verify that every crawlable HTML page includes exactly one valid canonical link element in the head section so search engines can consolidate indexing signals to the intended URL.
How do we capture it
- Request the target page URL using HTTP GET and record final response metadata (status, content-type, final URL after redirects).
- Continue only when the response is HTML (
text/htmlor equivalent) and status is a successful or indexable response. - Parse raw HTML text and extract all
<link>elements from the<head>block. - Filter extracted links where
relcontains canonical (case-insensitive, tokenized by whitespace). - Count canonical candidates and extract canonical
hrefvalues exactly as present. - Normalize each canonical candidate for comparison:
- Trim whitespace.
- Resolve relative URL against the fetched page URL.
- Normalize host casing and remove default ports.
- Preserve path and query as declared for audit comparison.
- Mark canonical as valid only when there is exactly one canonical link and one non-empty parseable absolute URL.
- Store extraction diagnostics for malformed tags, duplicate canonical tags, or unresolved relative values.
- Constraint: this check uses only raw HTTP response and raw HTML text. No JavaScript execution or browser-rendered DOM is used.
What to store
| Field | Type | Comment |
|---|---|---|
| page_url TEXT | TEXT | Original crawled URL requested by crawler. |
| final_url TEXT | TEXT | Final URL after redirect chain resolution. |
| http_status_code INTEGER | INTEGER | HTTP response status code for fetched page. |
| is_html_response BOOLEAN | BOOLEAN | Whether response content type indicates HTML. |
| canonical_tag_count INTEGER | INTEGER | Total number of canonical link tags found in head. |
| canonical_raw_hrefs JSONB | JSONB | Raw canonical href values extracted before normalization. |
| canonical_url TEXT | TEXT | Primary normalized canonical URL chosen from extraction. |
| canonical_is_parseable BOOLEAN | BOOLEAN | Whether canonical URL can be parsed as valid absolute URL. |
| canonical_is_present BOOLEAN | BOOLEAN | Whether at least one canonical tag exists. |
| canonical_has_single_tag BOOLEAN | BOOLEAN | Whether exactly one canonical tag exists. |
| canonical_capture_error TEXT | TEXT | Parser or normalization error summary when capture fails. |
| canonical_extracted_at TIMESTAMPTZ | TIMESTAMPTZ | Timestamp when canonical extraction was completed. |
Condition for trigger
This audit triggers when canonical coverage is missing, malformed, or ambiguous.
-
Trigger Rule 1: Missing Canonical Tag
- Target Field:
canonical_is_present - Evaluation Logic:
canonical_is_present = FALSE - Severity: CRITICAL
- Diagnostic Message: "No canonical tag was found on this page."
- Target Field:
-
Trigger Rule 2: Multiple Canonical Tags
- Target Field:
canonical_has_single_tag - Evaluation Logic:
canonical_has_single_tag = FALSE - Severity: CRITICAL
- Diagnostic Message: "Multiple canonical tags were detected; search engines may ignore or misinterpret canonical signals."
- Target Field:
-
Trigger Rule 3: Canonical URL Invalid
- Target Field:
canonical_is_parseable - Evaluation Logic:
canonical_is_parseable = FALSE - Severity: CRITICAL
- Diagnostic Message: "Canonical tag exists but the canonical URL is malformed or not parseable."
- Target Field:
Sources
- Google Search Central Canonicalization: https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls
- WHATWG HTML Link Type canonical: https://html.spec.whatwg.org/multipage/links.html#link-type-canonical
- RFC 3986 URI Generic Syntax: https://www.rfc-editor.org/rfc/rfc3986
Long description
Canonicalization defines which URL should represent a page when multiple URLs can expose substantially similar content. Search engines use canonical tags to cluster duplicates and transfer ranking signals to the preferred URL. If a canonical tag is absent, malformed, or duplicated, indexing systems may choose an unintended URL as canonical, fragmenting link equity and creating reporting inconsistencies.
Common failure scenarios checked by the crawler:
- Canonical link tag is missing from the HTML head.
- More than one canonical tag is present.
- Canonical href is empty, relative without valid base resolution, or syntactically invalid.
- Canonical href uses unsupported or malformed protocol.
- Non-HTML responses are incorrectly audited as if canonical were required.
How to Fix
Ensure every indexable HTML page includes one canonical link element in the head with a valid absolute URL. Keep canonical generation deterministic in templates so exactly one tag is emitted. Validate canonical output during deployment by crawling representative URLs and confirming canonical tag presence, uniqueness, and URL validity.