Skip to main content

<html lang="xx"> Language Attribute Validation

Issue No: 105

Category: Html Head Tags

Issue type: Issue

Priority: Critical

Description​

The lang attribute on the <html> element declares the page's primary human language using a BCP 47 language tag. Missing, empty, invalid, or mismatched values cause search engines to mis-classify page language, prevent language-targeted serving via hreflang, break screen reader pronunciation, and fail WCAG 3.1.1.

How do we capture it​

  1. Parse the <html> element and extract the lang attribute value from the raw HTML response.

    • Record the presence and exact string value of the attribute.
    • Normalise to lowercase for comparison.

  2. Validate the value against BCP 47 (IETF RFC 5646).

    • Split on - into subtags: language, script, region, variant.
    • Check the primary language subtag against the IANA Language Subtag Registry.
    • Flag subtags separated by _ (underscore) instead of - (hyphen) — a common error.
    • Flag deprecated primary language codes (iw → he, ji → yi, in → id).
    • Flag ambiguous Chinese without a script subtag (zh alone — should be zh-Hans or zh-Hant).

  3. Detect language mismatch using a language detection library (e.g. langdetect, franc).

    • Extract the visible text content of the page (strip HTML tags, scripts, styles).
    • Run language detection; record the detected language code and confidence score.
    • If the detected language differs from lang with confidence ≥ 0.85, flag is_mismatch = true.

  4. Check the HTTP Content-Language header in the response.

    • If present, compare to the lang attribute value.
    • Flag a conflict if the two values contradict each other.

  5. Check for hreflang consistency (cross-reference with the hreflang audit):

    • If hreflang annotations are present, verify that the declared lang code appears in at least one hreflang entry for the same page.

What to store​

FieldTypeComment
site_idintegerFK → sites(id)
page_idintegerFK → pages(id)
lang_attributetextRaw value of the lang attribute; NULL if attribute absent
is_presentbooleanWhether the attribute exists on <html>
is_emptybooleanWhether the attribute value is an empty string
is_valid_bcp47booleanWhether the value is a valid BCP 47 tag
language_codetextPrimary language subtag (e.g. en, fr, zh)
script_codetextScript subtag (e.g. Hans, Latn); NULL if absent
region_codetextRegion subtag (e.g. US, GB); NULL if absent
is_deprecatedbooleanWhether the language code is deprecated in the IANA registry
uses_underscore_separatorbooleanWhether _ is used instead of - in the tag
is_ambiguous_chinesebooleanWhether lang="zh" is present without a script subtag
detected_languagetextLanguage code detected from page text content
detection_confidencenumericConfidence score of language detection (0.0–1.0)
is_mismatchbooleanWhether declared lang differs from detected lang (confidence ≥ 0.85)
http_content_languagetextValue of the HTTP Content-Language response header; NULL if absent
has_header_conflictbooleanWhether Content-Language header contradicts the lang attribute
checked_attimestampWhen this check was last performed

Condition for trigger​

The following rules evaluate stored fields and produce diagnostics in the audit report.

  • Trigger Rule 1: Missing lang Attribute

    • Target Field: is_present
    • Evaluation Logic: is_present = false
    • Severity: CRITICAL
    • Diagnostic Message: "The <html> element has no lang attribute. Declare the page language (e.g. lang=\"en\") so search engines can correctly classify the page and serve it to users in the right locale."

  • Trigger Rule 2: Empty lang Attribute

    • Target Field: is_empty
    • Evaluation Logic: is_present = true AND is_empty = true
    • Severity: CRITICAL
    • Diagnostic Message: "The <html> element has an empty lang=\"\" attribute, which is invalid. Replace it with a valid BCP 47 language tag."

  • Trigger Rule 3: Invalid BCP 47 Tag

    • Target Field: is_valid_bcp47
    • Evaluation Logic: is_present = true AND is_empty = false AND is_valid_bcp47 = false
    • Severity: WARNING
    • Diagnostic Message: "The value [lang_attribute] is not a valid BCP 47 language tag. Use a registered IANA subtag (e.g. en, en-US, fr-FR, zh-Hans). Check https://www.iana.org/assignments/language-subtag-registry."

  • Trigger Rule 4: Underscore Separator in Language Tag

    • Target Field: uses_underscore_separator
    • Evaluation Logic: uses_underscore_separator = true
    • Severity: WARNING
    • Diagnostic Message: "The lang attribute value uses an underscore separator ([lang_attribute]). BCP 47 requires a hyphen (-). Change to the correctly hyphenated form."

  • Trigger Rule 5: Deprecated Language Code

    • Target Field: is_deprecated
    • Evaluation Logic: is_deprecated = true
    • Severity: WARNING
    • Diagnostic Message: "The language code [language_code] is deprecated in the IANA registry. Replace it with the current canonical code."

  • Trigger Rule 6: Language Mismatch (Detected vs Declared)

    • Target Field: is_mismatch
    • Evaluation Logic: is_mismatch = true AND detection_confidence >= 0.85
    • Severity: CRITICAL
    • Diagnostic Message: "The declared language is [lang_attribute] but automated detection identified the page content as [detected_language] with [detection_confidence] confidence. Correct the lang attribute to match the actual page language."

  • Trigger Rule 7: HTTP Header Conflicts with lang Attribute

    • Target Field: has_header_conflict
    • Evaluation Logic: has_header_conflict = true
    • Severity: WARNING
    • Diagnostic Message: "The HTTP Content-Language header ([http_content_language]) conflicts with the lang attribute ([lang_attribute]). Align both to the same BCP 47 value, preferring the lang attribute as the primary declaration."

  • Trigger Rule 8: Ambiguous Chinese Without Script Subtag

    • Target Field: is_ambiguous_chinese
    • Evaluation Logic: is_ambiguous_chinese = true
    • Severity: WARNING
    • Diagnostic Message: "The lang=\"zh\" attribute does not specify a script subtag. Use zh-Hans for Simplified Chinese or zh-Hant for Traditional Chinese so search engines can serve the correct script variant."

Sources​

Long description​

The HTML lang attribute provides the single most important language signal in a web page. While the HTTP Content-Language header and hreflang annotations also carry language information, the lang attribute on <html> is the authoritative in-document declaration that all four major rendering engines, all major search engine crawlers, and all screen readers rely on first.

Why the lang attribute matters for SEO

Search engines use the lang value to:

  • Assign the page to the correct language cluster for query matching.
  • Validate hreflang annotations (the lang code must align with the hreflang x-default or self-referencing entry).
  • Apply language-appropriate stemming, tokenisation, and stop-word filtering during indexing.
  • Qualify or disqualify the page for locale-targeted features such as rich results in specific language regions.

Why it matters for accessibility

WCAG 2.1 Success Criterion 3.1.1 requires the page language to be programmatically determined. Screen readers change pronunciation engines based on the lang value — a missing or wrong value means French pages are read with English phonology, making the content incomprehensible to visually impaired users.

BCP 47 tag structure

A valid tag is composed of IANA-registered subtags separated by hyphens:

language[-script][-region][-variant]
   en      Latn     US
   zh      Hans
   fr               FR

The primary language subtag (en, fr, zh) is mandatory. The script subtag (Hans, Hant, Latn) and region subtag (US, GB, FR) are optional but recommended when the meaning would be ambiguous without them.

Common failure patterns:

  • lang attribute missing entirely (most common on CMS-generated pages where the template is not locale-aware).
  • lang="" — attribute present but value is an empty string (often a placeholder left by a developer).
  • lang="en_US" — underscore used instead of hyphen (common copy-paste error from locale strings in programming languages).
  • lang="iw" — deprecated code for Hebrew (current code: he).
  • lang="zh" — ambiguous Chinese, does not distinguish Simplified from Traditional script.
  • lang set to the developer's language, not the content language (e.g. template default en on a Spanish-language page).
  • Content-Language: fr HTTP header while lang="en" on the page — contradictory signals confuse crawlers.

How search engines handle missing lang

Google and Bing will attempt to auto-detect the page language from the visible text content when lang is absent or invalid. Auto-detection is generally accurate for high-resource languages but unreliable for low-resource languages, mixed-language pages, and short pages with limited text. Relying on auto-detection rather than an explicit declaration is always an unnecessary risk.

How to Fix​

  1. Add lang to the <html> element with the correct BCP 47 tag:

    • <html lang="en"> — English
    • <html lang="en-US"> — American English
    • <html lang="zh-Hans"> — Simplified Chinese
    • Set it in the base layout/template so all pages inherit it automatically.

  2. Validate the tag against the IANA Language Subtag Registry at https://www.iana.org/assignments/language-subtag-registry or using the W3C I18n Checker at https://validator.w3.org/i18n-checker/.

  3. Replace deprecated codes with current equivalents: iw → he, ji → yi, in → id.

  4. Replace underscores with hyphens: en_US → en-US, zh_CN → zh-CN.

  5. Disambiguate Chinese: replace lang="zh" with lang="zh-Hans" (Simplified) or lang="zh-Hant" (Traditional) based on the actual script used.

  6. Align the HTTP Content-Language header with the lang attribute. Set Content-Language: en (language only, no region) at the server level if a single language is served from the origin; remove it if hreflang handles all language targeting.

  7. Audit CMS templates and locale settings: for multilingual sites, ensure the CMS outputs the correct lang value per language variant and that no template hard-codes a single language code for all locales.

Additional technical notes​

BCP 47 subtag hierarchy

A BCP 47 tag is composed of subtags separated by hyphens. Each position has a specific meaning:

language - script - region - variant - extension
  en       Latn     US
  zh       Hans
  sr       Cyrl     RS
  zh       Hant     TW
  • The language subtag is mandatory. It is a 2- or 3-letter ISO 639 code.
  • The script subtag (4 letters, title-case) is required when the language can be written in multiple scripts (Chinese: Hans/Hant; Serbian: Cyrl/Latn).
  • The region subtag (2-letter ISO 3166-1 country code) distinguishes regional variants (en-US vs en-GB).

Deprecated subtag examples to watch for

DeprecatedCurrentLanguage
iwheHebrew
jiyiYiddish
inidIndonesian
moroMoldovan Romanian
shsr-LatnSerbo-Croatian

Impact on Google hreflang

Google's hreflang annotation uses the same BCP 47 format. If the lang attribute is invalid or different from the hreflang declarations, Google may fail to match alternate language versions and show the wrong locale variant to users in different countries. Alignment between lang, hreflang, and Content-Language is a prerequisite for a functional international SEO setup.