XML Sitemap

What does an XML sitemap actually change?

Discovery, mostly on the pages that need it least visibly: a new page with few internal links, a large site where crawling is spread thin, a section that changed recently. It does not make a page rank, does not make it index, and does not override a noindex or a robots.txt rule.

The corresponding rule is that a sitemap should list canonical, indexable URLs and nothing else. Listing a redirect beside its target, or a noindex page, is a contradictory signal rather than a thorough one. This site enforces that as a build check rather than as a convention: every static URL declared in the sitemap is verified against the built route manifest, and one that does not resolve fails the build. It exists because the sitemap shipped for weeks listing nine URLs that returned 404, and nothing else caught it.

Why is a wrong lastmod worse than no lastmod?

Because trust in the field is decided per site rather than per entry. Google uses lastmod only when it is consistently accurate, and a content management system that stamps a new date on every save, including a typo fix, teaches the engine to ignore the signal for the whole site. The guidance from Google when this comes up is that sites with unintentionally wrong dates are probably better off without lastmod at all.

So the discipline is to stamp it on significant change only, and to make sure it agrees with the visible date on the page and the dateModified in the markup. Three surfaces, one truth. This site omits lastmod entirely for pages with no authored revision date rather than fabricating one from the build.

Do AI assistants read your sitemap?

None of them documents doing so, and it would be a reasonable thing to do, which is the honest state of the answer. What is certain is that Google’s surfaces sit on Google’s index, so a sitemap that helps Search helps AI Overviews by the same route.

Worth resisting: the recommendation to publish an llms.txt as an AI-facing sitemap. Ahrefs found 97% of llms.txt files received zero requests in a month across 137,210 domains, which is a strong result for a file the industry treats as essential. If you want an AI-facing equivalent of a sitemap, the honest version is the ordinary one: a complete, accurate XML sitemap of canonical URLs, which every crawler that reads sitemaps already knows how to use.

The one genuinely useful adjacent surface is a markdown twin of each page generated from the same source as its HTML, because it serves the content rather than a list of links to it. Nothing documents requesting those either, and the difference is that the cost of publishing them is a build step rather than a file somebody has to maintain by hand.

Related terms

• Robots.txtRobots.txt is a file at a site’s root telling automated crawlers which paths they may fetch.
• Crawl BudgetCrawl budget is the amount of crawling an engine will do on a site, set by what the server can take and by how much the engine wants the content.
• Canonical TagA canonical tag is a link element naming the preferred URL for a page when the same content is reachable at more than one address.
• llms.txtllms.txt is a proposed file at a site’s root that lists its important pages in markdown, so an AI agent can find them without parsing the site.