Hugo and sitemap.txt (& a bit about robots.txt)⁠🤖
Updated  by  nm  25-May-25

I’m slowly moving the Infinite Ink website out of the 1990s and, as part of this process, I’m learning about sitemaps, which were introduced by Google in 2005 (~20 years ago).

Page contents

News

2020-October-23  Published this evolving⁠[1] article.

About sitemaps

Sitemaps are used by search engines to crawl and index websites. Details are at:

 

To learn about sitemaps in Hugo, see:

Hugo’s default sitemap

Hugo’s default is to output a file named sitemap.xml in a project’s public/ directory using the built-in sitemap template, which you can view at:   github.com/gohugoio/hugo/blob/master/tpl/tplimpl/embedded/templates/sitemap.xml

In May 2025 this template looks like this:

{{ printf "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"yes\"?>" | safeHTML }}
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xhtml="http://www.w3.org/1999/xhtml">
  {{ range where .Pages "Sitemap.Disable" "ne" true }}
    {{- if .Permalink -}}
  <url>
    <loc>{{ .Permalink }}</loc>{{ if not .Lastmod.IsZero }}
    <lastmod>{{ safeHTML ( .Lastmod.Format "2006-01-02T15:04:05-07:00" ) }}</lastmod>{{ end }}{{ with .Sitemap.ChangeFreq }}
    <changefreq>{{ . }}</changefreq>{{ end }}{{ if ge .Sitemap.Priority 0.0 }}
    <priority>{{ .Sitemap.Priority }}</priority>{{ end }}{{ if .IsTranslated }}{{ range .Translations }}
    <xhtml:link
                rel="alternate"
                hreflang="{{ .Language.LanguageCode }}"
                href="{{ .Permalink }}"
                />{{ end }}
    <xhtml:link
                rel="alternate"
                hreflang="{{ .Language.LanguageCode }}"
                href="{{ .Permalink }}"
                />{{ end }}
  </url>
    {{- end -}}
  {{ end }}
</urlset>

 

As you can see in the emphasized line 4 above, this produces a list of all .Pages, which have not been sitemap disabled.

Using sitemap.txt instead of sitemap.xml

As part of my foray into the world of sitemaps, I’ve decided to use a simple sitemap.txt file rather than a complicated (to me) sitemap.xml file. To learn about the sitemap.txt file, see:

To set up Infinite Ink’s website to use a sitemap.txt file, I did the following three steps.

 

1. Override the default sitemap output file name

To tell Hugo to name the sitemap file sitemap.txt, I put the following in Infinite Ink’s config.yaml:

sitemap:
  filename: sitemap.txt

 

If your Hugo config file is in TOML rather than YAML, you can use this syntax:

[sitemap]
  filename = "sitemap.txt"

 

2. Override the default sitemap template

To use my own sitemap template, rather than what’s listed above, I created layouts/_default/sitemap.xml in Infinite Ink’s project root that contains this:

Infinite Ink’s sitemap template
https://www.ii.com/
{{ range (where site.RegularPages "Section" "in" site.Params.mainSections) -}}
https://www.ii.com{{.RelPermalink}}
{{ end -}}
https://www.ii.com/tags/
https://www.ii.com/portal/activism-hacktivism/
https://www.ii.com/portal/alpine-mail/
https://www.ii.com/portal/asciidoc/
https://www.ii.com/portal/backup-sync/
https://www.ii.com/portal/browsers/
https://www.ii.com/portal/business-economics-money/
https://www.ii.com/portal/cheatsheet/
https://www.ii.com/portal/cli/
https://www.ii.com/portal/content/
https://www.ii.com/portal/elsewhere/
https://www.ii.com/portal/emoji/
https://www.ii.com/portal/fastmail/
https://www.ii.com/portal/forked/
https://www.ii.com/portal/git/
https://www.ii.com/portal/git-bash/
https://www.ii.com/portal/gohugo/
https://www.ii.com/portal/golang/
https://www.ii.com/portal/health-life-personal/
https://www.ii.com/portal/intellij/
https://www.ii.com/portal/iusethis/
https://www.ii.com/portal/joplin/
https://www.ii.com/portal/markdown/
https://www.ii.com/portal/mastodon/
https://www.ii.com/portal/mathematics/
https://www.ii.com/portal/mathjax/
https://www.ii.com/portal/messaging/
https://www.ii.com/portal/meta/
https://www.ii.com/portal/nix-nux/
https://www.ii.com/portal/philosophy/
https://www.ii.com/portal/privacy-security/
https://www.ii.com/portal/productivity/
https://www.ii.com/portal/qutebrowser/
https://www.ii.com/portal/tech/
https://www.ii.com/portal/tumblelog/
https://www.ii.com/portal/twitter/
https://www.ii.com/portal/unicode/
https://www.ii.com/portal/utf-8/
https://www.ii.com/portal/vim/
https://www.ii.com/portal/vscode/
https://www.ii.com/portal/webdev/
https://www.ii.com/portal/windows/
https://www.ii.com/portal/words/
https://www.ii.com/portal/zeitgeist/


{{/*
* TODO:
*  - manually add portal when it’s ready to be listed in search engines
*  - get rid of trailing space on each line in sitemap.txt output file
*/}}

Note that I hand code which portals are listed because some Infinite Ink portals are not ready for the public.

 

💡
To learn about site.Params.mainSections, which is used in line 2 in the above template, see gohugo.io/functions/collections/where/#portable-section-comparison.

 

3. Update robots.txt

To tell web crawlers about this sitemap file, I added the following to the bottom of Infinite Ink’s robots.txt file.

Sitemap: https://www.ii.com/sitemap.txt

 

I did this directly on Infinite Ink’s web server, but it’s possible to use Hugo to maintain a website’s robots.txt.

To learn about robots.txt, see wikipedia.org/wiki/Robots.txt.

 

View Infinite Ink’s sitemap.txt and robots.txt

To view Infite Ink’s sitemap.txt and robots.txt, follow these links:

 

‼
Sitemaps (sitemap.txt, sitemap.xml, etc.) and robots.txt must be UTF-⁠8 encoded.

 

See also

For more about Hugo, see Infinite Ink’s…

 

Endnote

1. Many Infinite Ink pages, including this one, are evergreen 🌲 and regularly updated.

Please share & discuss 📝 👎 👍 📯