Plain text sitemaps in Django

by mark | 16 Jul 2022, 9:04 a.m.

I saw yet another 404. Something was looking for sitemap.txt. This is just a plain text file with a list of URLs, one per line. Putting this in was straightforward if you already have a sitemap.xml set up.

What you do is send the sitemaps object (with all of the URLs encoded in it) through a template that renders each of the items one per line. And then you make sure it is marked as returning plain text otherwise your browser will issue very peculiar HTML rendering errors. 

The first thing we need to do is set up the sitemap.txt URL. This is straightforward if you already have sitemap.xml set up as you will already have a sitemaps object with the lists in it. So the new URL looks like

    path('sitemap.txt', sitemap, 
        {'sitemaps':sitemaps, 'template_name':'sitemap.txt','content_type':'text/plain'},
        name='django.contrib.sitemaps.views.sitemap.txt',),

The template is also very straightforward. Note careful line breaks to ensure a plain list:

{% for url in urlset %}{{ url.location }}
{% endfor %}

And now you have a plain text list of URLs that some web crawlers seem to prefer. Look at mine!