Adam Caudill

Security Engineer, Researcher, & Developer

Hugo & Content-Based Related Content

During my Christmas vacation last year, I converted this site from WordPress to Hugo; while I’ve been happy with the change, a couple of features are missing. One of these is that there was a section with related content at the bottom of each post. I wanted to get it back.

Thankfully Hugo has native support for Related Content, so while I was hoping this would be a simple task, there’s a note that made things substantially more complicated:

We currently do not index Page content.

You see, Hugo uses the front matter of each post to determine what posts are related, this includes keywords, tags, and the published date — it does not use the content of the post itself. So if you’ve carefully added keywords to all of your posts, then there’s no issue, and you can stop reading now.

For me, this is a challenge. Over the years, this site has been manually curated HTML, WordPress (multiple times), Octopress, Jeykll, and now Hugo. While I maintained keywords in the past, they’ve not made it through all of the conversions — and I’ve stopped adding them to new posts. The prospect of manually adding keywords to all of my posts going back to 2003 wasn’t exactly exciting.

Finding a Solution #

After a bit of research and being disappointed that I couldn’t find anyone that had already solved this, I set out to find a viable solution to the problem. Thankfully, academia has largely solved this problem already with keyword extraction, a branch of Natural Language Processing.

There are a variety of methods and techniques available, as well as libraries in a wide variety of languages. Given that Python is popular in the academic data science community, I focused on that. I started looking into TF-IDF (Term Frequency — Inverse Document Frequency) which initially appeared to be a viable choice. However, after finding a useful comparison of the results, I looked at RAKE (popular library, but abandoned and due to a bug, does better at short demos than real tasks), BERT (another popular library, I moved on after digging into test cases), and finally settled on using TextRank (PDF) as implemented in Gensim1.

Using TextRank, I could easily read all of my posts, generate keywords, then, using python-frontmatter, write the updated front matter back to the file. Then use Hugo’s native Related Content feature.

Related Content in Hugo #

This feature is relatively simple and adds little time to the building process — so there’s little impact on workflow, and the live reload feature works just as well. While the feature isn’t particularly sophisticated, it works well enough to get the job done.

Using it can be as simple as adding a related.html file to partials with some simple code:

{{ $related := .Site.RegularPages.Related . | first 5 }}
{{ with $related }}
<h3>See Also</h3>
	{{ range . }}
	<li><a href="{{ .RelPermalink }}">{{ .Title }}</a></li>
	{{ end }}
{{ end }}

Then include it with {{ partial "related" . }} in your partials/article.html file. Simple as can be.

Bringing it to Life #

I wrote a relatively simple Python script to generate keywords for all posts, and this can be run before building the site (or committing changes, if you have an automated workflow). This script allows me to have exactly what I wanted: content-based related content, without manually managing keywords.

Important: This will replace all keywords; if you already have keywords on your posts, this will replace them.

This script is set up for my needs and my workflow; if you wish to use it, you may need to make some changes to ensure that it works correctly for you.

Hopefully, others will find this helpful, and help to address a limitation in Hugo.

  1. To be more accurate, was implemented in Genesis. It was removed in version 4.0 - thus requiring finding another library or using an older version. I opted for the latter. While they aren’t that happy with it, it seems to work well enough for this use case. ↩︎

Adam Caudill

Related Posts

  • The (Questionable) Future of YAWAST

    The last release of YAWAST was on January 1, 2020; while the release history was sometimes unpredictable, the goal was a new release each month with new features and bug fixes. I intentionally took January off from the project. In February, I left the company I was at; the team of penetration testers there had helped to inspire new features while looking for ways to make them more productive. But something else happened in February, an issue was opened – something that appeared to be simple, but in fact, made me realize that the entire project was in doubt.

  • Glue & Reuse

    This post was imported from an old blog archive, and predates the creation of On an average day, I take about 10 screenshots, and upload them to my FTP server. Now, this is easy enough, but it does take a minute, let’s break down the process: Press Alt+PrtScr. Go to the start menu, and use the “Run” command to open mspaint. Paste (and crop if needed). Save the image.

  • 1Password 8 Early Access: Security, Comments, & FAQs

    A few days ago, 1Password (my employer) released the first preview of the new application for macOS. The response has been rather dramatic. The release was followed by an excellent blog post by Michael Fey explaining the story of how we got here, and some of the decisions that were made in the process. I’d like to now to a few minutes to answer some questions, provide some insight, and share my thoughts on this release.

  • Write Like You Are Running Out of Time

    The cultural phenomenon that is Hamilton, brought back to the forefront due to its streaming release, is an artistic feat, but it also serves as an opportunity to refresh our memories on the history behind these characters, and look for opportunities to learn lessons that apply today. This is exactly what I’ve been doing. For all of his flaws, one thing that I have to respect about Alexander Hamilton (as well as his wife, Eliza) is the understanding of the long-term impact of the written word.

  • YAWAST: News & Mission

    It’s been some time since I last wrote about YAWAST on here, it was actually back in April when I posted the last update – that was for the release of YAWAST v0.7.0. Currently, it’s at version 0.11.0 and a lot has changed. It’s been rewritten from scratch, more people have become involved, it has moved to a (fairly) regular release cycle, and has expanded a fair bit in terms of functionality.