Fetching a Remote JSON API at Build Time in Hugo with resources.GetRemote

Most tutorials that put third-party API data on a static site reach for client-side JavaScript. The page loads, a fetch runs, and content appears a moment later. It works, and it throws away most of what a static site is for. Search engines see an empty container, every visitor costs you an upstream request, and a rate limit or an outage at the provider becomes a broken page for everyone.

Hugo can do the fetch during the build instead. The response gets baked into the HTML, the file on disk is complete, and the API is contacted once per build rather than once per visitor.

The Function

resources.GetRemote URL [OPTIONS] returns a resource, which you then unmarshal.

The current idiom wraps it in try, which gives you a value with .Err and .Value rather than failing the build outright:

{{/* layouts/shortcodes/artworks.html */}}
{{ $q := .Get "q" | default "landscape" }}
{{ $n := .Get "n" | default "12" }}
{{ $url := printf "https://api.artic.edu/api/v1/artworks/search?q=%s&query[term][is_public_domain]=true&fields=id,title,artist_title,date_display,image_id&limit=%s" (urlquery $q) $n }}
{{ $opts := dict "headers" (dict "AIC-User-Agent" "example.org ([email protected])") }}

{{ with try (resources.GetRemote $url $opts) }}
  {{ with .Err }}
    {{ errorf "artwork fetch failed: %s" . }}
  {{ else with .Value }}
    {{ $data := . | transform.Unmarshal }}
    <div class="art-grid">
      {{ range $data.data }}
        {{ if .image_id }}
          <figure>
            <img src="https://www.artic.edu/iiif/2/{{ .image_id }}/full/843,/0/default.jpg"
                 loading="lazy" alt="{{ .title }}">
            <figcaption>
              {{ .title }}{{ with .artist_title }}, {{ . }}{{ end }}{{ with .date_display }}, {{ . }}{{ end }}
            </figcaption>
          </figure>
        {{ end }}
      {{ end }}
    </div>
  {{ else }}
    {{ errorf "no response from %q" $url }}
  {{ end }}
{{ end }}

Call it from any page with {{< artworks q="Paris street" n="9" >}}.

That line is written with an escape, and it has to be. Hugo extracts shortcodes from content before markdown ever runs, so a literal {{< ... >}} call inside backticks or inside a fenced code block still gets executed. Documenting a shortcode in a post is the fastest way to break your own build with failed to extract shortcode: template for shortcode "x" not found. Wrap the inner delimiters in /* and */ and Hugo prints the call instead of running it. Plain Go template braces, {{ }}, are safe in content, since content files are not templates.

Note the if .image_id rather than with .image_id. Inside a with, the dot rebinds to the value you tested, so .title on the next line would be out of scope. That one catches people constantly.

Options You Will Actually Need

Options go in as a dict. The useful keys:

  • headers, a dict of header name to value. Many public APIs ask for a user-agent identifying your project, and some rate-limit harder without one.
  • method and body, for APIs that require POST for search.
  • timeout, as a duration string.
  • key, to override the cache key Hugo derives from the arguments.

A POST example, since several public data APIs need it:

{{ $opts := dict
  "method" "post"
  "body" `{"filters": {}, "limit": 20}`
  "headers" (dict "Content-Type" "application/json")
}}

Two Behaviours That Will Surprise You

A 404 is not an error. Hugo does not classify a 404 response as a failure, so .Err stays empty and you fall through to unmarshalling a body that is not the JSON you expected. If the endpoint you are hitting returns 404 for an empty result set, check the shape of the parsed data rather than trusting .Err alone.

The cache never expires by default. Remote resources go into the getresource cache, and its default maxAge is -1, which means entries live until you clear them. This is excellent for build speed and quietly fatal if you assumed a nightly rebuild would pull fresh data. Your “daily rotating gallery” will serve the same twelve artworks for a year.

Set it explicitly:

[caches.getresource]
  dir = ':cacheDir/:project'
  maxAge = '24h'

Worth knowing that getresource defaults to :cacheDir/:project, not the resources/_gen directory where processed assets and images land. Clearing resources/_gen does nothing to it. hugo --gc or deleting the cache directory is what you want.

When Build Time Is Wrong

Two cases. If the data genuinely changes faster than you deploy, and you cannot trigger a rebuild on a schedule, you need a runtime fetch. If the API needs a secret, a build-time fetch keeps the key out of the browser, which is good, but it will end up in your CI environment and your build logs if you are careless with errorf.

For everything else, including any archive of public-domain images, reference data, or anything you would otherwise paste into a data file by hand, fetch it at build time and ship HTML.

Related