Skip to main content

This Site: The Making Of

12 - 18 min read

Static Site Generation

I've been reading about Static Site Generation and decided I'd to learn about it by doing. My personal site was due a rebuild and this would present a good training ground. With no deadlines I'd be able to take my time, go the slow way around and learn as much as possible along the way.

After researching and having a play around with Hugo (written in Go) and Jekyll (Ruby) I ended up deciding on Metalsmith (for JS/Node). Metalsmith is extremely modular – the base system is only 3-400 lines of code and intentionally has very limited functionality – it just takes files from a source folder and pipes them into a build folder. Everything else is done with plugins.

What Metalsmith Gives Me

I ended up with:

  • A fully templated site. I only have to maintain HTML markup in a few places, all of which inherit a parent template with header/footer/nav etc
    • home page template
    • blog post template
    • item (artwork etc) template
    • contact, privacy and a simple page to show the RSS link). Over 150 pages of artworks are automatically built from some JSON files. The JSON is just an array of objects like this - each item becomes a page:
  {
    "id": "interaction",
    "title": "Interaction",
    "tags": "art,people",
    "desc": "<p>People bouncing off each other.</p>",
    "materials": "Acrylic on board",
    "w": "60",
    "h": "60",
    "d": "2",
    "date": "2008 01 01"
  },

becomes:

<section id="main-content" itemscope itemtype="https://schema.org/Painting" class="content">
    <meta itemprop="creator" content="Mark Mayes">
    <div class="side-column">

        <section class="item-details">
            <h1 itemprop="http://www.schema.org/name">Interaction</h1>
            <h3>Acrylic on board, 60&nbsp;x&nbsp;60cm</h3>
            <h2><meta itemProp="datePublished" content="2008-01-01"><time datetime="2008-01-01">2008</time></h2>
            <span itemprop="http://www.schema.org/description">
                <p>People bouncing off each other.</p>
            </span>
        </section>

        <footer class="page-tags">
            <h3>tagged in</h3>
            <ul itemprop="keywords">
                <li><a href="/by/art" rel="tag">art</a></li>
                <li><a href="/by/people" rel="tag">people</a></li>
            </ul>
        </footer>
        <nav class="prev-next-btns">
            <h2 class="screen-reader-text">Post navigation</h2>
            <a href="/art/play-time#pic" rel="next">Play Time</a>
            <a href="/art/sixty-one-aliens#pic" rel="prev">Sixty One Aliens</a>
        </nav>
    </div>
    <div class="main-column">
        <div id="pic" class="item-image">
            <img id="magnifiable-image" alt="Click to magnify" src="/images/items/main/interaction.jpg">
        </div>
        <p class="alert magnify-prompt">slide mouse/finger over pic to magnify</p>
        <a href="/images/items/hi_res/interaction.jpg" download="Interaction (Firm Gently)" class="download-wallpaper-btn">download wallpaper</a>
    </div>
</section>
  • Automatic creation of topic listing pages based on any tags I add to content.

  • Ability to write blog posts in markdown, with:

    • drafts functionality
    • automatic excerpt creation
    • automatic creation of collections (enables simple next/previous buttons, paging etc)
    • word count with estimated reading time
    • by inserting an image using standard markdown I get it resized into various sizes and proper srcset/sizes markup created (I wrote a custom renderer to make that happen. It was easy (writeup coming soon)
  • An automatic site map.

  • Contact page (via Netlify Forms).

  • Nice syntax-highlighting for blog posts containing code (via a metalsmith plugin which uses highlight.js).

  • An automatically-created tag cloud.

  • Proper permalinks.

  • Dates in my data automatically converted into multiple formats.

  • Automatic RSS feed.

  • Automatic link checking (internal and external) every time I build the site.

  • Every time I commit to git the site gets built and pushed live. Two versions of the site get built (a light and dark theme) and get pushed to firmgently.co.uk/gentlyfirm.co.uk

  • All of the above happens quickly - build time on my tablet for local testing takes just over 5 minutes.

    If I don't need to test locally (eg. if it's a quick blog post) I can skip that part and just push my changes with git. A couple of minutes later it will have been built and deployed by Netlify to their nice quick CDN. They do some kind of clever cache-flushing which means all visitors will start seeing the new version of the site immediately, despite being on a CDN – so, best of both worlds. And all this is for free, as in I am not paying anything for hosting.

    It's so easy and cheap to serve static content nowadays that Netlify do it just to be nice (they make their money in other ways like by selling add-ons to help overcome some of the limitations of static sites... well I'm sure they have other business reasons too but in my experience so far they have been pretty nice anyway). For example I can use their forms to allow people to contact me via the site. This is something that's not as simple on as static site as it is on a 'normal' website, where PHP or something similar will do the work of processing the contact form and sending me an email.

    I get up to 100 messages per month for free and if I go over that amount I have to upgrade to a (pretty cheap) paid form processing tier. If I'm getting over 100 messages per month through this site then that implies that it's become more of a focus to me and something I don't mind paying a little for. Netlify also handle spam detection etc so that's one less thing to worry about.

Metalsmith Plugins

Each plugin does one specific, encapsulated job. They work in a pipeline sequence: the source files are piped from one plugin to the next, transforming their contents as they go along.

The pipeline for this site ended up being quite long – there's more functionality than may be apparent and as mentioned each plugin only has one simple job to do. Despite the length I'm going to detail them here because if you're trying to learn about Metalsmith it might be useful.

The list below is based on the console output from a typical build at the time of writing. It'll probably have changed a little since then but the gist will be similar. I'm using a plugin called metalsmith-timer which times each step of the process and prints a label when its complete (this info helps with debugging and optimisation).

Timings

I've included the timings for each step below. It's important to note that my mid-winter development machine is a 5-year-old Android tablet running Debian in a chroot (it's surprisingly adept and I talk about it here). It's not a fast machine so the build times are a lot slower than they'd be on a more capable workstation. A full build on my tablet including image processing tends to take 5-6 minutes in total (vs 1 min on Netlify).

Due to my slow machine and based on the Netlify build times you can probably presume the timings listed here are about 5x slower than you'd experience on a decent modern workstation.

I also have a couple of specific build jobs defined, one which only recompiles CSS from SASS and another which builds the entire site but doesn't do the image processing. The latter is the job I use the most during development as I don't often need to add new images.

Step-By-Step Build Process

  1. initialised +0ms
    This is the start of the process (t=0) although actually all the source files have been read into memory at this point so a small amount of time has passed.

  2. images processed +3m
    A fair amount of image processing is done here, using metalsmith-sharp (a wrapper for the very fast sharp image manipulation library).

    • A high resolution image of each artwork is included in the source files. A medium-sized copy is created to be used as the main image and a small copy to be used as a thumbnail. The hi-res version is used in the image magnifier.
    • All images within blog posts are duplicated a couple of times at different size and renamed so that they can be used in srcset markup and the browser can choose to download the best size for whatever device the site is being viewed on.
  3. SASS compiled +325ms
    CSS is stored in several separate files so that things like colours, sizes, animations and functions (mixins) can be kept apart. SASS (via the metalsmith-sass plugin) is used to compile them into a single compressed CSS file so as to keep file size and HTTP requests down to a minimum

  4. JSON imported +12ms
    A few bits of useful data (markup metadata and some useful strings) are stored in separate files. Here metalsmith-data is used to bring them into metalsmith so that they can be used while building pages.

  5. files created from JSON +146ms
    Data describing all of the artworks are contained in JSON files. Using the metalsmith-json-to-files plugin, the JSON is parsed and an individual page is created from each item. It's a great plugin and I patched it for my specific case. By default all imported data comes into the created pages wrapped in a data object. This was clashing with another data brought in by a different plugin (metalsmith-data) so my patch allowed me to rename data to whatever I want via front-matter (some simple YAML in template files). In this case I renamed it to itemData.

    I later found that the tags I had in my JSON data for each artwork were inaccessible to other plugins because of being buried inside that outer itemData wrapper, so I created another patch which allowed me to select bits of data to be injected directly into the page without the outer wrapper. This meant that other plugins (eg. the tags object to be used in metalsmith-tags) could access the data and bingo.

  6. markdown converted +276ms
    I use markdown to create blog posts as it's simple, clean and can be written in any text editor. I want to blog more and removing as many barriers as possible means I'm more likely to get stuff written rather than procrastinating. metalsmith-markdown is used here to convert my *.md markdown files into html.

  7. posts slugged and renamed +162ms
    The markdown files have been turned into pages and as part of that process they've been endowed with proper titles. In this step those titles are used to create friendly URL slugs and in turn the files are renamed to match those slugs.

  8. excerpts grabbed +512ms
    metalsmith-excerpts is used to automatically create an excerpt for each post (it basically just grabs the first paragraph tag it finds in the markup).

  9. drafts ignored +4ms
    To create a draft post I can just include draft: true in the file's front matter, then the metalsmith-drafts plugin can tell metalsmith to ignore those files during build.

  10. permalinked POSTS +33ms
    The posts were created with paths such as /word/post-number-one.html. Here a plugin called metalsmith-permalinks is used to change those paths to things like /word/post-number-one/index.html. This means friendlier URLs (/word/post-number-one/) can be used and the server will default to serving the index(.html) page from that folder. /word/post-number-one.html -> /word/post-number-one/index.html -> /word/post-number-one/

  11. collections created +952ms
    It's useful to be able to work with the concept of collections, for example this site has word, art and photo collections. This enables things like easily providing previous/next links for navigation. metalsmith-collections allows collections to be defined based on the contents of chosen subdirectories. For example, all pages inside the 'art' directory are added to collections.art.

  12. metadata boilerplate added to posts +5ms
    Every blog post I wrote required me to add some metadata (layout: 'post.njk') in the front matter to properly integrate it into the site. I didn't want to have to remember to include this every time. The metalsmith-metadata plugin lets me automatically add this (or any other) data to every item in the word collection (ie. blog posts).

  13. tag cloud analysed/created +19ms
    metalsmith-word-cloud is a plugin which analyses all the collections in the site and creates data that can be used to build a tag cloud.

    I decided to try using a tag cloud as the main navigation for the site. For it to work I'm going to restrict the number of tags I use so that it doesn't become too unwieldy. It's an experiment, an indulgance I don't mind trying on my personal site. There's quite a lot of content here and while this unconventional nav might make it a little harder to find a specific artwork, I'm not sure that many visitors will be wanting to exercise that use-case. The tag cloud does lend itself to discoverability of new, related items though, and this is something I want to push.

  14. topic pages created from tags +167ms
    Every blog post and artwork has been tagged during the editing process. Here that info is used to compile a contents page for each tag, listing every item which includes the respective tag. metalsmith-tags is the plugin that does the work.

  15. permalinked TOPICS +59ms
    Permalinks (as described above) are created for the topic pages that were just made. It may seem strange that I'm using the permalinks plugin twice – why not just do it all in one go? The reason for this is that the items listed on the topic pages should link to their permalink URLs, so all of the blog posts and artwork pages need to have been permalinked already in order for those permalinks to exist. But then once the topic pages exist (including the adjusted, permalink URLs) those topic pages themselves should have nice friendly permalink URLs.

  16. dates formatted +171ms
    Dates have been written into the site data in specific formats. Here metalsmith-date-formatter is used to convert those dates into various formats for use in the created pages... for example artwork listings only show the year of creation whereas blog posts show the exact day. This is a purely stylistic choice – the exact date is actually included in the JSON data describing the artworks and I may decide to show it differently at some point. Of course there are various ways in which I could have extracted a year from a date but using a plugin feels more "correct" for metalsmith.

    metalsmith-date-formatter is a very useful plugin but I desired a bit more functionality – I wanted to be able to created various output styles from a single input date. I patched the plugin to add this funtionality.

  17. templates inherited +16s
    This is one of the most crucial steps and brings up a very important concept in static site generation. Templating engines allow bits of pages to be re-used throughout a site – obvious examples are headers and footers. They also allow data to be injected into the templates and processed in various ways using filters (eg. to convert a date from one format to another). Blocks of content can be defined and used or overridden in child pages. The concept of inheritance helps with modularisation and re-use. All very powerful, lovely stuff to work with.

    I'm using Nunjucks, a templating engine from Mozilla which is based on another engine called jinja2. metalsmith-layouts is the plugin which instigates the templating engine. Although I chose Nunjucks, metalsmith-layouts can also be used with other engines such as Handlebars.

  18. markup beautified +8s
    Some would consider this an unneccesary step – running the outputted markup through a prettifier to keep indentation and whitespace nice and neat. metalsmith-beautify does the job. I find genuine benefits in working with clearly formatted code. It's easier to read and debug – sometimes when I'm using my Android dev machine I just view-source in the browser (you can prefix any url with view-source: to do this in Chrome and Brave on Android). I am able to open full Chromium and access developer tools when I need to via Linux Deploy and XSDL server, it works very well but is a little slow on this old tablet. At some points during the development process viewing the markup is good enough.

    I might not have taken this step if it placed any burden on the client or server, but the only cost is borne by me, during the build process... for 8 seconds. Technically a few bytes of whitespace are added to the markup vs a compacted page, but I'm serving this up over the brilliant Netlify and their excellent Content Delivery Network, Iand my site has very little traffic. I really think it's fair to say that it doesn't matter. In this instance, the comforting feeling I get when I view source and see lovely nesting and whitespace is worth it.

  19. RSS feed created +360ms
    metalsmith-feed is used here to automate creation of an RSS feed of all the site's main content. It's another thing which some people find outdated or redundant but there is a niche of users who love consuming their content via RSS. When accommodating those users is as simple as one-time installation and configuration of a plugin I'm going to do that.

  20. sitemap.xml created +497ms
    It's good to have a sitemap for web crawlers and SEO. Well worth the half a second metalsmith-sitemap adds to the build process.

  21. links checked +17s
    By running the metalsmith-linkcheck plugin, all internal and external links are tested and any failures listed and output into a file.

Hosting with Netlify

The site is hosted on Netlify. They have a fast Content Delivery Network and offer add-on services to help add some dynamism (forms etc) to static sites.

CSS SASS mixins

I ended up creating a couple of useful SASS mixins which I'll write about soon.

Hire-me animation

The OTT 'hire me' animation on the home page is made with pure CSS along with a few SVG images. It got pretty complex so I'll written that up as a separate article soon too.

Conclusion, Pros and Cons

I spent longer on this project than I would on a client site, where I'd usually be looking for the most pragmatic solution. It was an indulgent process which I considered to be on-the-job training as I was learning and developing several new skills.

The static site creation process isn't inherently slow, it's just that I wallowed in the new stuff and explored lots of forks in the path along the way.

Now I have a fairly good understanding of the pros and cons of static site generation - I'm really glad to have added this way of building websites to my reportoire.

Static sites aren't going to suit every project but they are perfect for others - I'm looking forward to building more of them.

Pros

  • Static pages are very fast and cheap to serve because they don't require the server to do any processing.

Cons

  • Dynamic parts of the site (comments, contact page etc) require novel solutions as there's no back-end to speak of (although it's possible to connect to 3rd party servers or services to make these parts work).