React, Gatsby & PHP WooCommerce

About this project

I have had the pleasure to work on a complex commercial headless eCommerce project using WooCommerce at its core, and React + Gatsby for its front-end.

This is a B2B website where you can purchase in depth analysis reports to help grow your business. These products are digital only, there are no phyiscal copies. If you register a user and log in before checking out, you can store this on your user account. There are also subscriptions for a recurring fee you can have access to all reports.

Technical challenges

Many things on this project had to be made from scratch, simply because there were no ready made solutions at the time. Since making this project, some source plugins and resources for Gatsby, as well as updates in WooCommerce has made this project easier to reproduce. This project was made in early 2020, when Gatsby 3 was new.

Getting data from WordPress/WooCommerce.

There were publicly available source plugins and templates, both for WordPress and WooCommerce. Initially these were experimented with. Ultimately I ended up writing the Gatsby and WordPress APIs myself. The problem with what was readily available was that they did not follow the data structure that we wanted, and did not support Advanced Custom Fields.

Responsive images

WordPress & WooCommerce does not by default contain any support for responsive images. I made the decision to use gatsby-plugin-image which was new at the time. Since we used custom data from a CMS with our own solution for fetching said data I wrote our own solution to integrate with gatsby-plugin-image.

User login

Users need to be able to log in, log out, and their reports are stored on their user accounts. The state of the user is stored in cookies so that a user remains logged in until the cookie expires.

Hosting

Deploys needed to be automated. WordPress and our custom API is always running, but builds are made on command with webhooks upon changes inside WordPres as well as any changes to the GitHub repos.

What I am proud of

We took a challenging project with stride. Solved many challenges along the way. This project came with many firsts.

Significantly cut down build times by implementing cache and finding the root cause of an issue in Gatsby.

I built many critical parts of this application from start to finish and delivered a finished product that the customer was very happy about.

AWS Cloud Hosting

This project is fully hosted on AWS. API and WordPress are both hosted on separate EC2 clusters with an automated build pipeline. The front-end project uses CodeBuild and an S3 bucket for html cache as well as saving images.

CodeBuild

Heavy operations are performed on each build. Images are transformed to multiple sizes with srcset. It is important that the cache persists between builds, otherwise every deploy would take a long time. The cache is stored in an S3 bucket and fetched for every build.

S3 Bucket

An S3 bucket is like a virtual disk. It is used to store and retrieve data. This is used heavily on this project since we are generating different image sizes ourselves. We are not using a CMS or imgix which has this built in.

EC2 clusters

Two separate EC2 clusters are used. One for the WordPress / WooCommerce installation and one for the API. These are always running and are easily scaled to account for higher usage

CI/CD Pipeline

We have automatic pipelines set up. We have one for each application with their own rules and webhooks.

Custom WordPress Plugin

A custom plugin was built for this project. This plugin transforms data from WordPress and formats it in a way that is more suitable for the front-end. This outputs JSON data that is later transformed to GraphQL nodes in Node.js

We make sure the data is consistent. An "Image" object always contains the same information, for example: url, alt, desc, width, height.

Gatsby cache problem

This project was built on Gatsby 3, since then a lot of issues have been solved in the Gatsby core. When making this website cache was a well known problem.

At first we invalidated the cache on each build. This quickly became a big problem when more content was added. Thousands of images were added to WordPress, each image was transformed to different sizes and image formats on every build.

I dug deep into the code of Gatsby and found that the issue was caused by a race condition in the Redux state of the Gatsby build. Gatsby assumed cache from previous builds was available in Redux during the createPages cycle, this was not the case. The build code then decided nothing had changed since the previous build, this caused createPages to be skipped.

The "solution"

This is not my proudest solution, but if it works it works I suppose?

Putting a for loop around our createPages code in gatsby-node causing every page to be built twice. The first time only takes a few seconds since there was a bug causing all builds to be skipped. The second time around correct cache data is available and only pages that have been updated will be built.

This cut our build times down to less than 1/10th of what it would be with invalidating the cache each time.