One of the larger traffic projects we maintain is MCVersions.net, a website dedicated to providing easy Minecraft version downloads for users around the world. Minecraft is a sandbox video game released by Mojang in 2011, and remains the best-selling video game of all time, with over 180 million copies sold by late 2019.
The site sees large volumes of traffic, primarily from the modding community seeking an easy way to download Java executable files known as "jars" for particular Minecraft server versions, as well as users who simply want to use a specific Minecraft client version without the need for a launcher, or other potentially bloated software. Keeping the site up-to-date is therefore really important for these users, as when a new Minecraft version is released, MCVersions.net sees a huge traffic spike of users wanting to acquiring the jars for this version. Today, MCVersions.net serves over 100,000 unique visitors every month, and well over 1,000,000 total requests. The biggest sources of traffic to the website are the United States, Japan, Brazil, and eastern Europe.
Brief History
The Nodecraft dev team created MCVersions.net in early 2014, after a previous site we used for this info was discontinued. At the time, acquiring these older versions (especially the snapshots, betas, and alphas) was much more difficult than it is today, so MCVersions.net became the only place to really find an archive of this information quickly. To this day, we only ever hotlink downloads directly from Mojang operated servers (powered by Amazon S3), so users know the files they've downloaded are Mojang official and can be trusted.
We've since applied many updates to the project, but thanks to the wayback machine you can find the original site design. It hasn't changed too dramatically since this last update, but some essential improvements have been made, such as responsiveness and improved user experience with mobile devices, as well as introducing a page for users to quickly create a server for their Minecraft version of choice. A capture of the site from January 2020, before the redesign, can be found on the wayback machine also.
A fresh coat of paint
MCVersions hadn't seen a real redesign in almost 6 years, and was starting to show its age. With Mojang ramping up the release schedule for snapshot versions of the game, the page quickly became less and less usable, with over 370 snapshot releases available to date. On top of that, we wanted to make the website more engaging for users, and provide a landing page for each Minecraft version with easy to find client/server download links, as well as the ability to create your own server with our platform, for those who wished to do so. Besides a few colour and padding tweaks, these designs were what we were going to use on the new site.
Technology Stack
Our dev team at Nodecraft has always worked and taken inspiration with some of the latest tech stacks. For small information based website projects like these, we've recently adopted Jamstack as our team's favored way of deploy serverless websites. Much like the design, we wanted to update this project from its 6 year old tech stack.
Out with the old
When we first deployed MCVersions.net, the Nodecraft dev team heavily relied on PHP for our website, NodePanel (our custom game server control panel), and small projects like this. We kept things simple without a framework as it was solely rendering webpages while a CRON job ran hourly to ensure the latest versions of Minecraft were always up to date. We did later add extra DDoS protection and caching with Cloudflare, which helped to greatly improve performance to users who were not geographically near the legacy webservers it was deployed to.
While this setup worked for hundreds of millions requests, it became frustrating to maintain and update the PHP website and the legacy hardware it ran on. With the upcoming redesign, we wanted to find a new hosting solution for the site that didn't involve us maintaining that server hardware and automatically rolled out via CI, like the rest of our modern stacks do.
In with the new
When considering what tech stack to use, it's best to look at the project's requirements. MCVersions.net is a mostly static website that gets updates a few times a month and has very little human generated content. Without many dynamic content changes, this became an ideal project for us to deploy statically via Cloudflare Workers. We decided it would be a perfect fit for Workers Sites, Cloudflare's static site platform.
With Cloudflare's network, we could deploy the site to over 200 edge locations around the world, bringing us to "99% of internet users in the developed world within 100ms". This was our first introduction to the Jamstack, which promises "fast and secure sites and apps delivered by pre-rendering files and serving them directly from a CDN, removing the requirement to manage or run web servers".
Implementation Design
Since we wouldn't be relying on dynamic information in the Cloudflare worker, we needed to compile and build every page that makes up the entire website before publishing changes. The technical challenges involved not only getting the data for each release of Minecraft, but ensuring that MCVersions.net is up to date.
GitHub Actions
As the Nodecraft dev team has grown substantially in the last 6 years, we've sped up everything from the frequency of how often our team commits code into source control, to how frequently we deploy to thousands of game servers around the world. Today we rely on Github for hundreds of our code repos, but also our CI deployments via GitHub Actions. This devops paradigm allows everyone in our team to contribute towards the site design, technology, and other changes with confidence. GitHub Actions has become an invaluable tool in our arsenal, and powers almost all of our deployments today, not just for MCVersions.
The Modern CRON Job
Because we don't have access to the CRON jobs that run on all 200+ Cloudflare servers we needed a more modern way to ensure our system was able to check for updates and deploy changes. Thankfully Github Actions has a build trigger type that replicates CRON jobs called Scheduled events. This means we can trigger CI to rebuild the website every hour and check for updates, but it also means we could potentially be deploying as frequently. Pushing this many changes could cost a lot, and there's no point in re-publishing our site if nothing has changed after all!
Only publishing on changes
Detecting changes proved to be the most challenging part of this small project. To recap, let's go over what our build process has to do:
Gather Minecraft versions information from Mojang's Minecraft Launcher Manifest JSON
Retrieve a changelog and description for each version from the Minecraft Wiki API
Query the internal Nodecraft API for information about any ongoing sales, to update the pricing information
Compare all three points of data for updates into a variable using
set-outputif anything has changed, which we can use in future stepsUsing webpack, and EJS templates render and generate the website HTML
Deploy the site to Cloudflare Workers (more on that later)
There were a few additional edge-cases we wanted to cover here too, so that any push from a user in our team, or a merged PR, would always trigger a deploy even if nothing had changed, but that was relatively easy to do using the available contexts and expressions syntax in Actions.
Attempt #1: Artifacts
Our initial attempt at this was simple; we'd generate the data, store them as artifacts, and then compare them on the next build run to see if anything had changed. Unfortunately, it turns out that artifacts are only accessible within the same build, and aren't available in any subsequent builds. This makes sense, but wasn't something we realized.
There's a lot of discussion to support this feature as per this issue, so this may be a viable solution in the future.
Attempt #2: Caching
Our next attempt involved using the actions/cache Action to essentially cache the generated data, restore it on the next build run, move a few files/folders around and compare them, and if something had changed, trigger a deploy. In theory, this should have worked just fine, however it turns out that this action simply will not run in any events other than push and pull_request. This meant that our build that ran on a schedule, would never be restored with a cache, so there'd be no previous files to compare to.
Attempt #3: Committing back to the repo
Our third and final solution was just to commit back to the repo from within the Action itself, with the generated data, so on the next build run we'd generate new data, and then simply compare it to the existing data found in the repo.
To prevent this from triggering unnecessary new builds, our deploy step only runs under specific conditions, looking something like this:
if: github.ref == 'refs/heads/master' && !contains(github.event.head_commit.message, '[skip ci]')))When committing back to the repo, we just included a [skip ci] declaration in the commit description, so it doesn't trigger a new build. The new data will simply be used on the next schedule run.
This turned out to the be the simplest solution, and works perfectly.
Comparing the generated files
For actually comparing the generated files, GitHub Actions does offer a hashFiles function you can use, however we wanted to compare multiple files in a more readable and maintainable way, so opted to just write a quick Node.js script for this, which runs in the Action. Below is a pseudo-code excerpt from the very simple script we used to test this.
const yesUpdate = function(){
console.log('Data changed. Storing variable to trigger update push.');
console.log('::set-output name=data_changed::yes'); // GitHub Actions format to store variable for later
return process.exit();
};
const noUpdate = function(){
console.log('No data changed.');
console.log('::set-output name=data_changed::no'); // GitHub Actions format to store variable for later
return process.exit();
};
// check if any files are missing (not yet generated, or not in repo)
if(missingFiles){
return yesUpdate();
}
// compare old files vs new files
if(oldFiles !== newFiles){
return yesUpdate();
}
// otherwise the files match and there are no changes
return noUpdate();This is where we echo the magic ::set-output variable to use in the deploy step later. We simply gave this step an id in our workflow, so the variable could be read later.
- name: Compare new vs old build files
id: comparison
run: npm run check-filesPublishing to Cloudflare Workers Sites with Wrangler
The easiest part of this whole process was how we publish the website to Cloudflare Workers Sites. Cloudflare provides a CLI tool called wrangler which takes the grunt work out of these types of deployments. It was super easy performing dev deployments to test your Worker Site on a workers.dev subdomain before it goes live too, using wrangler publish and a workers_dev config directive.
Cloudflare also publish a Wrangler GitHub Action that they describe as <q cite="https://github.com/cloudflare/wrangler-action">Zero-config Cloudflare Workers deployment using Wrangler and GitHub Actions</q>. This was very true for us, and by far the easiest part of our GitHub Actions implementation. Our publish step for the master branch was as simple as the following, simply checking if this was a forceful push (by a human), or if we're in a schedule, and something had changed, via steps.comparison.outputs.data_changed.
- name: Publish
# only publish if a direct `push`/`repository_dispatch`, or if we're on a schedule and mcversions or sale data has changed
uses: cloudflare/wrangler-action@1.1.0
if: github.event_name == 'repository_dispatch' || github.event_name == 'push' || (github.event_name == 'schedule' && steps.comparison.outputs.data_changed == 'yes')
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
environment: 'production'Conclusion
Jamstack serverless deploys are awesome! Conventional mindsets can result in developers often clinging to their webservers, and there's definitely still use-cases where this makes sense, but for static websites with a global audience, being able to remove the maintenance and security concern that is a webserver, serverless just makes sense. Developers can simply write code they want to happen, and have it deploy to "the Internet" - they never have to worry about where or how it gets executed.
It's so easy to deploy static websites in hundreds of places around the world in just seconds, bringing you closer to your users than was ever possible in the past, and with companies like Cloudflare leading the charge with Cloudflare Workers, we can see a potential mind-set shift happening very soon in how we all develop websites. The Jamstack in particular is a very exciting new way to build websites, and we're very excited to watch how it develops over the next few years.