AWS GameDay : eSport for System Administrators

On November 8th, 2022, this author had the honour of participating to a Montreal edition of AWS GameDay. Teamed up with two other participants, our team (404 File Not Found) won first place.

Amazon Web Services (AWS) is a mature and complete cloud platform. Those unfamiliar with AWS should check out this introduction video. To promote the platform. they organize events like AWS Game Day, allowing participants to test their AWS skills in a risk-free environment, possibly discovering new services.

Players are broken up into teams of approximately 4, and given a starting architecture that they will then need to evolve throughout the day in response to internal and external events. There is no one right answer; the path that participants take is up to them, based on the AWS resources at their disposal in pre-packaged AWS accounts.

Could this be the new eSport for System Administrators?

The Mission

The competition is designed to mimic the first days of a system administrator working at Unicorn.Rentals.

Unicorn.Rentals is the world’s leading provider of equine-based mythical creatures, with a focus on responsible and ethical sourcing from their (super)natural habitat.

In the case of this competition, each team must migrate the infrastructure of a recent acquisition: an online store allowing customers to purchase Unicorns.

The online store operates on two servers, a Linux Wordpress/WooCommerce server and a MySQL server. In this scenario, hosting for these two servers will expire at the end of the day and is not renewable. Participants are tasked with moving these servers to a separate AWS account with no service interruptions.

The teams decide how to best solve this problem. However extra points will be awarded to teams who follow Amazon's best practice. For example, the first step is to create a VPC over multiple availability zones with both public and private zone.

Database Migration

Participants were informed that extra points would be awarded for using managed service. To host a managed MySQL database on AWS, there are two options : RDS or Aurora. Following best practices, the ideal choice is a multi-zone Aurora deployment in the private zones.

Handling the database migration is tricky, as the objective is to avoid downtime. The traditional solution would be to use a Master/Slave setup, where the Master is the original database and the Slave is the Aurora instance. Such a setup requires a bit of manual setup, as the original database must be dumped and then loaded into Aurora. AWS automates much of this process using the Database Migration Service (DMS). That said, replication bin logs must be enabled on the original database for the migration to work seamlessly. The tool worked surprisingly well, although some tweaking is required to allow for connectivity between Aurora and the original database.

HTTP Server Migration

The are multiple options for hosting a PHP site on AWS. Participants were encouraged to start with the easiest solution: to migrate the server as-is on Ec2. That said, hosting the site on ECS as a container deployment provides additional scalability.

An intuitive solution would be to create a new Ec2 instance and copy the files over. However, that solution can be quite costly in time if the original owners setup their server in a non-standard way. Amazon allows for migration of a server by making an exact copy of the server using Application Migration Service (AMS). Simply install the agent on the original server and AWS will create an "exact" copy in the selected zone. The tool proved a bit frustrating to use, as the IAM role running the migration must be carefully crafted. However, once copied, the new server launched without problems. Extra points were earned with a load balancer and SSL.

Those with extra time could try to migrate the Wordpress site using ECS. The first challenge was to determine the version of Wordpress used. No admin credentials were provided, so it was easier to find the version in the html source of the original site. An additional concern is how Wordpress stores its theme, plugins and upload content on the file system of the server, in the wp-content directory. When deploying Wordpress to multiple servers, that directory must be shared. With ECS, the easiest solution is to store the files on and EFS and mount those in the containers.

Points and Penalties

For each 3 minutes of availability, points were awarded. If the site was down, or unable to handle the requested traffic in a timely fashion, twice as many points were removed. To keep the competition interesting, load would surge, especially towards the end. In addition, when reaching certain point milestones, different sabotages would be introduced.

Easy points were available to those applying best practices to the account. For example, a reward was given to those enabling CloudTrail, ensuring proper monitoring of the account.

IAM and Security Groups proved to be the greatest obstacle to "404 File Not Found"'s success. However, with 2 hours left on the clock, the team successfully migrated the site. The rest of the time was invested fixing AWS sabotage and earning points through the application of best pratice.

A big thanks to Montreal's Amazon team for organising this event. They provided some great hints navigating the eccentricities of AWS while encouraging teams to push their competitive edge.

New eSport?

Although our local event was only one day, Amazon takes its Game Day every seriously. In the official event, three sets of qualifiers were organised, in three regions. The top 5 teams from each event were invited to participate in the final. Those winners were then invited and flown to the re:Invent 2022 conference in Los Angelas.

The final was streamed live on the Internet.

Although some might argue that the competition lacks the excitement of traditional eSport, there is no denying the competitive element. In addition, the amount of tooling needed to run such a competition is immense. Amazon should be commended for putting so much effort into this creative promotional tool. If other cloud platforms take example, this could be the beginning to a new type of professional eSport.