Creating an A/B test for 18 million visitors

How do you test the layout of your homepage when you have an average of 15 million visitors per month?

Online fashion retail platform: selling products from global luxury boutiques and brands.

With the size of their audience they were constantly trying to improve their website. Their internal team were busy optimising the codebase and the designers wanted to try out a new layout. Before updating the website, they wanted to understand whether it would be worth the hassle.

Having worked with us before, they asked us how we would go about running an A/B test without influencing their current website. We knew how their system worked and were keen on finding an effective solution for this 6 week test.

Not wanting to skew the results

When running an A/B it is important that each version works in the same way, otherwise the results may not be accurate. To make sure the users had the same experience between the two versions we used the client's APIs to provide the same functionality as the original layout.

The layout of the new homepage design

A quick turnaround needs a simple solution

The content of the homepage needed to change on a weekly basis. There wasn't enough time to build a Content Management System for this project so we came up with an alternative solution.

Their editors used a Google Spreadsheet to prepare all the content, in all 12 languages, for the different  pages and articles on the website. Not wanting their team to have to learn a new technology for the short lifespan of this project we decided to use their existing processes.

We created a new Google Spreadsheet that was structured to fit the requirements of the new homepage layout being tested. The content of this spreadsheet would be fed into the web app we created and used to populate it.

They couldn't get enough of a good thing

The solution we provided them with was such a success for their editors that they wanted to use it for a few special projects they were planning. We built the new layouts and prepared the spreadsheets. This became the data entry tool for a few months of special projects within one of their marketing and creative teams, leading to some very successful campaigns.

Kind words from the client

wearegoat have worked with me to build multiple executions for our platform. Our projects have ranged from single microsites to a bespoke content management system for enhanced product listing pages. I have always found them to be fantastically diligent in their output, incredibly insightful when planning how to execute projects and hugely responsive when moving forward with builds. They are hands down my go-to team of developers and I haven’t been able to give them a challenge that they haven’t managed to tackle.

Art Editor at leading global fashion retailer