Split Tests

It’s also called A/B Test. It’s even called Experiments on Google Analytics. Here is where dreams come true... well... at least the dreams where we see 100% engagement rates with 1000% year-over-year growth. 😃

Here is a quote from A/B Test page, on Wikipedia:

In marketing and business intelligence, A/B testing is a term for a randomized experiment with two variants, A and B, which are the control and variation in the controlled experiment.

In our case, we use A/B testing to test two different variations of a page view on our site.

IxDF is using Google Analytics to set up split tests. It’s fairly easy to set a split test on Google Analytics, and then publish it as well as stop using variations while the test is still running on production.

Tutorial: How to set up a split test on Google Analytics?

1. The first thing to do is of course prepare a variation view for a page we’d like to test. This could be anything. In this tutorial, we will be testing IxDF’s join page. The original page will show 3 different subscription plans: Student, Professional and Design League. The variation page will show 2 different subscription plans: Student and Professional. The goal of this test is to find out if the number of transactions will increase, if we ditch Design League.

This is the original page which can be accessed via /join or /join?v=a:

This is the variation page which can be accessed via /join?v=b:

1. Now, go to Google Analytics and log in. If you don’t have access to Google Analytics, you should consult Mads to give you access.
2. If you are going to test for a goal on Transactions or Revenue, you don’t need to add a goal. [@todo Document how to add a goal.]
3. Click on Experiments under Behavior, on the left-hand side menu.

1. Click on Create Experiment on the page below:

1. Start by filling the experiment contents.

• Name your experiment by describing the change, e.g, Join page without IxDF Design League
• Select objective for the experiment. Objectives are goals. We have defined Transactions and Revenue before. You can use them, or define new goals, as said above.
• If it’s a minor change, then select 100% of traffic to test. If you’re testing something major, you may want to start with a low percentage.
• Turn on email notifications if it’s an important change, i.e, the changes in payment flows.

1. Configure the experiment by providing at least 2 different pages including the original page. You can add more thana one variation if you want. Try to name the pages by describing the changes as below:

1. When you came to the step Setting up your experiment code, you will see something like the screen below. Click on manually insert the code.

1. Now you have the code. You need to put this code only on the original page and not on the variation pages. Place the code between tags

@section('split-tests')
<!-- Google Analytics Content Experiment code -->
<!-- End of Google Analytics Content Experiment code -->
@append

and above the tags

@section('content')
@endsection

1. If you’re not deploying to production instantly, click on Save for Later button on this page to come back and publish the experiment when this code is deployed.