In page A/B testing

Question

I looked all over google and the internet as to performing In Page A/B testing.

What I am trying to do is perform A/B testing on a single page, but that page's content varies on the referring url, performed through <?php include

Say if you come from Google, it displays, 'Hey, are you new here?!', or if you come from another page on our website it will display 'Let's get you started'. The goal is then to see which page has longer visit duration.

Does anyone know of how to do this through Google analytics/Optimizely or any other analytics plugin?

Answer 1

Ryan,

I believe this shouldn't be too difficult to do... just depends on the tools you use :-).

Personally, I can recommend Visual Website Optimizer, which allows you to segment a running test to specific segment based on various conditions. Referring URL is one of them (see screen below).

However, you can then use only one variation of the page, so if you have more segments that you would like to test, you would need to:

1. Duplicate the test itself,
2. Change the copy in the variation,
3. Set up the segment rules according to your needs,
4. Follow this procedure with every segment :)

I have done this, but can't say it had much impact. It was too much work and I personally prefer segmenting based on customer data (new/existing customer etc.), where you can notice much more impact and it's also then "easier" to report since the differences are quite noticeable.

Hope this helps!

Answer 2

You should be able to set this up in any modern A/B testing tool on the market. Here’s how to do it in Optimizely:

• Create a new experiment and go to Options and Targeting. Select the page that the experiment should run on and select the referrers it should run for in Visitor Conditions:

  

• Make sure that the messaging isn’t displayed for the the Original / Control / A in the experiment.

• For the Variation / A, use the visual editor to add an element with the messaging or select an existing element on the page to change it’s text. You can also write Javascript code to insert the element (via ‘Edit Code’).

• If you want to display different messages for different referrers, click on the the ‘Edit Code’ ribbon in Optimizely and wrap the Javascript in if clauses for each referrer (and create a backup message), like so:

  if (document.referrer.match(/^https?:\/\/([^\/]+\.)?reddit\.com(\/|$)/i)) {
    $('#welcome-message').text('Hi redditor!');
  } if (document.referrer.match(/^https?:\/\/([^\/]+\.)?huffingtonpost\.com(\/|$)/i)) {
    $('#welcome-message').text('Hi Huffington Post reader!');
  } else {
    $('#welcome-message').text('Hi! I’m a backup message, just in case !');
  }
  

• Select Options and Analytics Integration. Enable Google Analytics.

• Start the experiment. Within a few minutes you should see the first results in Optimizely. In a few hours, results will be available in Google Analytics, where you can drill down to see how things like visit duriation, pageview per visit and bounce rate changed based on custom segments (that’s how the variations are displayed in Google Analytics).