When attempting to run AB tests using Google Website Optimizer (GWO) and Google Analytics (GA), questions often arise. A wide variety of answers can be found on the internet, including work arounds and other hacks, but it’s not always easy to choose the most correct option. This is why I have compiled the following list of questions, with answers and solutions that are, in my opinion, the best method for dealing with such issues.
Question 1: Google Website Optimizer only works with one conversion tag, which represents only one step in the funnel (usually the “Call for Action” on the page that is being tested). How can I measure more than one step in the funnel?
Let’s say this is the funnel your users have to go through:
Google Website Optimizer requires you to choose the one step along the funnel that you wish to measure, such as “Sign Up”, “Email Activation” or “Checkout”. It will then present the landing page that has the best performance rates for this conversion step.
Usually, the “Call for Action” on the landing page is defined as this step. In the above example, this would be the “Sign Up” button or the “Sign Up, Thank You” page. However, as you probably already know, increasing the conversion rate of the first step does not necessarily result in an increase in conversion rates of the later steps.
In my experience, when using Google Analytics, the easiest way to measure the steps along the funnel is to link the test to Google Analytics. One way of doing so is to create a segment for each landing page version, using the “Advanced Segmentation” feature in Google Analytics
However, this solution has two major problems:
- The segments created are based on the landing page, which is good enough – if most of your conversions are performed during one single visit.
But let’s take the following scenario: A user arrives at the landing page, signs up, leaves the page, and returns to the page the following day – via the email activation link. The user then checks out. In such a case, Google Analytics will only present the “Sign Up” goal, as the user landed at the “Activation Link” page on his/her second visit.
You can try to fix this by setting up the segment based on the “Page” dimension, not the “Landing Page” dimension. However, you will then find yourself facing a different problem – you cannot be sure that the tested landing page was in fact the first page that the user saw when he/she first visited your site. - In addition, Google Analytics often uses its Sampling feature, which, in my opinion, does not present accurate, reliable results.
My preferred solution, therefore, is to use Custom Variables on the landing pages. The idea is to store the name and variation of the test within one of the custom variables (the test name will be stored in the custom variable name and the variation in the custom variable value). In the following example, I have used custom variable #3:
[javascript]
pageTracker._setCustomVar(
3,
"<testname>",
"<landing page version>",
1);
[/javascript]
For example, I’m running a test on my homepage and have an additional variation, besides the original page. I will add this variation to the landing page right before the _trackPageview command.
Following is the original code on my homepage:
[javascript]
pageTracker._setCustomVar(
3,
"HomepageTest1",
"original",
1);
pageTracker._trackPageview();
[/javascript]
And this is the code for my variation:
[javascript]
pageTracker._setCustomVar(
3,
"HomepageTest1",
"variation1",
1);
pageTracker._trackPageview();
[/javascript]
This way, when you go to the “Custom Variable” sections, you will see an extra variable on the list:
Clicking on “HomepageTest1” will show you the list of all your landing page variations. Now all you have to do is check each of your goals against each entry.
According to this screenshot, the original landing page performs better for the first and second goals, but variation1 performs much better for goal3 and goal4.
The best aspect of this solution is that you can break the test down into segments and then view the conversion rates for each of the segments. This can be achieved by setting the Custom Variable with all the necessary information. For example, in addition to the landing page variation, I also often store the “Traffic Source”, so that I can later view the test performance rates per traffic source:
Question 2: During the redirection, Google Website Optimizer loses the referrer, resulting in traffic from an organic search or a paid search being counted as “Direct Traffic” in Google Analytics. How can I fix this problem?
The Google Website Optimizer Control Script is the code that you place at the top of the original page, which is responsible for deciding whether to keep the user on the original page, or redirect him/her to a different one. This code does not store the referring site URL before redirecting the user to one of the variations. As a result, when the user arrives at one of the landing pages, Google Analytics cannot determine where the user came from. In this case, the referrer will be defined as the original page, not the actual referring site).
As far as I know, the two best methods for overcoming this issue were published quite a long time ago by Ophir Prusak:
- The first solution is to run Google Analytics prior to running the Google Website Optimizer Control Script. All you have to do is call _initData (which is responsible for setting your utmz cookie based on the referrer).
My only problem with this solution is the fact that the users have to wait until Google Analytics finishes “doing its thing” (mainly loading ga.js) before they are redirected. - The second solution is to store the referrer in a cookie before the Control Script and on the landing page, use the _setReferrerOverride method to manually set the referrer.
Personally, I prefer the second solution. I know it’s a bit messier and requires more code changing, but on the other hand, my users don’t have to wait, which in my opinion, is worth the extra setup work.
When using the second solution, please keep one very important thing in mind:
If your Google Analytics Code has cross domain tracking, you should make sure your Google Website Optimizer Conversion Code also has it. Otherwise, your Google Website Optimizer Conversion Code will override the traffic source settings, and any later pageviews or events sent to Google Analytics will be counted as direct traffic.
For example, if you have the following Google Analytics code on your page:
[javascript]
var pageTracker = _gat._getTracker("UA-XXXXXXXX-X");
pageTracker._setDomainName(".yourdomain.com");
pageTracker._trackPageview();
[/javascript]
Then your Google Website Optimizer code should also have the _setDomainName:
[javascript]
// Google Website Optimizer Tracking Script
var gwoTracker=_gat._getTracker("UA-XXXXXXXX-X");
gwoTracker._setDomainName(".yourdomain.com");
gwoTracker._trackPageview("/YYYYYYYYY/test");
…
// Google Website Optimizer Conversion Script
var gwoTracker=_gat._getTracker("UA-XXXXXXXX-X");
gwoTracker._setDomainName(".yourdomain.com");
gwoTracker._trackPageview("/YYYYYYYYY/goal");
[/javascript]
Also, don’t forget to set the _udn variable before your control script:
[javascript]
<script>
_udn = ".yourdomain.com";
</script>
<!– Google Website Optimizer control script –>
… Contents of the control script …
<!– End of Google Website Optimizer control script –>
[/javascript]
Question 3: I want to only run a test on a specific segment of the traffic (for example, only SEO traffic). How can I achieve this?
There are a few possible solutions. Some involve heavy JavaScript coding on the landing pages, as well as complications while managing and controlling the tests. If you are looking for a long term solution and you plan to implement more than a few tests, I highly suggest avoiding this method and choosing a different direction (unless you are willing to invest time and money in a decent Management System for all your tests).
Instead, I would like to recommend the following two solutions:
- For controlled traffic (not SEO), create dedicated landing pages. If possible, have a set of landing pages for each of your traffic sources. This way, you will be able to test a specific landing page for a specific traffic source.
- If you do not want to direct traffic to different landing pages, you might want to try BTBuckets. This amazing free tool will allow you to create different tests for different target markets, and manage your tests directly from Google Website Optimizer.
Question 4: How can I pass the Google Website Optimizer Validation without first deploying it to my production servers?
You do not really need to put the scripts on your production site in order for it to be validated during the creation of the experiment. In fact, I find it rather strange that Google Website Optimizer actually asks you to first place the code in production, then create the experiment, and only then actually be able to test it to ensure that it works properly. In addition, a number of important options are missing: the ability to “test your experiment” before deploying it to production, and a “reset” button that allows you to erase all data gathered by Google Website Optimizer up till now.
It’s actually quite simple to validate your pages without putting them on your production servers. Just follow the following steps:
- Create a text file and on it paste all the JavaScript codes that you’ve asked to be placed on your original page and on the conversion page.
- Create an additional text file and on it paste the JavaScript code of the variation page – one page for each variation.
- Now click on the “Validation” button and GWO will present a validation error, which will allow you to manually upload the files (original, variation/s and conversion).
One more thing to remember when placing Google Website Optimizer (GWO) on your pages: if you already have Google Analytics (GA) on the original or variation pages, you do not have to copy all the script from GWO, as you already have a call to ga.js. Just make sure the call to ga.js appears before the GWO scripts (and the GA script, if you’re not using the asynchronous version).
*** As I wrote in the beginning of this post, these are the best answers in my opinion. I would of course be more than happy to hear if you can suggest a better or different solution to each of these questions.
Follow @cardiolog 