“Over the past 7 days, 58% of users clicked X, 88% of those users checked in = don’t remove X!”
This is part of a tweet I recently read. I’m sure the guy who tweeted this knows what he is talking about. I, on the other hand, have no idea what X means. This tweet did, however, make me think. Now and again I find myself interpreting my data incorrectly. I know that I should look for a simple explanation but doing so sometimes leads me to the wrong conclusions.
Let’s take the above-mentioned statement and assume the following:
- This is the only data used for making this statement
- X is a button or a similar call-for-action UI element
Now let’s draw a flow, based on the data we know:
This flow is the most obvious and the most straightforward. If it is correct, then the above tweet makes perfect sense and X must not be removed. But what if it is incorrect?
Determining other not-so-simple flows is not trivial, may be time-consuming, and requires deep diving into the site’s usage data. However, if we don’t do so, we may reach incorrect conclusions leading to incorrect decisions.
This post presents a number of such scenarios.
What if X did not lead directly to Check-in?
When I see a multi-step funnel like this I always challenge all conversion steps – remembering that I set up the funnel using my common sense. With a Sign-Up flow, for example, my users have to click on something to get to my Sign-Up page (the first conversion step) and then they have to fill in the form (the second conversion step). All this, just to actually sign up (the third conversion step):
But is it possible that some of the clicks on the Sign Up button are not directly from the landing page? Is it possible that the call for action is the same on all pages but each page has a different conversion rate? Could it be that the button has a minor impact on the conversion and what is really important is the page itself?
What about the second step? Might there be other steps between entering the Sign Up form and achieving a successful sign-up? What if there are links to the Help, Price Plan or FAQ from the form itself? What if you have a promotion code field on the form and some of your users open a new browser and search for a valid promotion code on the Internet or your site?
When we set up a funnel, we usually set up what we know. It’s obviously harder to set up something that we don’t know, but unfortunately, this may sometimes lead us to incorrect conclusions and actions.
To emphasize this point, let’s take a different example and assume X is a page rather than a button. In this case, X is the “Customer Page” and Y is the “Price Plan Page”. According to the information we have, 58% went to the Customer Page (58% clicked on X) and 88% of them eventually checked in.
What if all of the converted users who visited page X visited page Y prior to checking in? What if all of the converted users of your site visited page Y prior to checking in? Is it possible that although X is an important step, it is not as important as we first thought? In that case, Y is much more important than X. Might there be other steps or pages that our visitors go through before (or after) X that actually have a greater impact on the decision to check in?
Some funnel analytic tools provide funnel exploration features that can help answer these questions. In ClickTale, if X is an important step along the funnel, it will automatically be added to your default funnel. In Performable, you have the Assists Report that analyzes your customers and shows the content and events that lead to a conversion. In addition, by using the powerful Lifecycle Report, you will be able to go to specific users and view the complete path that they went through on their way to conversion.
Order-specific steps
What if X is not necessarily prior to Check-in?
In my previous post, I talked about the difficulties in tracking funnels with order-specific steps. Using a tool that does not take the event date into consideration, if X is an event that can occur both before and after Check-In, then relying on X clicks and Check-in events as a funnel will probably lead to the wrong conclusion.
Take for example, the following two flows – they will provide the same results.
These two issues emphasis why you should treat X as an assist, not as a step along the funnel: With most funnel analytic tools you have to define X as a step or a property. Therefore, setting X as a step will result in a funnel with only 51% of the conversions (88% out of 58% = 51%), whereas storing X as a property will provide a partial solution (most of the tools do not allow you to drill down by more than one attribute). I will talk about this in detail in one of my next posts. At present, I have found that the best implementation of assists is in Performable, which has built-in support.
Later conversions
Another important aspect, which could be interpreted incorrectly, is later conversions. Let’s recall the original tweet: “…58% of users clicked X, 88% of those users checked in…”
Now, if there are additional steps along the funnel after Check In (such as Check Out), then only looking at the Check In also depicts an inaccurate picture.
As there is a clear connection between User Expectation and Conversion Rate, then theoretically, if the users know all the following information in advance – then the conversion rate at the next stage will be 100%.
Imagine the conversion rate if I placed the following button on the landing page and ensured that all the users read it before clicking:
I am pretty sure that this button would result in an extremely high conversion rate down the funnel (much higher than with a standard “start now!” button), but with a much lower click-through rate on the button itself. Changing the text on the button will not only affect the immediate conversion rate (i.e., Call for Action on the Landing Page) but probably also ALL later conversions, such as Registration, Verification, Billing Information, Trials and eventually, purchasing.
Is it possible that X has a negative influence on some visitors?
Calculations show that 51% of all users checked in through X (88% out of 58% = 51% – assuming there is one check in per unique user). This means that more than half the people who landed on the site clicked on X to check-in.
Even if this is the most optimal overall conversion rate we managed to get after testing the funnel with numerous variations (different colors, sizes, messages, flows, etc.), it is a good idea to test additional variations by breaking down the users into segments. You may then find that X in a specific segment is not the optimal factor. For example, you might find that the overall conversion rate for the specific segment of non-English speaking users is extremely low (the reason being, perhaps, that the landing page or the check-in process is written in complicated English).
The most basic form of segmenting is by traffic source, as different traffic sources often bring different types of users, who are seeking different types of information or services. A user who googled “free hosting” is probably looking for something different than a user who googled “Microsoft hosting”. Therefore, if X is a button with the text “See Price Plans”, then there is a good chance that the “free hosting” user will not convert…
Can Site Performance be improved without improving the conversion rate?
You should also keep in mind the cost of attracting a user, which differs between traffic sources. An SEM user, for example, will cost you more than a user who was referred to your site through a blog post. The “life time value” of your users may also differ between traffic sources. Combining user behavior, user cost, and user value will help you determine which segments are worthwhile improving and investing time and money in, with the aim of improving conversion rates. There is a real possibility that a 20% increase in your overall conversion rate could actually have a negative effect on your revenues, while a 5% improvement in the SEO traffic could have a positive effect on your revenues. It’s definitely worthwhile making the effort to segment correctly and effectively.
Can removing/replacing the X actually improve the conversion rate?
In short, yes! You should always look for ways to increase conversion. You will never be able to convert 100% of your users but a significant increase is definitely worthwhile.
If the user has to go through page X in order to convert, consider removing this page and exposing its content on the landing page. This simple method of deleting one stage along the flow could quite probably result in an improved conversion rate. If X is the Call for Action, try other variations: change its color, position, size, or context.
When you finish optimizing your funnel, segment your traffic and optimize each segment separately. When you are done, start again! Maintaining your funnels should be an integral part of your optimization efforts.
Follow @cardiolog 