Comparing Conversion Rates is Nonsense

In the e-commerce industry I often see people comparing conversion rate samples between different sites as a way to measure the attractiveness of their store and products. This is problematic because a site’s conversion rate is influences by so many different factors.

Let’s say you have a site where you sell cellphone batteries. Now, two sites - with the same layout, product selection, level of usability, prices, etc. - can easily have very different conversion rates. While the first one converts 0,01% of its visitors, the other one could be converting 3%. All due to a factor such as traffic quality.

The 0,01% conversion rate could be because the majority of the site visitors came from some generic search engine keyword like “cellphone” (these visitors might be looking for reviews, actual cell phones, manuals or games) or they could come from a service like digg.com, lured in by an intelligent headline.

A 3% conversion rate could be achieved for the very same site, if the majority of the site traffic instead came from very site-related keywords such as “buy cellphone battery”, blog posts with reviews of the site, price comparison sites and so on. Returning visitors of course convert great as well.

Focusing your time and energy comparing conversion rates between different sites just isn’t very meaningful since all factors need to be accounted for – and this is typically data you don’t have access to. Without the full context, most numbers are of little value, and this goes for conversion rates too.

Just because it can be measured, doesn’t mean it should be compared.