Chris Goward presented a webinar on October 30th 2014 titled The 2 Ways to Use Heatmaps to Light the Fire Under your Website Tests (recorded). The webinar, co-hosted by WiderFunnel and CrazyEgg, sparked some great questions during the Q&A period. We are happy to share them with you.
Q: How do heatmaps compare to qualitative analytics?
A: Yes – heatmaps are a part of qualitative analytics. They are sort of a mixture of qualitative and quantitative. They share the features of quantitative in that you are using high volumes of clicks to get an aggregate view of how people are responding to the pages. It’s qualitative in that it doesn’t directly give you a yes-or-no answer to something.
Q: How do you determine the right amount of traffic to create a valid assessment of a heatmap? Or to make it statistically significant?
A: The thing to understand with click heatmaps is that you are not aiming for statistical significance. A click heatmap is an indicator of how people are acting that leads to hypotheses. It’s an exploratory method. Only in the reductive-conclusive side, when you are talking about A/B tests or multivariate tests, that’s where statistical significance matters. For click heatmaps it actually doesn’t, but the more clicks you get the more of an aggregate view you will get, and potentially, the more important hypotheses you’ll get. As a general rule of thumb, we’re seeing between 2,000-5,000 clicks on a page, is going to give you a good sample that shows you how people in general are responding to the page. You can use fewer clicks as well, it doesn’t matter as long as it gives you an idea because you are ultimately going to test it with an A/B test and that’s where you get statistical significance to find out what really works.
Q: What are hazy click zones?
A: ‘Hazy click zones’ are areas on the page where there aren’t a high concentration of clicks and you see a lot of clicks being spread over different areas of the page – especially where there is meant to be calls-to-action. This is showing you that people aren’t sure where they should click. Hazy click zones happen where there is a lot of diffused clicking, as people are just sort of browsing by braille and clicking on anything they think might be an indicator. If you see this, it means your design isn’t guiding your user clearly to show what they should click on. When a page is designed well from a conversion perspective, there will be no question of what to click. The call-to-action (CTA) will be prominent and attractive, and you will see concentrations of clicks. The click heatmap will show more on the orange and green and white spectrum, rather than just the blue haze that we have seen on that previous example.
Q: Is there a way to track click heatmaps based on returning versus new visitors?
A: Yes – you can do that. Actually, CrazyEgg has that as one of the reports in their confetti report, and you can just select new versus returning. Also, you can look at it using the cloud click heatmap if you were to segment your test by new vs. returning using a testing tool and funnel new vs. returning visitors into different variations/ different pages and tracking them separately using click heatmaps. That is a technical way of doing it, but it could be valuable. It is one example – of many examples – of possible segmentation hypotheses. I have another webinar coming up, probably in the next couple months, talking about segmentation. In the meantime, there is a lot more information on our blog about this. The blog post: 8 Steps to Amazing Website Segmentation Success goes into a lot of detail on how to test segmentation hypotheses and how to get the best results with segmentation. There is a lot of misunderstanding about segmentation today, so this is an important blog post to check out.
Q: Do heatmaps show you what people click on, or what they hover over?
A: The CrazyEgg heatmaps show what people click on. It is technically possible to have other heatmaps that show what people hover over. That’s only if you are using mouse interfaces, it doesn’t work for mobile of course. So ya, it is possible as there are other tools that do that as well. And in fact, heatmaps themselves are just a reporting method, the interaction can be a variety of things. There are also eye tracking heatmaps, which have similar looking reports, but use a smaller samples of visitors and use cameras to track where people are looking. That is just another way of doing the same thing – just a lot more expensive because you have to have individual users sitting in front of the camera.
Q: How are heatmaps impacted by tablets and ipads?
A: They interact in the same way, but of course they use finger presses rather than clicks. People interact differently with an ipad or a tablet, and because they are scrolling with their fingers the patterns will look different. They are still valuable, especially because they show directly where people are clicking just the same as a click heatmap, but there are some nuances to it.
Q: Does CrazyEgg show all pages of the website like Google Analytics?
A: CrazyEgg shows you the pages that you set up in the tool. To collect reporting for a page you have to put the script on the page and also set up that page within CrazyEgg. So reporting is not by default across the website like Google Analytics. Heatmap reporting is more focused, and should be used when you have a question about a particular page, rather than just generally splaying across the website.
Q: I work with a tool that uses Heatmaps and Clickmaps. Can you share a bit more on the value of Clickmaps?
A: Heatmaps are actually a type of clickmap (and vice versa). Heatmaps are a type of report that show concentration of activity, which can be clicks, mouse hovers, eye focal points, or something else. You may be referring to a “confetti” type of clickmap report that shows the individual clicks rather than the heatmap cloudy view. The confetti view allows you to see concentrations of individual segments with different coloured clicks. I explain the different type of clickmap reports in the first few minutes of this recorded webinar.
Q: What would be your take-away from a test version that had better scroll data, but worse results on the main CTA?
A: There could be several conclusions, depending on the context. Perhaps there’s too much content on the page that’s causing Distraction. Or, more CTA opportunities are needed further down the page. But, the reason they’re scrolling depends on the content. You can learn from where they pause and click on the page too. Most importantly, though, is which version has the highest total conversion rate.