{"id":5077,"date":"2017-04-18T09:00:48","date_gmt":"2017-04-18T07:00:48","guid":{"rendered":"https:\/\/www.red-orbit.com\/blog\/?p=5077"},"modified":"2017-05-18T11:37:11","modified_gmt":"2017-05-18T09:37:11","slug":"4-analytics-mistakes-you-can-learn-from-without-repeating-them","status":"publish","type":"post","link":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them","title":{"rendered":"4 analytics mistakes you can learn from – without repeating them"},"content":{"rendered":"

Web analytics<\/a> is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just begun. Human nature, the complexity of the web, lack of real data and not enough consideration about what you wish to measure and what these data would tell you, can often cause a wrong interpretation of data.<\/p>\n

People have well-formed ideas about the world and the things making it up; but these ideas are not always correct. The situation is no different when it comes to the web, to data and user behaviour. A combination of assumptions and convictions (which are often wrong) and poor-quality data or analytical methods that were not thought over well enough, quickly brings about completely fictitious results, achievements, findings and trends, and this can lead to wrong decisions.\u00a0<\/p>\n

Anyone engaged in advertising analytics and optimisation can come up with a number of mistakes, surprises and strange things they experienced in their career. Below I present some of mine.<\/p>\n

Mistake #1 \u2013 Too good results of submitted online registrations for a service<\/h3>\n

What was the mistake:<\/strong> this illusion sneaked into the analytics and is not the result of a wrong implementation or interpretation of data. An extraordinary number of submitted online registrations was recorded on a project. The problem was that the online registration wasn\u2019t binding – another offline confirmation step followed, and the service wasn\u2019t available to everyone. So users submitted registrations to \u201ctest\u201d if the service was available to them. Of course, not all of them intended to complete the registration; and on top of that, all were not even able to complete it.<\/p>\n

How did we discover the mistake:<\/strong> honestly? The results were too good to be true. Facebook as a channel usually acts mostly as an awareness tool – it\u2019s good for promotion and to expose the brand, but in most cases (with some exceptions, of course) it doesn\u2019t generate last-click transactions. In this specific case, it brought atypically high numbers every week, so we had a closer look at the matter.<\/p>\n

How was the mistake corrected:<\/strong> we gave the users what they wished and needed: a separate tool to check if the service was available to them. And as expected, we immediately noticed a decrease in the number of registrations for the service. Those that remained counted. The advertising optimisation could continue smoothly, and decisions were made based on real, not fictitious data.<\/p>\n

<\/div>\n
\n
Good results can be fictitious because you didn\u2019t consider how your service will be used.<\/a><\/div>\n

Click To Tweet<\/a><\/p>\n

<\/div>\n<\/div>\n

Mistake #2 – wrong assessment of advertising as successful<\/h3>\n

In my life, I made many analytical and advertising mistakes; this is a major one, and most informative at the same time.<\/p>\n

What was the mistake<\/strong>: our definition of the advertising target was completely wrong. Our task was to generate transactions from new users. All existing users were removed from the advertising, and then the transactions from advertising were measured. Logical.<\/p>\n

We forgot that in almost 100% of cases, several days before the transaction, a registration and order are submitted – this is the nature of the product. Only a few days after the customer submitted the order, the payment is made too. As our advertising was focused on transactions, we of course advertised to those who already bought the product, but didn\u2019t pay for it yet. All revenues we recorded would have happened with or without advertising.<\/p>\n

How we discovered the mistake:<\/strong> my life as advertiser taught me that sometimes it makes sense to export specific transactions from Google Analytics and compare the data with the data from the back-end system (CRM, payment or booking system, etc.) I was shocked to find out that nearly all transactions brought by the advertising were made by customers who were already registered in the system, and not new customers.<\/p>\n

How was the mistake corrected:<\/strong> we ensured that the client’s order placement system sent data to Google Analytics after each order, before the transaction was made. At the same time, we focused all advertising on generating placed orders, while completely omitting optimisation on transactions.<\/p>\n

It makes sense to compare data from Google Analytics and your systems to avoid deviations and mistakes.<\/p>\n

Mistake #3 \u2013 the ranking of products in a category impacts the product conversion rate<\/h3>\n

What was the mistake:<\/strong> we monitored the analytics to see how the product ranking affected the purchase conversion rate. We tested in two categories with the highest number of visitors, as the quickest way to obtain a sufficient sample to make the decision. It indicated that the products ranked between the first and eighth position, were sold best.\u00a0 Based on this test result, we of course newly ranked the products in all categories and ranked those we wished to sell most to the first eight positions. But you probably know already that the sales of these products didn\u2019t increase.<\/p>\n

The trick was that the algorithm of the online shop automatically ranked products so those with the highest discounts were ranking best; but this wasn\u2019t reflected by the test categories when we performed the testing. And of course, the final product ranking was a complete failure.<\/p>\n

How did we discover the mistake:<\/strong>\u00a0 the catastrophic result made us repeat the testing, which indicated that the highest conversion rate moved lower in the category. After some effort and research, we found out that products with the lowest margin have the highest conversion rate; and a low margin means a high discount. This moment followed:<\/p>\n

\"\"<\/p>\n

How was the mistake corrected:<\/strong> all was set back to the initial situation, we patted our shoulder and on that day didn\u2019t open the analytics or Excel again.\u00a0 We also came to terms with the fact that discounts are what is selling, and not everything lies in the user experience – people sometimes wish just the product with the lowest price.<\/p>\n

<\/div>\n
\n
It’s not always about user experience – people sometimes just want the lowest price.<\/a><\/div>\n

Click To Tweet<\/a><\/p>\n

<\/div>\n<\/div>\n

Mistake #4 \u2013 the major discovery of CRO\/UX optimisation<\/h3>\n

Where was the mistake<\/strong>: based on an A\/B test of the purchase process, we determined the winning version of the test, which promised a high increase of the conversion rate. The testing lasted for 14 days, we were too quickly satisfied with the results, and did not consider the statistical significance.<\/p>\n

How did we discover the mistake:<\/strong>\u00a0 very easily. When we installed the winning version of the A\/B test on the entire website, the conversion rate decreased overnight, to a record low of the last 8 months. We checked the A\/B test results again, and found out that the nature of products purchased in the winning version was the cause of a very high deviation from the average, and distorted the test results.<\/p>\n

How was the mistake corrected<\/strong>: It wasn\u2019t. Unfortunately, we gave up on this test. Due to the too low number of visitors we couldn\u2019t perform the test in the short time, and due to the product nature and seasonal and other external influences, we couldn\u2019t extend the testing period. In practice this means that we couldn\u2019t obtain a sufficient number of users in the test to make it statistically significant, so we omitted the testing. Sometimes it is still better to throw in the towel.<\/p>\n

<\/div>\n
\n
When A\/B testing, always consider possible influences from external factors you didn\u2019t foresee!<\/a><\/div>\n

Click To Tweet<\/a><\/p>\n

<\/div>\n<\/div>\n

What is the common factor in all four mistakes? Although very different, they all occurred because of two very basic reasons: when interpreting or collecting data, I made the conclusions or assumptions too fast, based on my past experience, or I didn\u2019t take enough time to think what I was testing and especially which other factors in the behaviour of users could impact my test.<\/p>\n

The analysis of results and online behaviour doesn\u2019t permit quick decisions, assumptions and half-baked tests you didn\u2019t spend enough time on. If your testing will be quick, your analysis of the test will most probably be quick too; both will drastically increase the probability of mistakes in the analytical process.<\/p>\n","protected":false},"excerpt":{"rendered":"

Web analytics is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just begun. Human nature, the complexity of the web, lack of real data and not enough consideration about what you wish to measure and what these data would tell you, can […]<\/p>\n","protected":false},"author":7,"featured_media":5078,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false},"categories":[6],"tags":[11,41,31],"acf":[],"yoast_head":"\n4 analytics mistakes you can learn from - without repeating them<\/title>\n<meta name=\"description\" content=\"Web analytics is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u010crt Podlogar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them\",\"url\":\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them\",\"name\":\"4 analytics mistakes you can learn from - without repeating them\",\"isPartOf\":{\"@id\":\"https:\/\/www.red-orbit.com\/blog\/#website\"},\"datePublished\":\"2017-04-18T07:00:48+00:00\",\"dateModified\":\"2017-05-18T09:37:11+00:00\",\"author\":{\"@id\":\"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/47ed62e14959d60e2ab92ae78822dccc\"},\"description\":\"Web analytics is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just\",\"breadcrumb\":{\"@id\":\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.red-orbit.com\/blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"4 analytics mistakes you can learn from – without repeating them\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.red-orbit.com\/blog\/#website\",\"url\":\"https:\/\/www.red-orbit.com\/blog\/\",\"name\":\"Red Orbit Blog\",\"description\":\"We are pushing the boundaries of digital marketing.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.red-orbit.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/47ed62e14959d60e2ab92ae78822dccc\",\"name\":\"\u010crt Podlogar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f79fa7209a6334b4a377f982acfd075a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f79fa7209a6334b4a377f982acfd075a?s=96&d=mm&r=g\",\"caption\":\"\u010crt Podlogar\"},\"url\":\"https:\/\/www.red-orbit.com\/blog\/author\/crtpodlogar\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"4 analytics mistakes you can learn from - without repeating them","description":"Web analytics is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them","twitter_misc":{"Written by":"\u010crt Podlogar","Estimated reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them","url":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them","name":"4 analytics mistakes you can learn from - without repeating them","isPartOf":{"@id":"https:\/\/www.red-orbit.com\/blog\/#website"},"datePublished":"2017-04-18T07:00:48+00:00","dateModified":"2017-05-18T09:37:11+00:00","author":{"@id":"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/47ed62e14959d60e2ab92ae78822dccc"},"description":"Web analytics is not easy. Once you finish the technical implementation of the analytics tool and begin the data collection, your work has only just","breadcrumb":{"@id":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.red-orbit.com\/blog\/4-analytics-mistakes-you-can-learn-from-without-repeating-them#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.red-orbit.com\/blog"},{"@type":"ListItem","position":2,"name":"4 analytics mistakes you can learn from – without repeating them"}]},{"@type":"WebSite","@id":"https:\/\/www.red-orbit.com\/blog\/#website","url":"https:\/\/www.red-orbit.com\/blog\/","name":"Red Orbit Blog","description":"We are pushing the boundaries of digital marketing.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.red-orbit.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/47ed62e14959d60e2ab92ae78822dccc","name":"\u010crt Podlogar","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/www.red-orbit.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f79fa7209a6334b4a377f982acfd075a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f79fa7209a6334b4a377f982acfd075a?s=96&d=mm&r=g","caption":"\u010crt Podlogar"},"url":"https:\/\/www.red-orbit.com\/blog\/author\/crtpodlogar"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-16.jpg?fit=1140%2C580&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p8AeJs-1jT","jetpack-related-posts":[{"id":5358,"url":"https:\/\/www.red-orbit.com\/blog\/8-valuable-tips-improve-tracking-analytics","url_meta":{"origin":5077,"position":0},"title":"8 valuable tips to improve tracking in analytics","date":"11\/27\/2017","format":false,"excerpt":"Let me tell you a little secret. Our Bo\u0161tjan always says: \u201cWhen we have activities with a high ROI and good results, everything is just fine. When things are not looking good, we joke that it must be analytics\u2019 fault.\u201d #agencylife And yes, sometimes analytics is broken. This is a\u2026","rel":"","context":"In "Google Analytics"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-35.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":5024,"url":"https:\/\/www.red-orbit.com\/blog\/analytics-data-why-should-we-care","url_meta":{"origin":5077,"position":1},"title":"Analytics data: why should we care?","date":"03\/20\/2017","format":false,"excerpt":"Robert is a Google Analytics specialist with a lot of experience in everything web related. He is able to give meaningful insights from a data first seen 5 min ago, as well as to play drums in front of 10.000 people. He studied psychology, therefore he over analyses everything. He\u2026","rel":"","context":"In "Conferences"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-10.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6125,"url":"https:\/\/www.red-orbit.com\/blog\/google-analytics-broken-default","url_meta":{"origin":5077,"position":2},"title":"Google Analytics is broken by default","date":"02\/22\/2019","format":false,"excerpt":"The default Google Analytics setup is broken. Yes, it is, it really is. What you should be looking at in terms of Google Analytics needs some tweaking from the start. It\u2019s not only about setting goals or measuring events but also about changing the way things are being measured. Out\u2026","rel":"","context":"In "Google Analytics"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-84.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":5460,"url":"https:\/\/www.red-orbit.com\/blog\/why-not-all-organic-traffic-is-seo-traffic","url_meta":{"origin":5077,"position":3},"title":"Why Not All Organic Traffic Is SEO Traffic","date":"01\/29\/2018","format":false,"excerpt":"Understanding Why SEO Traffic Is Not the Same as Organic Traffic in Analytics All of the traffic coming from search engines (excluding paid traffic) is considered organic. A widespread view considers organic traffic SEO traffic, implying that SEO performance can be evaluated by looking at the organic traffic figures. But\u2026","rel":"","context":"In "Search Engine Optimization"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-42.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":5430,"url":"https:\/\/www.red-orbit.com\/blog\/the-role-of-automation-in-web-analytics","url_meta":{"origin":5077,"position":4},"title":"The Role of Automation in Web Analytics","date":"01\/23\/2018","format":false,"excerpt":"Automating analytics tasks saves my team at\u00a0Jeffalytics\u00a0400 hours a year.\u00a0 While we still do our fair share of heavy lifting and crunching of data, automation of repetitive tasks gives us back 50 days a year in productivity gains. Time that can be put to good use\u00a0taking action\u00a0on the insights we\u2026","rel":"","context":"In "Web Analytics"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-39.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":6177,"url":"https:\/\/www.red-orbit.com\/blog\/when-the-numbers-just-dont-add-up-kinks-and-quirks-in-google-analytics","url_meta":{"origin":5077,"position":5},"title":"When the Numbers Just Don\u2019t Add Up: Kinks and Quirks in Google Analytics","date":"03\/01\/2019","format":false,"excerpt":"My talk (which you can catch at 9:50 am on Friday 15th March) is a deep dive into the promised land of Google Analytics, and the kinks and quirks that come with it. I\u2019ll be covering some common themes and issues that can cause your numbers to be off balance,\u2026","rel":"","context":"In "Google Analytics"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.red-orbit.com\/blog\/wp-content\/uploads\/blog-feautured-image-88.jpg?fit=1140%2C580&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/posts\/5077"}],"collection":[{"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/comments?post=5077"}],"version-history":[{"count":15,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/posts\/5077\/revisions"}],"predecessor-version":[{"id":5099,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/posts\/5077\/revisions\/5099"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/media\/5078"}],"wp:attachment":[{"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/media?parent=5077"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/categories?post=5077"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-orbit.com\/blog\/wp-json\/wp\/v2\/tags?post=5077"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}