独家优惠奖金 100% 高达 1 BTC + 180 免费旋转

Bringing your cohort analysis in Google Analytics to another level

Chapter 2 - A Beginner’s Guide to Cohort Analysis

Last week, we started a 3-part series on cohort analysis:

The first part of this series primarily focused on the concepts of cohort analysis. It introduced to you “anchors” that you should pay attention to when conducting any sort of cohort analysis on your web traffic or other forms of data.

While geared towards a digital analytics audience, we intentionally kept things generalized and didn’t show you SPECIFICALLY how to use Google Analytics to conduct cohort analysis on your website — that’s reserved for this article.

Since we have already published an article before introducing the Google Analytics cohort analysis interface (see below), this article is going a little more in-depth than usual to focus primarily on how you can use the platform in practice.

First, we are going to start with examining in detail a few of the options you can choose when conducting cohort analysis (cohort size and metrics), then we are going to introduce to you two primary use cases of cohort analysis in your business and offer you a brief playbook on how you can conduct those analyses.

Without further ado, let’s begin.

The first options we are going to address is the cohort size of your cohort analysis — which just so happens to be one of the three key anchors we mentioned in our last article.

Google Analytics allows you to define your cohorts by day, week, or month, and defaults to the “day” option.

Based on which cohort size you have selected, your date range options will also change to up to 30 days if you select day, 12 weeks if you select week, and 3 months if you select month.

As Google Analytics users, all of us probably have had the experience of visiting the cohort report, flipping through different cohort sizes, realizing that the data from those options is not drastically different from each other, and then later deciding just go with the default option (day) for our analyses.

However, by doing that, we missed so many juicy insights from the cohort analysis report, and risk coming up with bad insights based on the noises in our data instead of the signals.

So let’s walk through each of the options carefully and see what they really mean, and how to use different options for our analyses to get different insights.

Let’s start with days, which is probably the most common option we do our analyses on (being the default option).

If you selected the day option, you will be able to see cohort data up to 12 days after the day of the initial visit, depending on what day the cohort starts (this applies even if you set the date range to be beyond 12 days, such as 30 days).

For example, if you are trying to measure the cohort behavior of a group of users that visit your site on July 1st, you will be able to see how many of them returned for a visit all the way until July 13th, 12 days after their first visit.

In general, I would only recommend using “day” as a cohort option if your cohort size is consistently above 100 users for all days.

This is because, by selecting the day option, you are slicing your data down to a very small sample size, which is subject to a huge statistical swing if that sample size is too little.

This statistical swing will cause your data to go up and down without apparent reason, and might lead to incorrect attribution if you are not careful enough.

Furthermore, you should only use the “day” option with a specific goal in mind, such as measuring the short-term effectiveness of a specific campaign.

This is because your user data, even if you have a reasonable sample size to avoid intrinsic statistical volatility, is subject to so many external change factors such as the day of the week, changes in multiple different traffic sources, etc.

While it is very exciting to go full detective mode and try to figure out all of the deciding factors that can cause one cohort to retain a lot better than the other, going in without a solid objective will leave you with hardly any insights that you can generalize and take actions upon.

It is also strongly recommended to pair the day option with some sort of segments, so you can remove a lot of external factors such as channels you don’t care about, unengaged visits, and so on.

Now let’s talk about the week option, which is the option that I strongly recommend using as the default option of your cohort analysis if you want to a general understanding of how you are retaining users on your website.

If you select the week option, similar to day you can see your cohort data up to 12 weeks after the initial cohort, depending on when the initial cohort was, and you can get data about your cohorts up to 12 weeks prior.

The reason I suggest using this as the default option is because “week” gives you a lot more data to work with, and can also ward you against many external noises, such as weekly seasonality (which doesn’t exist if you analyze your data by week).

The below graph illustrates a comparison between the week and day option when it comes to user retention. And as you can see, weekly data fluctuates a lot less than daily data.

However, the week option is not perfect, and one thing it falls really short on is measuring the immediate impact of your advertising campaigns or promotions.

Every time a customer visits your website, you have a certain window (usually a couple of days) to convince your customers to buy your product or submit a lead before they forget about your company completely, and that window is usually shorter than a week.

Therefore, when comes to analyzing the conversion window of your customers for a specific campaign, the day option becomes a lot better than the week option, simply due to its recency (more on this later).

The “month” option measures your user retention in the long term, and is therefore much more useful for the part of your website that requires long-term retention, such as a blog.

Sadly, Google Analytics only lets you go back three month (at most) in your analyses.

Given that the last row/columns should be discard for analyses due to incomplete data collection, you only have access to 6 numbers when analyzing your cohort by month, making the analysis a lot less impactful.

Now let’s talk about the metrics you can use for a cohort analysis.

Google Analytics offers you 14 metrics to use for your cohort analysis, and defaults to percentage retention, which describes the percentage of users that come back at each of the later cohorts.

While 14 metrics might seem daunting, they are really just different ways to measure three key user experience questions. Let’s look at each in detail.

In Google Analytics, you can answer this question in three levels:

Users and User Retentions — How many users, as individuals, are coming back to our website? User retention is merely a percentage representation of users.

Sessions — How many sessions are visited by users after the initial cohort time?

Pageviews — How many pages are visited by users after the initial cohort time?

Out of all three, User Retention is probably the “purest” answer to the question, but sessions and pageviews can also give you a slight idea of how engaged those users are in the future — pick whichever one you like.

The questions can be answered in two ways: how frequent do users visit during each of the future time periods, and how engaged are users during each of their visits.

Sessions Per User — this metric is the direct answer to the first question. It shows you how frequently your users come back in one specific future period.

Pageviews Per User — this metric is the direct answer to the second question.

Session Duration and Session Duration Per User — Both of these metrics serve as an answer to both questions. The “per user” option is considered better as it is less impacted by the amount of users visiting in total.

Overall, session duration per user is recommended as the metric to measure user engagement within each future period as it kills two birds with one stone.

However, feel free to use a combination of pageview per user and session per user as those metrics are perhaps more familiar to your intended audiences.

Now onto the ultimate question that is relevant to everyone running a business out there.There are so many perspectives to look at when trying to answer this question, and my honest suggestion is just to pick one that works for you and your company.

Transactions and Transactions Per User — Just at it is stated, the “per user” option is recommended to remove the effect of user fluctuations in each of the future periods.

Revenues and Revenues Per User — Just at it is stated. The “per user” option is recommended to remove effect of user fluctuations in each of the future periods.

Goal Completions and Goal Completions Per User — Notice that this is the completion of ALL of your goals, which is very sad for me since it is a vanity metric. I do NOT recommend using this metric as your goal could vary from the ultimate conversion, to engagement conversion, to other usages. I certainly hope they enable tracking of completion of separate goals soon.

Overall, you can answer this question very well if you are an ecommerce company, since you have two options to choose from.

But if you are a non-ecommerce company, you are out of luck as the Goal Completions metric is a really vague proxy of the information that you are looking for, I would still recommend using it but with a cup of salt (a grain is too little).

Now we have arrived at the playbook section of this article, in which I am going to walk you through two use cases of cohort analysis so you can get started analyzing right away.

The first case we have here is analyzing the performance of your advertising campaign.

Before we start doing cohort analysis, you need to have the following items configured/prepared in order for our analyses to achieve maximum accuracy:

With those two steps prepared, we can now go into the Cohort Analysis report in Google Analytics to start our analyses.

Since we are using the Google Merchandise Store for our illustrations and I do not have access to their marketing plans specifically, we are going to cover most of the steps with plain text.

If your user journey is significantly larger than 12 days, you might need to use both the day cohort size and the week cohort size for your analysis (but the principle is the same), else you only need the first one.

Then, you want to adjust the date range so the first day of your campaign is visible as a row in your cohort table.

If the start of your campaign is over 30 days prior to the present date, select the last day you can reach in which the impression stage of your campaign is still active.

If even that is not possible, you cannot do this analysis.

Now, what you need to do is align your user experience plan with each row of the cohort analysis.

For example, if your campaign starts on July 1st, you will want to first look at data at Day 0 on the July 1st cohort, which is an illustration of how many users/session/transactions occurred on that specific day.

Then, starting with that row, you want to average the metric from that point up for all columns (as illustrated below). We cannot use Google Analytics’ sum here since it includes incomplete data and dates in which your campaign hasn’t started.

This data serves as a baseline for your analysis and gives you a general idea of how you are doing if you are not doing anything to improve your user experience.

The important thing to notice here is that right now, over 12 days later, you can no longer do ANYTHING to improve that retention.

The only thing you can do is test alternative content to see if you can improve the experiences of new users that are coming into your website starting TOMORROW — and the baseline we just established can help you do exactly that.

Now, what you need to do is identify potential areas of customer experience that you want to improve on. And then, try different things every couple of days in attempt to improve your cohort metrics, and then wait to see if your metrics are increasing in different parts of your customer journey.

The key here, especially if you don’t have a lot of users to test with, is to test a few variations slowly. While you might want to get as many variations in as possible, too little data for comparison will have a significantly higher chance of giving you the incorrect signal of what is working and what is not.

In the next few days, you will see your result coming in for those days you ran experiments on. Based on the results, rinse and repeat until you get your desired outcome. :D

Now let’s talk about the second case, which applies mostly to people who are trying to use cohort analysis as a metric-indicator of how well you are retaining your users over a long period of time.

To do this, I would recommend using the week cohort option due to reasons explained earlier in this article.

This analysis is, in fact, significantly easier than the previous one — you just need to look at the cohort table carefully:

To reduce analysis load, i would recommend only analyzing your data through the fourth week after the initial cohort, as data retention data gradually becomes too little to matter in future weeks (depending on your overall traffic size).

What you want to do is to look at each column from top to bottom to identify an overall trend in changes of the cohort metric (retention rate in this case).

When looking through the data, you need to be perfectly aware that many of those ups and downs might be simply due to natural fluctuations in your data or external noises. Therefore, you only should look at the overall trend of the data (I recommend plotting it in Excel or Tableau), unless there is a significant outlier that deserves a look (+/-30% compare with the average) .

In general, if you are not doing anything drastic on your website such as changing some major pages or launching a user retention campaign, this number is very likely to stay constant throughout the weeks.

However, if you perform potential actions that will increase your retention such as redesigning the website or launching a email retention campaign, this analysis will be a very good before/after measure of how effective your efforts are.

If you want to find additional ways to track user retention on your website, I would recommend looking into metrics such as XXDayActiveUser or simply number of returning sessions — those metrics are usually easier, but less sophisticated alternatives compared to the analysis introduced here.

Alright, that concludes everything we are going to cover today about doing cohort analysis in the Google Analytics UI.

Hope the analyses introduced here can provide actual value or inspiration for the analytics at your own company.

Next time, we are going to dive into the technical side of Google Analytics, and explore how we can use the Google Analytics Reporting API to further expand upon the cohort analysis feature, and draw some insights that we are unable to obtain today.

Stay Tuned!

Add a comment

What is your view?

Send

Related posts:

The Almost Unbearable Dichotomy of the Creative Life

This obsession with productivity means that it’s easy to believe there’s something wrong with us when we’re not reliably and mechanistically productive. ​But a productive outcome is just one very…

10 Signs You May be a Psychic

Have you ever experienced instances that are beyond explanation? Do you wonder that you could be psychic? Do you have a super-strong sense of intuition that you have always neglected? Does your gut…

10 Common Mistakes Everyone Makes In Headless Ecommerce

If you are an e-commerce business or even are related to the e-commerce space, you must have come across the word Headless Commerce. With an API-driven decoupled architecture, there is no doubt that…