Is Peloton hiding a retention "smoking gun" in its churn disclosures? Maybe not.

[Joint work with Val Rastorguev]

I had made some positive comments regarding Peloton's unit economics over various social media recently in advance of their upcoming IPO (link), while cautioning that (1) I had not done a detailed analysis of the data yet and (2) I was particularly unsure how the usage of prepaid subscriptions affected Peloton's retention profile. Since then, a number of people have brought to my attention a very interesting set of tweets about Peloton's churn figures (link) that called into question the relevance of the churn figures that Peloton had disclosed, precisely because of those prepaid plans. To better understand the arguments there, Val and I decided to dive in and take a closer look. In this short note I summarize the bearish argument, discuss what we found through our modeling, and provide some further lifetime-related thoughts.

The essence of the bearish argument is that a churn rate is supposed to represent the number of customers lost relative to the number of customers at risk. Peloton defined their churn rate in a very normal way, by dividing the number of customers churned (net of reacquisitions) by the average number of customers at the beginning of each month during the quarter, divided by 3. This is actually a conservative definition relative to other companies -- for growing companies, the number of subscribers at the beginning of the period will always be lower than, say, the average number of customers during the period as a whole. But even though this definition is common, its denominator overstates the number of customers at risk because many subscribers have their "asses glued to their seats" (no pun intended) through their signing up for prepaid plans that lock them into subscriptions for 12, 24, or 39 months. Adjust the denominator and the "adjusted churn rate" goes up. A challenge is that Peloton discontinued the practice of using prepayments in June 2018 -- starting in July 2018, every subscriber must sign up for a month-to-month plan.

No doubt, that argument is sound, and if it were the case that (1) a substantial proportion of Peloton's subscriber base were signing up for 39 month plans, (2) those prepaid customers then churned out en masse when they first had the chance, and (3) the base rate of churn for month-to-month customers was otherwise quite high, Peloton would be in big trouble. This is a testable hypothesis, so we went about testing it using the available data and some acquisition/retention modeling.

The model

I began my modeling genuinely expecting something along these lines to be the case. We posited the following very simple, arguably pessimistic model for customer acquisition and retention:

1. The flow of customers acquired over time can be modeled using a very standard, boring so-called Weibull-Gamma model with just one single covariate for Peloton's marketing spend. This covariate allows for the possibility that there may be an additional "boost" to acquisitions in periods in which Peloton spends more on marketing (and vice versa). We let the data tell us how large and in what direction that association is (although we of course expect it to be positive).
2. Customers have a per-month probability of churning when they are able to, but with a wrinkle. Customers acquired before June 2018 have the ability to select into one of four plans -- they can either go month-to-month, or they can prepay for 12, 24, or 39 months. There is some probability that customers go month-to-month -- we let the data tell us its best guess of what this probability is. If they don't go month to month, we pessimistically assume that 25%, 25%, and 50% of prepayments have a term of 12, 24, and 39 months, respectively (pessimistic because this makes the average prepaid term back end loaded). We had to fix these values because we found that the data was simply too limited to be able to estimate them outright. The first month after the end of a prepaid customer's term, we assume there could be a more substantial rise in churn as people who wanted out earlier now finally have the ability to do so. After that month is over, those customers go back to their usual pattern.
3. Customers acquired after June 2018, in contrast, are all month-to-month customers like those who had started month-to-month before June 2018.

That's it -- no surprises. The retention model is especially simple - it has only 3 parameters. We had to keep the model extremely simple because of how limited the data was. In the spirit of full transparency, and to show that we are not hiding anything, this is a link to the downloadable spreadsheet. You can see the model parameters for yourself (they are the figures in red), in addition to how all the math comes together to give us model-based expectations of acquisitions, losses, net churn rate figures (defined as Peloton did!), and ending subscriber counts.

Validation

Despite the extreme simplicity our model, we achieved a remarkably good fit to the observable data. I've reproduced the plots below:

Are the fits perfect? No. But for a Weibull-Gamma with one covariate acquisition process and a simple one-segment geometric retention process, we were quite content. The two most evident mis-fits were for Q3 2017 and Q4 2018 -- the fact that the actuals spiked in different calendar quarters suggests it is not seasonality. It would be wasteful to attempt to curve fit those points even though we easily could.

Note too that we would in general advise against using a one-segment geometric retention process. It will be pessimistic. My paper with Peter Fader and Bruce Hardie shows how we can allow customers to have different retention rates instead of imposing that they all share the same one. But in the Munger-ian spirit of inversion, this also means that if anything, our conclusions will be conservative.

Results

When we found the best-fitting parameters to all of the observable data, this is what those parameters suggested:

1. In fact, the base retention rate for month-to-month customers is really quite good. No smoking gun there. The reported figures are consistent with our inferred month-to-month monthly churn rate.
2. The model suggests that a large proportion - 90% - of customers were signing up for the month-to-month plan pre-June 2018. This significantly limited the "damage" that could have been caused by prepaid plans.
3. The expected lifetime continues to be very high -- the model-based projection is 15.7 years.

It could very well be that these figures are inaccurate. The data is highly limited, after all, and we would very strongly request that Peloton provide more customer-level disclosure (e.g., retention curves for older acquisition cohorts, broken down by plan, and retention curves for some younger acquisition cohorts to see how things are evolving). With more data we can replace assumptions with observables, and fit a richer model. We would welcome that.

I also believe the average lifetime of a Peloton subscriber will not end up being anywhere close to 15.7 years. One reason why is because for this business there are multiple potential sources of churn (I suspect my colleagues/friends David Schweidel and Mike Braun would agree!). One cause that has been largely unobserved yet is physical wear and tear on the bikes. At some point those bikes will break down. While some customers may simply buy another bike, others may churn. How long will it be before these bikes start experiencing issues like these? I don't know but probably within the next 15 years. Also, this expected lifetime is long enough that physical mortality will be an issue (for lifetimes to average to 15 years, there will have to be a substantial number of customers whose lifetimes are 25+ years). And so on. This 15.7 year figure largely ignores other "causes of churn" that fall outside the range of our data, but should not be ignored. Finally, 15.7 years is extremely high for this category and in general, relative to the hundreds of other companies we have performed modeling on. It is for reasons such as these that I suspect the actual lifetimes may fall closer to 5 to 7 years. Still, Peloton has done a stellar job (and this is holding aside the wonders they've worked for drivers such as CAC).

But the fact that such a simple, conservative model fits so well and implies such relatively benign retention patterns leads us to believe, as scientists, that there may not be some smoking gun hidden behind the S-1 curtain, as some are suggesting. It is not that obvious to us, in any case, based on the available data.