Observer Analytics Blog

Immersive product analytics. Designed for VR & AR, built for you!

Subscribe Newsletter

Search Blog

VR metrics need to adapt to the state of the market

TL;DR — In these early days of VR, weekly retention is more telling than daily retention.

 

After the release of HTC Vive and Oculus Rift in early 2016, the hype of virtual reality was at an all-time high. What quickly followed was an emergence of skeptics complaining that the lack of content for these devices rendered the hardware useless, and the mainstream utility of VR was still some ways out. Then in 2017, content creators doubled down on VR, releasing more quality content, which span a variety of use cases including gaming, utility, training, and simulations.

Now that the VR landscape has some quality content, it’s a good time to focus on early VR users to understand the value that they’re finding from these experiences. If VR is to become the ubiquitous technology we know it can be, it’s important to understand what’s working in these early days in order to steer the market in the right direction moving forward. A “good VR app” is certainly subjective and simply looking at the number of downloads may seem telling enough. However, to become a “killer experience” for VR, developers must focus less on the vanity metric of downloads and more on engagement and retention metrics; they should strive to build something that sticks.

While it might be common practice to implement analytics into a project, many make the assumption that they can use the same standards built for web and mobile. Yes, there is overlap and some metrics can work in these adjacent industries, but these standards emerged only after millions of users and apps. Today, we need to measure VR content against metrics that are applicable to VR, and not pigeonhole ourselves into analyzing metrics that may or may not be relevant.

 

Stop analyzing daily retention — focus on your weekly retention.

One interesting metric that we’ve looked at over the last few months is retention. Currently, VR is not something that is as easily accessible as a website or mobile app. There’s a lot of hardware required and for the most part, it’s still a tethered experience. Because of this, daily retention metrics aren’t as valid in this nascent market. Unlike mobile and web apps where a user can quickly pull out their phone or tablet at any time of the day regardless of where they are, VR is an activity that is typically confined to your home. Compared to mobile apps, VR apps are seeing fewer weekly uses, but at longer durations. This is why our clients believe that weekly retention metrics are more telling than daily retention: VR has yet to become a daily habit for most users.

When analyzing aggregate data over the last month you can see a clear distinction between daily and weekly retention rates.

 

Virtual Reality User Retention - Analytics

 

Virtual Reality - Weekly Retention

bounded retention, cohort size of over 4500 users

 

As you can see above, if you simply analyze Day retention metrics, VR app performance falls flat rather quickly. Because users generally don’t use VR on a daily basis, daily retention metrics undervalue player consistency. Comparing week four to day thirty retention we see a number that is 5X greater if you look at the week over week instead of the day over day. You don’t want to discount a large percent of users as churned if they are returning on a weekly basis.

Another reason why we suggest looking at weekly retention is because we are seeing playtime spikes occur Friday through Sunday when people have more free time. This aligns well with the current VR customer archetype: the early tech adopter and weekend hobbyist. Our data shows that the number of sessions played per day increases by over 40% on weekends.

 

So where do we stand?

It’s clear that there is a huge behavioral change between current technologies and virtual reality, so let’s not use the same benchmarks to gauge the success of VR. We are at a point where educating others about current market conditions is key to helping realign expectations. The only way to do this is to focus on finding out what makes VR content great and stop comparing it to web and mobile.

 


 

Want to start tracking metrics about your VR application? Reach out to us at hello@observeranalytics.com!

 

Lucas Toohey

CEO & Co-founder @ Observer Analytics