How A+E Boosted Viewer Engagement One A/B Test at a Time

Insight/Action/Outcome: A+E Networks used Amplitude to revamp its streaming content strategy with Amplitude. Amplitude enabled A+E to take a smart, data-driven approach. The company continues to use Amplitude Experiment to dive into customer behavior. By testing everything from the placement of its “Continue watching” list to the buttons in the navigation bar, A+E makes product updates and sales offers it knows will boost viewer engagement and shore up retention, including one deal that outperformed another option by 250x.

You’ve probably heard the saying, “If you love your job, you will never work a day in your life.” That’s me in a nutshell! I’ve managed to blend my two big passions—technology and TV—into a career I adore. Like any true New Yorker, I value freedom and variety: the freedom to choose while having many options. That's exactly what A+E Networks is all about.

We’ve been around for almost 40 years, starting as a premium cable channel and growing into a multinational media company with multiple brands. Instead of sticking to subscription-based streaming services, we create, develop, and share a wide range of entertaining content across mobile, web, and TV platforms. We aim to tell meaningful stories to the broadest audiences possible, and we rely on data to make it happen.

Making the most of our data

As a Digital Product Manager, I collaborate with the Engineering, Marketing, and UX design teams to strategize our subscription streaming across platforms. We already had Amplitude but weren't making the most of its capabilities. Data came from multiple sources, and different teams had different naming conventions for data and different ways of organizing it. There was no transparency, so sharing information and making decisions together was tough. We even tried another testing platform but found ourselves stumbling in the dark, unsure of the optimal processes that would lead to more insightful outcomes.

Since we were really trying to double down on a data-driven decision-making model, we wanted to make full use of the data we collected through Amplitude. And that’s what data is for right? To shed light on how we can make our products better, increase engagement, and learn how people are interacting with our apps across websites, TV, and mobile devices. So, I started to think about how we could really optimize our use of Amplitude.

I started by re-organizing our classification system to make it easy to find, understand, and use our data. I also trained colleagues on how to use the tool. We asked such questions as, “What should we label the navigation bar,” “Why do some visitors leave without watching a video,” and “How can we get new users interested in older content”. With Experiment, we can test our hypotheses based on user experience, competitor analysis, intuition, or viewer preferences. With Amplitude Analytics, we can gather data and drill into it to find the necessary answers.

Experimenting for better engagement

Have you ever been hooked on a show you can’t wait to jump back into? Would you rather see the last show you watched on the top of your homepage or scroll through eye-catching promotions before locating it?

We noticed our competitors doing the latter and wondered if we should follow suit to boost our viewer engagement as well. So we tested the concept using two different setups with Experiment: one with the "Continue Watching" feature placed up top and another with it appearing further down past other show previews.

As we suspected, our viewers prefer having the "Continue Watching" section at the top. We ran this test on 3 of our brands, and we saw a 16 percent decrease in clicks when we moved the row down. Without data, that decision would’ve been a shot in the dark.

In another case, we wanted to examine whether having episode highlights on the home screen was attracting viewers. We were grouping episodes into a "best of" series and wondered if we should promote individual episodes so heavily. What if viewers wanted to watch a series from the beginning? Would our approach turn them off?

To test this, we set up two lists. The first showed an overview of TV series, movies, or specials; the second featured individual episodes, enabling the viewer to select one based on popularity and recommendations from their personal algorithm.

We learned that people engaged better with the displayed content when they saw the overview first, even if it took a few more clicks to get to the first episode. Since then, we’ve revamped our design to make viewing experiences smoother from the get-go.

Digging into data also helps us improve how we package content. In a recent experiment, we rolled out A+E Crime Central, an app based on the History Vault app. History Vault’s top navigation bar features a series of documentaries selected to appeal to our target demographic. But we weren’t sure if the term “documentaries” was appropriate for Crime Central viewers. Would “specials” be a better choice?

To find out, we tested "documentaries" against "specials." The results were not what we expected: “documentaries” increased engagement by 14%. That's what’s so magical about data—it keeps you on your toes. You can always tweak and test to better match what your audience wants—and their needs can change faster than you think!

Dialing into audience preferences

One of my favorite things about Amplitude is that it’s a treasure trove of opportunities for improvement. What's even cooler is how statistically significant the data is. Unlike face-to-face interviews or phone calls, data analytics lets us gather feedback from thousands of viewers super-fast.

Think of Amplitude as a direct line to your audience. Even though you’re not chatting face-to-face, you can use the data you gather to understand what hooks your audience and what makes it stick around. It's about the stories the numbers tell us, imbuing the data with meaning. What are people watching? Where and how are they watching it? Figuring this out helped us make spot-on decisions for our viewers and tailor our content to fit them.

With Amplitude, we can test scenarios that impact our bottom line. For example, we had a bundle deal where subscribers could access all three service subscriptions at a lower price than purchasing individual subscriptions. We wanted to find an effective way to retain customers at the end of their subscription period.

We faced two scenarios: offer customers the chance to downgrade to a single subscription or stay on the bundled subscription at a discounted rate of 50% for the next three months.

Turns out, it was a clear choice for customers: the 50% off bundle outperformed the downgrade option by 250 times! The bundled offer is now our primary retention play.

Charting a course with data

All these stories start with data and end with more viewers tuning in to the shows they enjoy most. No one wants to waste time scrolling for a movie or show suggestion. We've all been there, frustrated by the lack of options. My job is to zap away that frustration and make finding engaging content a breeze.

I set the stage internally, and then our Product, UX, and Editorial teams dive into reports and metrics. We use Amplitude to track all the services and products we launch and how our customers react to them. Shared dashboards showing engagement levels make it easy for our team to get information at a glance.

Think of getting into data analytics like you’re gearing up for a road trip. You gas up, check your car, and plan your route for a hassle-free journey. Similarly, in data analytics, you start with good data for good results. By focusing on tracking things from the start, you'll be on the road to success, regardless of your competitors’ actions.

Without Amplitude, we might reach conclusions—and they might even be correct conclusions—but getting to them would be a long, bumpy road. That’s why we’re happy to have the support of great people like Derrick Johnson from Amplitude’s Customer Success team, who is always ready to suggest new features or solve queries almost instantly.