Why We Decided to Cut Scope and Measure More

In 2010, Facebook identified a problem: Users were struggling to upload photos. The product team decided to roll up their sleeves and dig into what was going on.

They chose to tackle the issue by conducting a waterfall analysis to look into the data at each step in the photo-upload flow. To add photos, their user flow looked something like this: Navigate to the photo-upload dashboard; press a button to launch an operating system (OS) dialogue that allows photos to be uploaded from your computer; select the photos; press upload; and then wait for the photos to compress and upload.

The process was messy and heavily relied on whatever OS users were working with. And it turns out customers weren’t really getting it. Here’s what Facebook’s waterfall analysis showed:

• Only 87% of users reached the dashboard to select and upload photos.
• 57% of users managed to select photos.
• 52% uploaded photos.
• 48% were actually successful in their upload—4% abandoned the upload as a result of issues such as bugs and loading times.

On top of it all, Facebook realized that as many as 85% of users were only uploading a single photo. This data helped Facebook identify a significant product flaw: Adding multiple photos was not intuitive. So they changed the flow to display a user-facing upload tip that described how to add multiple photos at once. In two months, the number of users adding one or more photos per session went from 15% to 60%, and the photos per attempt jumped from three to eleven.

Facebook’s story resonates strongly here at Amplitude. To create better product, we adopted the practice of constantly asking ourselves and our fellow product team members, “What can I do over the next few days (and/or weeks) that will have the greatest potential impact on the company’s success?” This question caused us to shift our approach to embrace product analytics.

The decision to measure more led us to identify three core tenets in the role of product analytics in the product-development cycle: integrate instrumentation into the development cycle, use metrics to assess the success or failure of our features, and come up with a development process that allows us to learn quickly and adjust midcourse to steer toward success.

Another key takeaway? Measuring too much can be as problematic as not measuring enough. Read more about our shift to product analytics to learn about how micro-sprints coupled with product analytics radically transformed our product-development cycle.