When your team is trying to build something that doesn’t currently exist, the development process is full of moments of uncertainty.
These are times where you don’t just not know what to build, you don’t even know how you would build it. These might be moments where you don’t really know what your real problem is—just that you have one.
One litmus test for this: if you’re working on your product and you hit a roadblock, and you can’t find a satisfactory answer to your question on Stack Overflow. When you’re building a relatively simple CRUD app, the project planning process doesn’t have to be overly sophisticated.
Do some user research. Make some mockups. Define your requirements. Hand it off to your developers and design team, and let them execute on it. Ship.
But when you’re building a product with lots of technical or business unknowns—something many startups and product teams are doing—this process breaks down. The methodology that is supposed to make you go faster actually makes you go slower. It stresses and frustrates your team. And it can leave you drastically behind on your deadlines.
The problem occurs when the irrational optimism of project planning meets the inherent uncertainty of building product.
The uncertainty of technical products
The problem with the conventional planning process is that it doesn’t allow you to account for unknown unknowns, and unknown unknowns are what building a complex technical product is all about.
At RankScience, where we’re building a continuous optimization platform and CDN for website SEO, we encounter plenty of technical unknown unknowns—for no more exciting reason than the fact that what we’re doing hasn’t been done.
Over the last year, we’ve developed a system for planning that allows us to get stuff done while avoiding all the pitfalls of conventional project planning.
The main component of the system is three techniques that come from Agile/Scrum:
- Research: Studying the problem and trying to understand the different ways you could solve it, thinking of pros and cons
- Spike: A basic throwaway implementation, not meant to be kept, but to help you learn about the problem space
- Tracer Bullet: A basic narrow implementation designed to see whether or not your idea will work in reality, but not meant as a throwaway
We run these three techniques in cycles depending on what stage of uncertainty we’re at with a particular problem.
This methodology helps us get work done while overcoming the pitfalls of conventional project planning:
- We can move fast: But not so fast that we bury ourselves in technical debt.
- We can make adjustments on the go: Rather than being beholden to a sprint date.
- We can learn iteratively: Rather than relying on the team to know everything about the problem space beforehand.
If we’re very uncertain—we have no idea what to do about a problem—we do research. Once the research we’ve done has revealed a possible solution, we use a spike to learn more about the domain. Lastly, we’ll do a tracer bullet to see whether our solution will work as a long-term piece of our architecture. If that’s successful, we move on. If not, we can go back to Square One on that problem.
Each step gives feedback both to us and to the next step in the process. The constant feedback loops that this system gives us may be the most important reason that it’s useful.
When you just have a plan and try to execute on that plan from the beginning, you can easily just end up toiling in silence on it. Deviate from the plan, and the whole thing can fall apart.
The idea behind this system is acknowledging that the system will fall apart—you will need a Plan B, and maybe even a Plan C—and that you should optimize for that.
1. Research
The research stage is for when:
- you know you have a problem
- you don’t know how to fix it
It doesn’t matter if you’re at the beginning of a total architecture rebuild or just trying to add a new button to your site.
When you have no idea how to fix your problem, you shouldn’t even bother coding. Instead, you should do your research—develop a list of different options, and then create pros and cons for each one.
These pros and cons should be based on technical factors like:
- How difficult the implementation would be
- How expensive it would be
- How much time it would take
- What future design decisions are locked in by this choice
- How easy to use the result will be
The factors you weigh the most will likely be unique to your team. At RankScience, right now, the three main factors we’re working to optimize for are:
- Our operational burden
- Our bus count
- Our team velocity
We’re entering the research phase of every uncertain decision thinking about how we can reduce our operational burden, increase our bus count, and improve our team velocity.
One common outcome of research is documentation—in this case, a slide deck.[/caption]
Research should be time-boxed as an activity (so that you don’t spend an endless amount of time reviewing different options) and it should also have a concrete output that will mark the end of the research phase. That will usually, but not always, be a decision about which possible solution to pursue. It’s also possible that your research cycle will reveal that you shouldn’t try to solve the problem you were studying at all, or that you would be better off hiring someone from the outside to look at it.
2. Spike
A spike is fundamentally for situations where:
- you know you have a problem
- you have some idea of how to fix it
The spike is a scrappy, throwaway solution to a problem. It sometimes gets flak as a waste of time, but a good spike can teach you things about your problem that you could never learn by jumping straight into building.
A spike is work you throw away when you’re done—but used correctly, it can help you learn a lot about the problem you’re trying to solve.
The spike itself is not going to help you build your final product any faster, but the lessons you acquire from watching people and reality interact with it will.
A good example is the paper prototype.
Mocking up a paper prototype of your idea, for example, might seem pointless in an age where you can rapidly create mockups in Sketch and export them to any format you want—but human user testers can work and play with paper. They can get a tactile sense of your product. They can make suggestions much more easily.
In conventional project planning, your minimum viable solution to a problem is how you begin actually solving that problem. In a spike, your minimum viable solution to a problem is what helps you understand what that problem really is.
3. Tracer bullet
The tracer bullet is a special kind of bullet that emits light as it travels through the air. In product development, the tracer bullet is a technique for situations where:
- you know you have a problem
- you know how to fix it, but don’t know how long it’ll take
The tracer bullet was born as a software development idea in _The Pragmatic Programmer _by Andrew Hunt and David Thomas. It is, at its simplest, an implementation of your solution that is: - very small in scope
- running in a production environment
- kept, not thrown away
Imagine rigging up a system so people can subscribe to your company’s email newsletter from your homepage. Your spike for that task might be a paper prototype. Your tracer bullet might be a text field on your website and a button—a button which you can verify:
- successfully receives the text from the field as input
- sends that text to Segment
- goes into Customer.io from Segment
- enters your newsletter’s recipients list
You might not be validating the email addresses people enter. There might be any number of improvements to make before you consider the project finalized. But if that flow works (and in a production environment) then you know that the flow will work when you put the finishing touches on.
The tracer bullet isn’t a finished product, but it does produce more finished product than a spike, which is almost always “thrown away” after you’ve learned everything you can from it.
A tracer bullet is there to prove out the methodology of your solution—which means it should encompass a broad enough scope that you can generalize from it, yet not so broad that you expend too much time working on it.
The research-spike-tracer bullet cycle
When you’re working with uncertainty in the product development process, you need to adjust how you think about “moving fast.”
Moving fast no longer becomes about getting a completed plan together as quickly as possible, because there is no realistic expectation that a complete plan is even possible.
When you’re working with products with extreme uncertainty or technical risks, you should use research, spikes, and tracer bullets to cut iteratively through that uncertainty—and get little glimmers of answers along the way.
