What is Hypothesis Testing?
- Proposition – for example: How many users will pre-order our product?
- Test – e.g., a website with a picture of the product and an accompanying pre-order button.
- Results – Evaluate the number of pre-orders.
- Action – Refine the test (e.g., A/B testing or different market segment) or tweak business model (see pivoting).
Does Hypothesis Testing Apply to Iterative Software Development?
A shortcoming I repeatedly see in Agile software development is what I derisively call bogus prioritization. That is, the business or product owner sets the direction of the development team based on seat-of-the-pants guesses of what’s valuable, rather than on something more verifiable.
I propose that we – the iterative software development community – think of each release of new features as a business proposition, much like a startup company. Part of the Lean Startup philosophy is continuous testing and refinement as you converge to a better business model. Can we do the same with each software release?
We might ask,
- What is a hypothesis and how best to test it with real customers?
- That is, how do we make a bold hypotheses like “mCommerce will double eStore revenue” testable?
- What hypotheses are we hoping to get answers to in this release?
- How might prototyping of features be used to test hypotheses?
- Will test-marketing feature sets be off-putting to our loyal customers? To new customers?
Believe those who seek the truth. Doubt those who find it.
~ Derek Sivers