<img height="1" width="1" src="https://www.facebook.com/tr?id=1483239291704574&amp;ev=PageView &amp;noscript=1">

the hype innovation blog

From I don’t know to Let’s find out: why experiments are so vital for innovation

Posted by Oana-Maria Pop on Jul 10, 2014 12:10:00 PM


We are all familiar with the expressions “too big to fail” and “too big to innovate” – the former alluding to the expectation of bailout should things take sudden, economically dramatic turns, while the latter to complex bureaucracy and short-term share-price orientation conspiring to make big firms miss the next big thing. What about “too big/ too smart/ too confident to experiment?”

As repeatedly stressed by scholars and practitioners alike, experiments (yes, thought experiments too – read more about them in my next post on corporate foresight) provide invaluable learning. It is therefore safe to say that organizations, large and small, refusing or reckoning it unnecessary to fund this exercise are not doing themselves a favour. In fact, they run the serious risk of losing ground to those that invest in experiments and make testing part of their culture: the enlightened ones, as HBR Professor Stefan H. Thomke would surely agree. We will get back to his work in a moment. Before we do, let’s bring back the question we mentioned a moment ago:

Why do companies avoid experimenting when all the signs suggest that they should not?

One answer is for fear of being (mis)judged. As Stephen Dubner and Steve Levitt note in their new book Think Like a Freak, the hardest three words in the English language are “I don’t know”. Failure, they argue, is hard to talk about because it signals weakness. Moreover, as the desire to please one’s superior grows, failure becomes more and more of a taboo. In most organizations at least. Were you working for Pixar, failing fast and failing often would be the ruling paradigm. In addition to more personal motives, experimentation is not practiced enough due to resource constraints (e.g. providing physical settings such as laboratories and the necessary know-how), an unsupportive company culture, time constraints, or simply inability to lay down a repeatable model for efficient experimentation. The list of excuses could go on.



(Image source: http://d1mpb3f4gq7nrb.cloudfront.net/img/toons/cartoon6798.png)

Whatever the reason, sooner or later innovators realise that they must experiment in order to survive. At best, they will incorporate learning at every stage and maintain their edge. At the very least, they will know exactly where they stand with respect to their product, service or business model.

Putting assumptions not to rest but to the test is certainly not easy. After all, innovation competence building never comes easy. In a recent post for the Xerox PARC Blog (i.e. Xerox’s Palo Alto Research Center) titled How to Use Experiments to Build an Innovation Culture Tim Kastelle brings back the story of Flikr to illustrate how crucial experiments are to acquiring expertise (Here is a 2012 post on the same topic). Kastelle’s view on why experimenting is so rarely done in business adds to the list we started: risk aversion, or the reluctance to test one’s assumptions for the fear of looking unprofessional and even insecure; the comfort of staying away from inferred facts - usually challenged by experiments; sheltered logic or the “trap of overthinking” – the further away innovators are from their customers and implicitly the realities of the market, the better sheltered the logic is; and finally, experimenting is seen as a way to close the gaps between higher management aspirations and groundwork reality. Experimentation is uncomfortable simply because it forces all layers of an organization to acknowledge the true state of things.

Consequently, “I don’t know” needs to unretire and start a second life. Preferably alongside “Let’s find out”. I noted the concept of enlightened innovators at the onset – the ones that had built experimentation into their corporate DNA. What building blocks do they employ, if any? Is there a tested model for continuous experimentation? Harvard Business School Professor Stefan H. Thomke*, author of the widely acclaimed hardback Experimentation Matters suggests there is. In his book, Thomke brings a number of important ideas into the limelight. My favourites lie in Chapter 3Experimentation and Learning”.

The first idea is actually a list – a list of facts that affect learning through experiments:

  1. fidelity of experiments – the degree to which a model and the testing conditions represent a final product, process or service under the actual use conditions;
  2. cost of experiments – total cost of designing, building, running and analysing an experiment;
  3. iteration time – the time from planning experiments to when analysed results are available;
  4. capacity  - the number of same fidelity experiments carried out per unit of time;
  5. strategy – the extent to which experiments run in parallel or in series;
  6. signal-to-noise ratio the extent to which a variable of interest can be obscured by experimental noise;
  7. type of experiment – degree of variable manipulation: incremental or radical changes.

The second important takeaway from the Chapter is a note on the very nature of experiments. Thomke argues that when managers, engineers and scientists want to go beyond the passive learning through observation – e.g. the astronomer who observes the processes in the Heavens** – or exploration – e.g. the anatomist who dissects plants and living organisms – they may choose to carry out experiments. In other words, experiments are manipulative by nature and therefore require processes (speaking of which, you might have heard of Facebook’s recent “timeline manipulating” experiment). Enter the third important point: experimentation can be a four step iterative cycle: design-build-run-analyse. Team New Zealand Yachts, mentioned in the book, makes an illustrative case.

All in all, Thomke emphasises that experiments can be organized to maximise learning. As new technologies such as computer assisted simulations amplify the 7 factors creating higher performance, more innovation and more value for everyone is imminently created (see here the NOTOX case, or how new models in cosmetics are replacing animal testing with virtual bodies).

Summing up, experimentation matters and organizations that set their biases aside to build a culture of experimentation are likely to thrive. From “I don’t know” (Design) to “What if..” (Build) and “Let’s find out..” (Run) and further to “Let’s take notes of what we’ve learned” (Analyse), the cycle of learning and through it, innovation success, is repeatable and most of all, tangible. No miracles needed.


(source: http://floppingaces.net/wp-content/uploads/2013/02/81_12708920130213120149.jpg)

In my next post I will take you on a journey to the realm of corporate foresight.

This is where the “What ifs” really get interesting.

I have also referred to Prof. Stefan H. Thomke’s work in an older post on the emergence of the so-called Creativity Era. Here, the angle was as follows: Companies need to invest in responsive innovation management processes to observe changing demands and emerging opportunities, and to manage creativity in advance of competitors. One such system is the experimentation loop. Enlightened experimentation therefore makes a recommendation for long-term success. 

** See here what happens when the search for here Earth 2.0. is done through a dirty lens.

download the free white paper on disruption

Topics: People & Culture

Subscribe to the Blog

Subscribe to the Blog
Business Innovation Brief