Consider for a moment the following three statements:
- Innovation results from lucky accidents;
- The more open the innovation process, the less disciplined;
- Innovation can’t be measured.
What do they have in common? If you’re inclined to say: “They’re all truisms!” (= Truths too obvious to mention) I’d pat you on the back and kindly invite you to join us at the next HYPE Innovation Managers Forum in Bonn for two days of enlightening conversations about innovation practice. But luckily, that’s not the case. With you, dear reader, I don’t have to allude to anything. You’ve been following this blog (and others), not to mention implementing innovation strategies for long enough to recognize these common misconceptions about innovation. So in today’s post I’ve compiled some insights to take your existing wisdom further. More specifically, the post is about recent developments in open innovation metrics (yes, they matter!) and is inspired by a study titled Measuring Open Innovation published in Ernst & Young’s Performance Journal in mid-2014.
For the study, Marc Erkens and Susanne Wosch of Ernst & Young, Germany joined forces with Frank Piller and Dirk Lüttgens at RWTH Aachen University, one of Europe’s leading institutions for science and research, to develop a metrics-based management toolkit containing Key Performance Indicators (KPIs) for a set of open innovation (OI) methods. Let’s see what they found.
The old & new
Although OI projects are fundamentally different from traditional in-house innovation projects (nothing new, right?), metrics on both sides are still considered an “emerging discipline”. To quote the study: even experienced managers still go blank when asked how to assess, control, and measure the performance of open innovation (OI) activities. With no clear guidelines to monitor the process, OI measures today tend to be extremely generic and mainly focus on patent data (1) or volume of ideas submitted internally via idea competitions.
While basic metrics can and do provide minimal support, it is only tailor-made metrics (usually, a specific combination of them) that can truly make a difference – i.e., help innovation teams reach their goals.
Principles to secure before starting
To design and implement a successful metrics-based OI performance measurement system, companies must adhere to three principles. According to the study, these principles are:
1) Use unique metrics for each OI method;
2) Consider different types of measures - use a variety of KPIs that assess the input, process, output and outcome;
3) Think about how to utilize OI metrics effectively - distinguish between those for: instrumental use (direct decision-making), conceptual use (general enlightenment and understanding) and symbolic use (after the decisions have been taken, to justify them).
Metrics for ideation contests: A close-up
Having secured the background information, the study continues with an illustration of three KPI bundles (scorecards) for the three most prominent methods of OI:
- the lead-user method – or incorporating input from users at the edge of important trends;
- ideation contests – creating task-specific challenges directed at independent agents (e.g. customers, suppliers, individuals);
- broadcast search – seeking full technical solutions rather than just ideas; broadcast searches can be performed by so-called “innovation brokers” such as Innocentive or Ninesigma.
We talk a lot about idea campaigns here on the Blog and that is because campaigns are the driver for idea management success. It’s therefore useful to look at some of the most important metrics to consider when measuring their success.
Source: performance.ey.com
To decode the chart, a quick classification of KPIs in OI is in order.
- Input KPIs – measure input elements within an OI project such as human or financial resources;
- Process KPIs – measure how inputs are transformed into outputs or how the efficiency of the innovation process changes; e.g., time variances, budget variances, error ratios;
- Output KPIs – measure the results of the OI process such as number of ideas, number of patents filed, number of publications published;
- Outcome KPIs – measure the value of an innovation in terms of market acceptance (proposed value, actually, since companies can only formulate “value propositions”; the client is ultimately responsible for creating the almighty value-in-use).
Additionally, it’s useful to know that the study’s results represent the opinions of 117 experts from corporate functions and management consulting, whom were asked to assess the relevance of the metrics (or KPIs) according to their own experience (and arguably, intuition).
Now onto the conclusions
In the initiation phase of idea campaigns, input measures such as quality of the IT platform, including the cost of implementing the platform, its scalability, but most importantly, its user-friendliness ranked the highest. Output measures such as diversity of external participants and outcome measures such as customer potential (whether the participants represent the mass market that the company is targeting) followed closely.
In the implementation phase of idea campaigns, however, the output measure of productivity was the most prominent. In other words, the percentage of winning ideas that ultimately become projects was seen as a key indication of success. In addition, process KPIs such as depth of contestant community interaction and another output KPI, frequency of use of platform (an indication of the company’s reputation and image) were also highlighted as important.
In terms of overall measures, the clear winner was the input measure of top management commitment to the innovation project. Also standing out were KPIs related to customers’ benefit from innovations created through the campaign (labelled “originality”), and those related to expected increase in revenue, as a consequence of new products, services and processes.
Summing up, the realm of OI metrics is undergoing an exciting transformation. While there are no standard measures of success for the time being (no “acid test” for NPD), diversifying the criteria by which open innovation initiatives are assessed is a tremendous step forward in improving new product, service and process output. Perhaps once an updated package of innovation management standards would do the trick (2).
End Notes
(1) Analysing patterns of R&D spending will not give you the full story, but it will make lovely charts like in PwC’s recently published Innovation’s New World Order.
(2) In 2015 the European Committee for Standardisation has made available the Innovation Management Standards on its website. The publication contains 6 parts and offers guidelines for the introduction, development and maintaining of a framework for systematic innovation management practices. Current chapters include: Part 1: Innovation Management System; Part 2: Strategic Intelligence Management; Part 3: Innovation Thinking; Part 4: Intellectual Property Management; Part 5: Collaboration Management and Part 6: Creativity Management.
Related Posts...
- Brush Up On Your Open Innovation Know-How With The Best Of Henry Chesbrough’s Work
- Outside / Inside Innovation: Combining Open Innovation with Employee Networks to Drive Success
