Reassessing the Fundamentals of Measurement: Report From a 2007 Metrics Conference  

Event Summaries—Management Roundtable's twelfth annual conference on Product Development Metrics began on November 5, 2007 in Chicago and focused on two themes: Increasing Return on Innovation and Achieving Organic Growth. This summary report of the conference proceedings emphasizes the areas where speakers saw a need to question some of the fundamental assumptions around metrics. Keynote speak Larry Keeley, for instance, cited a 96 percent failure rate for innovation projects. For Keeley, this sobering statistic suggests that the degree of effectiveness of innovation initiatives is largely unknown in most companies. Keeley and other speakers offered some counterintuitive suggestions for companies confronted by the challenge of applying metrics to innovation. Case studies from such firms as Rockwell Collins, Intel, Kimberly-Clark, Tellabs, DJO, and MEDRAD demonstrated how a range of companies in several different industries met the challenge of measuring innovation, with a special emphasis on tying metrics to overall strategy.
(6 pages)

Intellectual Property Considerations in Technology Development 

Presentations—Tanya K. Moore, Senior Director, IP Licensing, Microsoft  Corporation
In this presentation, Microsoft’s Tanya K. Moore discussed the most important considerations governing IP and Technology Development.  First, Moore presents the various types of IP, which include much more than trademarks, patents and copyrights.  She then addressed various strategies for leveraging IP including budget and the business plan considerations, make vs. buy valuation scenarios, patent cross-licensing for freedom of action, and outbound technology licensing as part of a go-to-market strategy.  Moore also discussed concepts related to IP valuation and encumbrances, as well as some of the organizational implications of IP management.  This document includes presentation slides and a text summary of Moore's remarks.
(14 pages)

Architectural Trade-Offs & Target Costing for Co-Design  

Feature Articles—m>Brad Goldense, President, Goldense Group Inc.
The evolution of technology has led to increased complexity in design planning. In today's world of design there are typically several design alternatives available for any given feature. Designers must make choices early in the project as to the best way to implement a given feature. How, then, do most designers choose? Many choose the alternative that facilitates rapid design and/or those that fall most naturally into their own design competencies. Many designers interact with projects, and the planning thereof, at a level of detail below the level at which management makes its trade-offs. Yet, many design trade-off decisions left to lower levels of the organization significantly impact management-level Economic Trade-off Analysis. How does one improve the linkage between the impact of trade-offs made at lower levels and the bottom-line, management-level, analysis? In this feature article, Goldense presents a tool which, in conjunction with Economic Trade-off Analysis, may provide a scalable methodology for helping make such trade-offs. This tool, the "Target Costing For Co-Design Matrix" [TCCM] will add some work to the project up-front, during the Definition Phase, but like most other up-front analyses, claims Goldense, it is usually a worthwhile investment.
(4 pages)

Insights from Alberto Culver:  Implementing In-Flight Metrics 

Interviews—Dr. John R. Berschied, Jr. is Group Vice President of R&D for Alberto Culver Consumer Products Worldwide.  In this conversation, John shares his views on the use of in-flight metrics to assess R&D program and portfolio direction.  He notes that many R&D metrics are either retrospective (e.g., percentage of sales on R&D or percentage of sales from products introduced in the past three years) or process-oriented (e.g., execution on a Phases and Gates process or use of technology platforms).  In-flight metrics are less common than retrospective metrics but equally essential for predicting the outcome or the probabilities of certain outcomes in R&D projects.  In-flight metrics also may be used to assess whether or not product development efforts, in general, are headed in the right direction.  These metrics are generally qualitative and should be used in combination with quantitative metrics.  Download the document below for a text summary (5 pages) and click here to download a related slide presentation (34 slides).

Defining Metrics for Resource Capacity Management: Interactive Exercise Where You Are the Experts 

Presentations—In this slide presentation, Wayne Mackey of Product Development Consulting Inc., defines a set of three simple steps for metrics success: 1) define and connect the improvement goal, 2) determine the ‘causal actions’ that lead to each goal and 3) determine the ‘critical few’ causal actions and then set metrics. The presentation draws the distinction between process and results metrics and provides a simple example. A tree diagram illustrates an example of engineering metrics demonstrating how metrics cascade down from a strategic goal to the executive, functional VP, project manager, and project metrics levels. The presentation then asks the participants in the exercise to define metrics for resource capacity management, through several stages, providing a checklist to make sure that goal is adequately and clearly defined and another checklist to ensure that ownership of the metric is properly managed.
(17 slides)

Measuring the Value of Open Innovation: Metrics, NPV and ROI 

Event Summaries, Presentations—A Presentation by Simon Hayes, Senior Director, Strategic Alliance, Cisco Systems
Cisco Systems has created a comprehensive measurement system for Open Innovation projects and alliances. Cisco’s experience suggests that it is necessary, first, to have a consistent methodology for measuring business return – one that embraces all of the types of projects under the corporate umbrella. Cisco tracks the metrics at the project, alliance and alliance portfolio levels. According to Hayes, comprehensive NPV, across all of these levels, is the best measure when comparing one project with another. Including non-financial, strategic measures (e.g. early market entry or the opportunity to create a new standard) in your calculations is essential – it is difficult to quantify them, but estimating them is better than ignoring them. An up-to-date metrics dashboard allows for mid-course corrections, and Cisco has created an automated, frequently updated dashboard solution. Finally, it is necessary that everyone involved on both sides feel ownership over the metrics and are accountable for them. To be successful in strategic alliances, says Hayes, it is absolutely necessary to develop a sense of joint destiny with your partner.  Download the presentation slides (19 pages) here and then download the text summary below.
(6 pages)

Open Innovation and Metrics: Designing Partnerships for Measurability 

Interviews—An Interview with Wayne Mackey, principal consultant, PDC Inc.

In this August 2006 interview, consultant and metrics expert Wayne Mackey, a principal consultant with Product Development Consulting, Inc., discusses the special challenges of measuring open innovation projects. Mackey’s research indicates that communication is the single most important focus for open innovation metrics. Mackey discusses the different types of open innovation projects and where the levers of control reside in each case. Mackey also provides a framework for categorizing the roles and responsibilities on each side of the partnership and gives ten general metrics that form a starter list for measuring open innovation. Finally, Mackey generates a short list of “do’s” and “don’ts” for measuring open innovation projects. Mackey’s work suggests that best practice organizations are consciously designing partnerships for measurability.
(7 pages)

GUIDE TO LEADING PRACTICES: Metrics 

Benchmarking, Surveys & Metrics, Methodologies & Best Practices—This report presents several leading practices related to product development/R&D metrics derived from practitioner experience and benchmarking research.  Management Roundtable has culled these practices from our knowledge base and formulated them as simple, actionable, bullet-level statements.  In addition, the GUIDE cites the source for each practice, presents a brief discussion of each, and provides links to further information.
(13 pages)

Metrics Wisdom from DuPont Corporation 

Feature Articles, Methodologies & Best Practices—As an internal consultant within E.I. DuPont de Nemours, Greg Ajamian has accrued broad experience working with DuPont’s 20 strategic business units with 90 separate businesses. In this report, Greg shares numerous insights with regard to product development metrics, with clear and concise pointers and tips that address all phases of the product lifecycle, with a specific focus on the fuzzy front end and the product portfolio.
(8 pages)

Product Development and R&D Metrics: Summary of Audio Session 

Event Summaries—Related Links:  Audio - mp3 or wma | Transcript (16 pages)

This report summarizes a discussion among Management Roundtable members and experts John Cordes, National Semiconductor; Ken Delcol, MDS Sciex; and Steve Evans, Cranfield University.  This summary covers such questions as “Why do you measure and what measurements do you use most?” “How can a company use metrics to encourage positive behaviors or discourage negative behaviors?” “What metrics are falling out of favor and which are becoming more popular?” “How do you successfully implement a metrics strategy within an organization and what are the potential pitfalls?” “How do you select and tailor metrics by projects?” and other questions and challenges associated with metrics.
(8 pages)