Wharton@Work

June 2011 | Nano Tools | 

Adaptive Experimentation

Nano Tools for Leaders® are fast, effective leadership tools that you can learn and start using in less than 15 minutes — with the potential to significantly impact your success as a leader and the engagement and productivity of the people you lead.

Contributor: Yoram (Jerry) Wind, The Lauder Professor, Professor of Marketing, The Wharton School, University of Pennsylvania

The Goal:

Encourage innovation, find successful strategies, and create competitive advantages through continuous, adaptive experimentation.

Nano Tool:

Finding your next successful strategy in an environment of increasing uncertainty and complexity is daunting. The traditional approach involving the implementation of one strategy — unless you have uncanny insights into how it will work — won't yield the answers you're looking for. Even if perceived to be successful, what will it reveal about the next step? Adaptive Experimentation (AE) involves the creation and implementation of continuous experiments to improve your strategies over time. By pursuing several approaches, you will be able to better identify strategies for the next level of experimentation, yielding results that improve with each step.

Whether used to develop new products and services, target marketing programs, create better customer experiences, or redefine a brand positioning, the benefits of AE include identifying stimulating breakthrough ideas, creating an innovation culture, and developing strategy. Innovation is encouraged through experimentation. Measuring success or failure is embedded in the process, and because failure yields valuable information, permission to fail is an integral element. AE also creates a compelling competitive advantage: continuous experimentation means your competitors can't discern your master experimental design.

How Companies Use It:

  • To learn how best to reach the consumer, direct mail pieces historically have been based on adaptive experimentation. Enormous differences in results were achieved, and easily measured, through changes in an advertisement's layout, font sizes, or graphics. Experimentation captured the best iteration of every possible variable.
  • Google's use of experimentation in every activity, referred to as the "one percent rule," is legendary. An integral part of the way they do business, the approach involves measuring results to reveal what is working and what needs further experimentation.
  • To make supply chain improvements that add consumer value and competitive advantage (by reducing both inventory gluts and shortages), Procter & Gamble is investing in "sense and respond" capabilities. These improvements are part of a vision called the "consumer-driven supply network," and are being identified at a variety of points within that network. They include inventory strategies such as continuous product replenishment (CPR), produce-to-demand manufacturing, and dynamic replenishment and distribution.
  • Anheuser-Busch employed adaptive experimentation for all its advertising strategies for years and as a result has been the most profitable beer company.

Action Steps:

Adaptive Experimentation is a continuous, rather than a linear, process. The learning that takes place in step 5 informs the refinement and determination of the next objectives in step 1, meaning that outcomes improve over time.

  1. Determine your objectives: what achievement(s) is worth the time, effort, and resources needed to run an adaptive experiment?
  2. Create a culture of innovation: involve your organization's architectural processes and structures, including technology, reward systems, and incentives. To get a buy-in from everyone involved and encourage innovation, assure them that performance areas will not be penalized, and compensate as if the old strategy was being used.
  3. Design the experiment: develop many full executions, including a selection of markets in which to implement them. Strategies should be innovative, involving significantly different alternative approaches that encompass radically different methods for resolving a challenge or taking advantage of an opportunity.
  4. Implement in a controlled and regulated manner: an ongoing measurement system must be in place to evaluate your efforts as well as to monitor markets and control for external effects.
  5. Analyze your results, and use lessons about what worked and what didn't to develop your next series of experiments.

Additional Resources:

  • "From Mental Models to Transformation: Overcoming Inhibitors to Change,"Yoram (Jerry) Wind and Colin Crook, Rotman Magazine (Rotman School of Management, University of Toronto), 62-68, Spring 2009. Examines strategies for challenging mental models and overcoming inhibitors to change.
  • The Power of Impossible Thinking: Transform the Business of your Life and the Life of your Business. Yoram Wind, Colin Crook, with Robert E. Gunther (Wharton School Publishing, 2006). Explains why it's so hard to change mental models and offers practical strategies for developing new ways of seeing, changing to a new model, swapping among a portfolio of models, and improving your models through constant experimentation.
  • "Marketing by Experiment,"Yoram (Jerry) Wind, Marketing Research, 10-16, Spring 2007. Explores why researchers, in today's swiftly changing business environment, must experiment with diverse strategies. "Adaptive experimentation" allows them to challenge their assumptions and devise fresh tactics. By becoming advocates of adaptive experimentation and faithfully implementing it, they can secure a seat at the table and influence critical business decisions.
  • The Future of Advertising. This project of the Wharton SEI Center for Advanced Studies in Management looks at the future of advertising that is emerging from the interplay of emerging new media channels and a world in which consumers are in charge.
  • Wharton Fellows is a series of Master Classes for senior executives who are leading organizational transformation. The program focuses on two themes: the challenging of mental models, and the need for continuous experimentation.
  • Yoram (Jerry) Wind, The Lauder Professor and Professor of Marketing, is academic director of, and teaches Adaptive Experimentation in, Wharton's Global CEO Program and the Wharton Fellows Master Classes. He also teaches in the Advanced Management Program.

About Nano Tools:

Nano Tools for Leaders® was conceived and developed by Deb Giffen, MCC, Director of Innovative Learning Solutions at Wharton Executive Education. It is jointly sponsored by Wharton Executive Education and Wharton's Center for Leadership and Change Management, Wharton Professor of Management Michael Useem, Director. Nano Tools Academic Director, Professor Adam Grant.

Download this Nano Tool as a PDF