Wharton@Work

September 2020 | Strategy

Leveraging Analytics Today and Tomorrow: Test and Learn

The pandemic has left many companies with mountains of historical customer data that can no longer effectively help them move forward. The models they used to formulate insights and predictions just months ago are less predictive about future customer behavior. The good news for firms that are developing their analytics capability is that the playing field is more level. Today, there are opportunities for firms if they are prepared to be nimble and ready to embrace risk, says Wharton marketing professor Raghuram Iyengar.

“In today’s pandemic world, and in the post-pandemic future, customer behavior has fundamentally changed. Companies that were already digital have a lot of data, but to successfully harness the power of analytics, new competencies have to be built.”

Iyengar is academic director of Wharton’s Analytics for Strategic Growth: AI, Smart Data, and Customer Insights, which is now being offered online. He says a critical focus — one that is explored extensively in the program — is developing a test and learn culture. “To make decisions about what directions to take now and going forward, you need a new process for looking at data. If you already had a data-driven culture, you find now that your strategic plans and dashboards don’t make a lot of sense. But to manage your business, you need to know what to focus on. That process must start with exploring new initiatives.”

Months into the pandemic, many companies still are not experimenting, in part because budgets are tight and crises tend to make people more risk averse. There was also an assumption that after a relatively short shutdown things would get back to normal, meaning any investment in test and learn would provide only short-term benefits. Clearly now that is not the case.

Natural inertia is hard to break, says Iyengar, “but when the old way is no longer working, and if you are not sure how to move forward, testing and experimentation should be embraced. Traditional pattern recognition models are likely to be flawed. To ascertain new truths about customers today and into the future, senior leaders must empower people to take risks, be willing to fail, and look critically to learn from your results.”

One way to start is to examine the assumptions you are basing your current and future strategies on. What do you assume will happen with your customers when or if things return to normal? Some firms believed the current crisis to be a short-term situation, and that their new online customers would leave and old customers would come back when things returned to normal. Iyengar says no matter what the assumptions, “you have to be very clear about when they were made, what information you used to make them, and whether you have adjusted your models when new information came in to challenge them.”

“Your models are only as good as the data used to develop them,” says Iyengar, “and there is no such thing as free data — you have to get out there and work to find it. When there is a seismic shift in behavior, you need a proactive approach. Historical data isn’t going to help you as much, so you have to be willing to take on risk and even embrace uncertainty.”

In the case of large companies doing business across geographies, a good source of information could be customer behavior in regions that reopened first. “There is a question about whether the traditional sales that moved online will be sticky,” says Iyengar, “but we are already getting data in on this. Firms need to find it and make sense of it, using new approaches.”

Ultimately, in these challenging times, all companies are in the same place. No one has solved the question of what consumer behavior will look like months or years from now. But, says Iyengar, that means you can advance if you accelerate and learn faster than your competitors. “There is great opportunity for companies that are trying to become more experimental. It has to start at the top, though. Senior leaders must be willing to be curious, collect new data, and test and learn. There is risk, but there are also possibilities for tremendous gains.”