DAVENPORT'S "HOW TO DESIGN SMART BUSINESS EXPERIMENTS"
/
Airport time is always one of discovery. Today's was Tom Davenport's Harvard Business Review article How to Design Smart Business Experiments. Timely, as I'm teaching evidence-based management in my Organizational Design course on Wed.
I've pointed the students to my June post calling for greater focus on evidence-based management (e.g., making decisions based on evidence, often from small research studies). In that post I list (it's a short list) some materials appropriate for teaching the topic in core MBA courses, like my Organizational Design course. Davenport's work is a nice addition to that list, as is his book (with Jeanne Harris) Competing on Analytics.
Some key points from the new article:
- There are times when testing doesn't make sense: Some times a small scale test isn't possible, and so intuition must rule; time is sometimes of the essence and experience is clearly available to make an informed decision.
- Many transaction oriented companies have excellent baseline data and the ability to quickly test new ideas. For example, eBay "..has also built its own application, called the eBay Experimentation Platform, to lead testers through the process and keep track of what’s being tested at what times on what pages."
- This isn't just about technology or a testing process alone -- In the words of this blog "you can't change just one thing" in implementing a management practice:
Establishing a standard process is the first step toward building an organizational test-and-learn capability, but it isn’t sufficient unto itself. Companies that want testing to be a reliable, effective element of their decision making need to create an infrastructure to make that happen. They need training programs to hone competencies, software to structure and analyze the tests, a means of capturing learning, a process for deciding when to repeat tests, and a central organization to provide expert support for all the above.Also helpful is that Davenport describes a seven-point process that includes knowledge management: For example, check first to be sure that the test hasn't already been done; after the test, add to a learning library, publicize tests and results. Super article, though I'm left wanting two things: More information on Capital One University's professional training on testing and experiment design, and greater focus on testing internal organizational changes (much of the focus is on customer-facing experiments). Suggestions for additional MBA appropriate readings/materials on these topics are greatly appreciated.
