In the movies, the machines will become our enemies. In The Terminator, the rise of the machines leads to an apocalyptic future; in 2001: A Space Odyssey, Frank is murdered and Dave is locked out of the spaceship by HAL; and in The Matrix, humans are used as a power supply after the sun is blacked out. No wonder we’re a bit afraid of the machines.
While these machine-controlled futures are rather unrealistic, many businesses today are in danger of an increased reliance on machine learning as the end all, be all of data analytics. Let’s face it, data science talent is scarce. And as more businesses are forced to consider full integration of analytics into their decision-making processes, a machine learning-only solution with limited human oversight is tempting. But this approach exposes us to potential risk … eerily similar to those in the movies.
Here is how the scenario is playing out:
- We are in the early days of the Big Data revolution. We are still exploring the capabilities that Hadoop infrastructure provides business. Many of the heavy infrastructure investments have been made, but the initial query tool and data wrangling investments have not matched the infrastructure gold rush. For many businesses, these investments are still ahead.
- As companies ramp up their analytics investments to leverage Big Data, the use cases will explode. This will dramatically increase the demand on analytics professionals and further strain the lack of experienced resources.
- If this happens, late-comers will be forced to consider shortcut alternatives, including automated artificial intelligence options. For many, this will be good enough. For some, it won't. For others, this may be catastrophic.
Now I should make clear that this vision is entirely conjectured. Today's machine learning techniques are rooted in solid statistical and mathematical science, and fortunately there is still enough good data science to go around. The nightmare scenario -- too much data, too much demand, and not enough talent -- is possible but not imminent.
We do have to remain vigilant and cautious. In the meantime, there are new and different methodologies available to help overcome the talent gap. For example, Michael Schrage, in a recent HBR article, suggested a new way to think about developing analytics talent: homegrown teams. Mr. Schrage writes:
"...seed-fund and empower small cross-functional data-oriented teams explicitly charged with delivering tangible and measurable data-driven benefits in relatively short periods of time. The accent is on the word team; the emphasis is on building greater data capability than better digital infrastructures. The goal is to make all of the organization — not just the geeks and quants — more conversant in how to align probability, statistics, technology and business value creation. No black boxes or centers of analytic excellence here; they want data science to be a cultural value, not just a functional expertise."
There's nothing new about teamwork and shared collaboration. As a matter of fact, in addition to addressing the data science need, this is likely the best way to adapt and avoid a Terminator apocalyptic future as well.