How does your company define Big Data? How are you using it? At FICO World our chief analytics officer and FICO Labs head, Andrew Jennings will be leading an executive panel that will explore what Big Data means, what are the challenges in getting started with adopting Big Data approaches, how to get real-world benefits and what comes after Big Data.
In advance of FICO World we sat down with Andrew to get a preview of the FICO World talk. Below are excerpts from the interview, as well as link to the full webinar.
FICO Labs: From an analytic perspective, what do you see as the major opportunities for using Big Data?
Andrew Jennings: I generally see opportunities for Big Data in three areas:
- Variety – it is nice to have more data, but generally speaking, more different data is more important, than more of the same data. Generally, you will see more lift in increasing the variety and extending the number of columns, rather than extending the number of rows.
- Situations where there is high complexity and rare events. Fraud is a classic example of where an event is rare. The more transactions you see, the more types of fraud you can detect.
- Speed of response. Because computing speeds are much greater, you can act quicker and refresh models faster which is really advantageous from the analytics side.
Andrew Jennings: In terms of challenges, two challenges are analytic things and the third is generally important.
- Day-to-day, it is a challenge to free up people to work on something new.
- Your analytics people need to know a broader number of tools and techniques. Either you need to train your analytic people, or get in the marketplace and hire new people who already know these tools and techniques.
- You need to find something which is meaningful to start with. I suspect disillusionment comes when you have a grand plan to buy servers, install Hadoop with the hope of finding something. It is better to have business goals and use Big Data with a feasible plan to get value. It is important to have the objective in mind and the means of getting into the marketplace and starting to generate value. Be clear about what problem you are looking to solve and why it’s important.
Andrew Jennings: I look forward to my conversation with Kenneth. My background is as an economist, and econometrician. I was brought up on topics of specification bias, spurious regression, and around the time of path breaking time series analysis to demonstrate causation. In terms of economics it is easy to draw graphs with one thing going up, and one thing going down, or both things going down or both going up. Implying one thing causes another is not necessarily correct.
For some things correlation maybe OK however in the areas we typically work in we are trying to demonstrate the impact of an action and so we need to establish cause as well as effect. The route is to go from action to effect. On Saturday I was reading a New York Times Op-ed, questioning research that found a relationship between public adverts for fast food and obesity. The thesis was that if you reduced the amount of adverts in an area, it would be beneficial to public health. You don’t have to be that thoughtful to reverse the logic to say that there are more adverts because there are more obese people in an area.
Another article I saw recently in a similar vein was about public parks and obesity. The study found that more public parks were in areas with people who are less overweight. The thesis is to build more parks which would be better for public health. I am sceptical that the existence of a park would lead to this result. Maybe people that are less overweight are drawn to an area because of the existence of the park.
I consider myself an informed skeptic. I need to understand the impact of the variable on an outcome. In the retail world it is figuring out who is sensitive to price and who isn’t. You need to understand causality, not just correlation.