Defining Big Data Characteristics – A Data Scientist’s Life Hack
Unless you are good at generating the right data characteristics, or features, which are relevant to your business decisions, you cannot increase the value of big data
In my previous post I talked about external data and data ingestion capabilities a financial platform must have. Now I’d like to describe the capability of developing big data characteristics, how important this capability is for a financial platform and how banks can use this capability to boost the value received from the data they can access.
Most data scientists and analysts would be aware of the so-called four Vs of big data:
- Velocity – The speed at which the data is generated
- Volume – The size of the data
- Variety – The different types of the data
- Variability – The rate of change in a data set
Now people talk about the seven Vs, including:
- Veracity – The reliability of the data
- Visualization – The ability to see the data to discover patterns
- Value – The business benefit we can gain using that data
I believe the most important factor of all is value!
What is the value of the data for your business decision and how can you maximize it?
One common misconception is that if the 6 other V’s of data are improved, the value of the data will be increased. In fact, there is no such linear relationship. Unless you are good at driving the correct characteristics, or features, from the data, which are relevant to your business decisions, you cannot increase its business value.
What Is a Feature?
Features are data elements that are used as inputs to an analytical model. Choosing what features to use is important for a machine learning model. It is the key because some of the features will actually drive outcomes but some of them will not.
Few organizations are able to generate decision-ready characteristics to take the full advantage of all available data. In order to do that, you should have experienced data scientists in your organization. However, each business decision requires different data characteristics and there are so many different decisions a typical bank takes in their business processes, it’s almost impossible for one data scientist to get to know which characteristics are more predictive than others in every one of these decisions.
From Decisions to Questions to Characteristics
Following up on the same unsecured instalment loan example in my previous posts, below is an example set of decisions a bank takes for managing an unsecured instalment loan through the customer lifecycle and some predictive characteristics which your data scientists need to develop:
There are hundreds more characteristics generated and analysed before a final customer decision is given at banks. Above is only an example to show that there is a wide variety of predictive characteristics a bank needs to generate from the data available at the time of that specific customer decision.
To manage this variety, banks tend to work with different analytics consultants specialized in each one of these decisions, or build separate specialized teams working in silos, typically without having access to the same data mart. These teams also develop different characteristics, either using the machine learning processes or with the help of data analysts’ experience. While these characteristics can be used for different decisions across the organization, each team cannot access the other team’s characteristics to validate and/or understand how valuable these are to their own business decisions.
Centralized Characteristic Generation
That’s why centralization is important to maximizing the effectiveness of the characteristic generation process and making it more cost-effective. We also observe that organizations are starting to simplify their machine learning pipelines and create a more collaborative environment for data analysts working on machine learning. Having a centralized characteristic library would definitely accelerate this effort. By sharing features / attributes / characteristics across an organization, different data scientists and decision modelers can use them for different decisions and there will be no need to compute them many times.
Now whether your external data comes from a credit bureau, social media or a government website, the bank and its competition have access to the same external data. The competitive advantage comes in the speed with which you can understand the predictive value of data characteristics and translate those characteristics into decision-ready keys. There is no time to waste via making champion-challenger tests on the data characteristics. The bank must have an established, easy-to-access characteristic library in order to do that, such as the feature store in the FICO Platform.
Another feature banks should consider in a characteristic library is how much they allow different approaches to analysts when they’re exploring the data. An example is when data scientists need to analyse the data without having a clear definition of what they’re looking for. This often happens in a pre-crisis period, where the portfolio delinquencies are on the rise and portfolio managers try to understand what is causing the issue. In this case, analysts explore the data, looking for some co-related patterns. The characteristic library must then provide different ways for analysts to view and investigate the data, pull in new data, merge, create and use new data characteristics. This makes the analysis powerful when it comes to working on machine learning and helps with operationalizing machine learning into production systems.
In my next post, I’ll write about microservices, which are another tool data scientists use to make strategy generation easier and more effective.
In the meantime, you can explore these links for more information on how to build and use a data characteristics library:
https://www.fico.com/en/latest-thinking/product-sheet/fico-master-file-characteristic-library
Popular Posts
Average U.S. FICO® Score at 716, Indicating Improvement in Consumer Credit Behaviors Despite Pandemic
The FICO Score is a broad-based, independent standard measure of credit risk
Read more
The GSE Data Is Public. Here's What It Shows About Credit Score Modernization.
Independent Milliman analysis of GSE historical data confirms what anyone can now verify directly: FICO® Score 10T is the most predictive score.
Read more
FICO® Score High Achievers: Is Age the Only Factor?
<p>FICO’s latest research on <a href="http://www.fico.com/en/blogs/risk-compliance/us-credit-quality-rising-the-beat-goes-on/" target="_blank">national FICO® Score distribution</a> shows that US credit quality continues to trend upwards, with an increasing number of consumers scoring in the highest scores ranges. Given that, we decided it was time to refresh our study on “<a href="http://www.fico.com/en/blogs/risk-compliance/infographic-what-does-a-high-credit-scorer-look-like/" target="_blank">FICO® Score high achievers</a>,” where we examine the credit behavioural profiles of consumers with higher credit scores.</p> <p>As expected, our latest study showed that older people generally have higher scores, as has been the case in our <a href="http://www.fico.com/en/blogs/risk-compliance/age-before-beauty-credit-worthiness-before-youth/" target="_blank">past research</a>. This is largely because they have trade lines that have been open longer, which leads to higher scores (assuming the trade lines are in good standing and all other things equal). The most credit savvy among you will remember that length of credit history accounts for roughly 15% of the overall <a href="http://www.myfico.com/CreditEducation/WhatsInYourScore.aspx" target="_blank">FICO® Score calculation</a>.</p> <p>But for younger consumers, it isn’t too helpful to say, “just wait until you are 50 years old, and that will fix things!” Thus, I decided to dive deeper with our analysis, including looking at the credit behavioural characteristics of younger consumers with high FICO® Scores.</p> <p>In this analysis, I looked at credit characteristics for the two highest score bands, 750-799 and 800-850, which have about 16% and 23% of the population, respectively. I further broke this down by looking at the characteristics of the youngest 15% in each group, which occurs at age 29 for the 750-799 group and 43 for the 800-850. We used FICO<strong>®</strong> Score 9 for our analysis, and examined typical variables of the categories that go into the make-up of the score.</p> <p><a href="http://www.fico.com/en/blogs/wp-content/uploads/2016/10/FICO-Score-High-Achiever-table.png"><img class="alignnone size-full wp-image-29367" src="http://www.fico.com/en/blogs/wp-content/uploads/2016/10/FICO-Score-High-Achiever-table.png" alt="fico-score-high-achiever-table" width="715" height="747" /></a></p> <p>Since there’s a lot to dissect in this chart, I’ll highlight a few findings that struck me: <ul> <li><strong>Credit history.</strong> Higher scores are undoubtedly associated with a longer time on file. What’s striking is the higher average time on file for the 800+ group, both overall and for revolving accounts (the most common trade line). Clearly, it helps to start credit use early in life – but that’s not the sole determining factor for high scoring. The younger cohorts in both the 800+ and 750-799 groups have time on file medians significantly less than their older counterparts.</li> <li><strong>New credit.</strong> High credit scorers do, in fact, pursue new credit. We see little variation in months since last trade line opening and last inquiry among all groups observed.</li> <li><strong>Pay on time.</strong> As you might expect, the further in the past the delinquency, the higher the score – and that’s <u>if</u> a delinquency ever happened. 95% of the 800+ people and 72% of the 750-799 group have never been delinquent, which is a striking contrast to the 57% for the total population. The percentages for the younger groups are even higher, especially so for the 750-799ers. Collection items are also scarce for these high scorers. While 23% of the national population have collection items greater than $99, the figures are only 0.5% for the 800+ group and 3% for the 750-799 group. It is clear that, in conjunction with starting a credit file early, paying on time is a key trait of the FICO® Score high achiever.</li> <li><strong>Balance and utilization.</strong> Finally, we look at how credit is being used, focusing on revolving product use. (Since most people only have one mortgage or one auto loan, variation in instalment product figures is minimal and driven by payment history, which is near perfect for high scorers, as we’ve seen.) FICO® Score high achievers tend to have more open revolving accounts. While the % active trades and utilization are generally lower, by no means do high scorers avoid the use of credit products. What drives the difference here is the amount of <u>available</u> revolving credit utilized. The 800+ group is roughly 1/4 of the national median, both in total and on their highest utilized card; they also have lower balances and more cards<em>. </em>The 750-799 group is more active and higher utilized, but still only at 2/3 of the national figures. The young 750-799 group compensate with lower revolving balances and, as a result, lower utilization, which is in line with what we see for the 800+ group.</li> <li><strong>Benchmarks.</strong> I’d like to point out that while this study provides good benchmarks of the typical credit traits of the FICO® Score high achiever, individual credit profiles will vary widely. And keep in mind that when we sort groups of consumers as a whole by different behaviours, some results may look superficially odd. For example, the 800+ band has a longer median average open account time (128 months) than average revolving months on file (140 months).</li> </ul> So, what do we conclude? A huge takeaway is that high scorers don’t have too much in the way of serious delinquencies and negative public record data.</p> <p>In most respects, the young 800+ people are like their peer group. They really only differ markedly in their shorter credit history and higher non-mortgage balance, both of which are a function of life stage. Our results show that you don’t have to be a credit product hermit; these high scorers have more credit cards. The takeaway is to start credit activity early and maintain well-managed and moderate usage.</p> <p>In many ways, the young 750-799 people are more interesting because their age profile is much younger. Their credit characteristics differ more from their peer group, particularly in terms of length of credit history. They compensate by having almost pristine credit records and lower balances.</p> <p>Ultimately, 39% of US consumers are FICO® Score high achievers, which they accomplish by following common-sense credit practices. It ain’t rocket science!</p>
Read moreTake the next step
Connect with FICO for answers to all your product and solution questions. Interested in becoming a business partner? Contact us to learn more. We look forward to hearing from you.