In Fraud Detection, We Can Live with the “Black Box”

When people speak about fraud detection and fraud scoring, they will frequently use the term "black box." If I had never worked in the anti-fraud area, this phrase would most likely evoke memories of the 1990s dance-floor classic "Ride On Time" (a great track, incidentally!). But no, in our industry it refers to the hidden analytic capabilities, algorithms and statistical calculations that make up the assessment of fraud risk.

For many, "black box" has come to be a bad thing. Our insatiable desire to understand how things work leads people to be critical or suspicious of any process that seems unclear or opaque  — especially if it’s making judgments about us.

This criticism always surprises me. There are many technological, chemical or biological marvels that, despite a rounded education, I could never hope to fully explain, but my lack of understanding doesn’t diminish their value.

Nevertheless, under the new banking regulations, most organizations using a proprietary risk scoring process must now be able to state what influences these scores (for good and bad), even when the underlying analytics and calculations remain protected intellectual property. In the risk scoring business, consumers and regulators demand the opposite of the black box: transparency.

FICO Falcon Fraud Manager includes world-leading analytics and models, and while the analytic formulae aren’t as widely known as those in, say, the FICO Score, they’re not a pure black box either. Each fraud score comes with a series of three reason codes that indicate the most influential factors. These can aid in the investigation, if needed. Falcon clients can also contribute to Falcon’s fraud risk assessments by incorporating their own models, scores, user-defined profiles and conditions.

Will consumers and regulators next demand that we open the black box that scores transactions for fraud, and make the calculations more public? I doubt it. All parties would recognize the danger in helping criminals to outwit the system.

So even while we pursue transparency in some areas, we shouldn’t be too quick to damn the “black box.” Sometimes a little mystery is a good thing.