Hits And Niches

by FICO

"The era of one-size-fits-all is ending, and in its place is something new, a market of multitudes", so says Chris Anderson (who blogs at www.longtail.com) in his recent book "The Long Tail". The phrase "The Long Tail" comes from a classic Pareto distribution or power curve that has a "head" consisting of "hits" and a long tail consisting of "niche" products (as shown on the right). He begins with three observations:

The "tail" of available variety is far longer than we realize
This tail is now within reach economically thanks to the Internet
All the niches aggregated makes for a significant market

He believes that where the 20th century was about hits, the 21st will be about niches. He illustrates this with what he calls "The 98 percent rule" that 98% of online products will be sold often enough to notice. For instance, 95% of netflix movies are rented in a quarter, 98% of amazon's books sell at least once a quarter and so on. Indeed if an online business has 20, 30 or 40 times as many products as an offline retailer (and they do), then the products that are only available online amount to 20-40% of sales. "In an era without the constraints of physical shelf space and other bottlenecks of distribution, narrowly targeted goods and services can be as economically attractive as mainstream fare". So far, so good.

He then outlines a number of themes, the first three of which seem particularly relevant from a decisioning point of view

There are far more niche products than hits
The costs of reaching these niches is falling fast
Consumers will only buy from these niches if they have ways to find the niches they value quickly.

Automating decisions using an Enterprise Decision Management (EDM) approach, especially those around cross-sell/up-sell, recommendation, precision marketing and so on is clearly going to be key for #3. The value of automated decisions in self-service also matters in this world as the number of choices, and variety of channels, will mean customers helping themselves more and valuing those companies that help them help themselves and that those that can recreate the feel of a corner store across a huge range of products and customers.

Chris identifies three forces that are driving this - moves to democratize production and distribution and the power to connect supply and (potentially thinly spread) demand. This last force is about lowering the "search costs" or reducing the economic cost of finding what you want. In a world where finding something is expensive unless it is very popular (the old model), hits matter and niches do not. In a world where the Internet and related technologies reduce this cost, niches are more viable. Decisioning technologies can reduce the "search cost" for something in several ways:

Make it easier for customers to specify their own rules
Make it easier to handle many more customer segments
Analytically derive segments based on customer behavior
Predict niche interests for customers using information about other customers
And so on

In the book he quotes Frog Design (a consultancy) "Information gathering is no longer the issue - making smart decisions based on the information is now the trick". This is the essence of my regular comparisons with BI/DW (information gathering) and the predictive analytics in Enterprise Decision Management (improving the quality of decision made using this information). In particular the kinds of analytic models that take your behavior - both implicit (what you buy) and explicit (what you recommend) - and use predictive analytics to make better decisions. Predictive analytics take the past behavior of all customers and use it to infer the likely future behavior of a specific customer. This is akin to what is sometimes called the "wisdom of crowds".

Now one often hears discussion of these kinds of analytics only in the context of using customer recommendations or explicit preferences to make targeted recommendations to a customer. However, what a customer looks at and, even more, what a customer buys are also elements of behavior. This kind of behavior also does not require a customer to invest any time in writing a recommendation (something only a minority will do). Using customer behavior and comparing it to others to make predictions about likely future behavior is the basis for everything from fraud detection to credit scoring to retention risk to, yes, product recommendation. Don't think you need customer recommendations to leverage your customers' collective wisdom - you know a lot about what they think from what they do. Further Chris points out that if I can match customer behavior to specific individual characteristics (like location, gender, age) then I can segment my customers very precisely (though there are obvious privacy issues to be considered). This segmentation is crucial to success as "In a world of infinite choice, context - not content - is king" (Rob Reid, Listen.com founder). Not only does explicit segmentation of products and customers into like groups allow for targeting, it also improves the value of recommendations and makes it easier to infer from previous customer activity.

One of the issues in the long tail is the signal to noise ratio. As Theodore Sturgeon, a Science Fiction writer, said "ninety percent of everything is crud". What makes niches different is that one person's noise is another person's signal. In general Chris suggests that this means we must replace the kind of pre-filters (editors, buyers, marketers) that try and promote the most likely to succeed products with post-filters (blogs, playlists, reviews, customers and their recommendations) having made the widest set of products available. If you have this option, if your products/services lend themselves to mass customization and niche-targeting, then you had better be good at turning lots of data into useful, predictive insight that let's you connect customers with the products they want. Otherwise you risk having customers picked off, niche by niche. His discussions of how niche products, such as narrowly focused blogs, pick off customers from broader and less differentiated products one at a time reminded me of Clayton Christiansen's "Innovators Dilemma" where new competitors take your least profitable customers initially and then work up the value chain. All that's different is that these niche competitors never become a direct competitor except as a swarm. As a swarm, though, they target niche after niche and drive you out of the market.

The last section of the book lays out the 9 steps Chris suggests:

Increase the inventory you make available
Make customers do work
I am not sure this is essential. You can gather an enormous amount of information from what they do without making them work. Clearly the more they tell you, the more useful you can make everything. Regardless, the lesson for EDM is to capture this information and use it to drive recommendations, cross-sell, pricing etc. This involves precision and agility.
One distribution method does not fit all
This means supporting lots of channels. One of the challenges with multiple channels is ensuring consistency across them. Focusing on decisions as a single point of automation and sharing those operational decisions across channels is important for the customer experience and to make multiple channels work.
One product does not fit all
The era of mass customization and off micro-variations between products is upon us. Not only does this mean thinking about the information content of your products (the easiest part to vary), it also means thinking about automating decisions relating to products to handle the increased complexity.
One price does not fit all
Move to variable and dynamic pricing as quickly as you can and automate the way you price products so you can show regulators and auditors that you have a repeatable, reliable process for generating prices.
Share information
Such as how you made a recommendation for example or why your credit score is what it is. Check out the way myfico does this for credit scores, for instance.
"And" not "Or"
Trust the market
Use the data you gather to respond with post-filtering - don't try and pre-filter. To do this you must be able to respond quickly - you must be agile.
The power of free

Chris boils the whole thing down to this:

Make everything available
Help me find it

Decision automation and management can't help with the first one but it's going to important for you to succeed in the second one. I'll close with my favorite comment from the book - one from Raymond Williams, Marxist sociologist, who said