Customer relationship management (CRM) systems have long been hailed as a way for companies to find, influence, and retain customers. CRM solutions have evolved to go beyond simple contact and sales management application to now link sales, marketing and customer support operations into a single, cohesive platform.

However, the true power of CRM systems lies not only with established processes, but with data. Data shapes decisions, influences marketing and dictates the way in which customers are treated.

This article takes a closer look at data spider technology, a powerful tool for unlocking a data set’s predictive variables, and the value that is being garnered when applied against transactional data to help segment data to drive marketing and individual customer treatment.

Data spiders work by harnessing the power of genetic algorithms to exhaustively search for the predictive information in raw transaction data. With the immense amount of data collected, organizations are finding the use of data spiders of great value within their cache of transactional data to improve targeting for cross-selling, improve fraud detection, as well as predict online purchase decisions.

First let us consider the use of data spiders within a marketing application. A simple transaction database contains purchase histories of up to ten products. The question becomes, what combinations of past purchases are predictive of future revenue? The number of unique product combinations is in the millions. From a marketing perspective, untangling the web of how consumers make decisions is invaluable in understanding a consumer’s intent and/or willingness to purchase a product.

In general, consumers do not openly reveal the larger context of their buying patterns and marketers must therefore use educated guesses and secondary means of trying to determine their best prospects. As a result, Marketers must use educated guesses and secondary means of trying to determine their best prospects. Obvious correlations are asserted. For example, the correlation can be made that new homeowners are good prospects for furniture sales or that new parents are good prospects for baby formula sales and diapers. These correlations are often missed as there become too many factors and variables (in the millions) that can be measured to predict the willingness to purchase a product or service.

A popular process to distill this information into the larger context of helping predict purchase decisions is with data mining. However, data mining when tested against significant amounts of data, often falls short because it is limited in it s ability to search over the possible indicators that are predictive: the variables in data mining algorithms are often pre-determined by analysts based on their domain knowledge and do not encompass more than a small portion of all the possible predictive variables. Because the variables are pre-determined, they are static and often not relevant for new data feeds.

The story is the same with other existing analytical methods — they are all dependent upon the skill and labor intensive effort of the analyst in determining which indicator variables are predictive. In effect, just as with marketers who use only obvious relationships to target prospects, existing analytical technologies also leave unexplored a large portion of the tangled web that is created by all of the relationships and constraints involved in the consumers’ decision whether to purchase a product or not. This means that marketers, by only considering a small portion of the possible indicator variables, have so far only had a glimpse into understanding what really drives consumer purchase decisions. This has limited, and often wasted, the effectiveness of their marketing efforts.

Using data spiders, marketers can be sure that all of the potential variables and relevant patterns that exist in the data and that define the many relationships and constraints that play a role in determining whether a consumer will purchase a product or not will be explored and those that are truly predictive will be identified.

With its extremely thorough data evaluation, Data Spiders make it easier to sift through this data and create the most predictive characteristics for a modeling project. Any source of transaction data can be used as raw data, including click stream data for fraud or cross-selling; retail sales records for offers strategies; call detail records (CDRs) for churn prevention; and monthly master file snapshots for policy decisions.

Today, vast amounts of transaction data are collected and stored in massive data warehouses. For many institutions, this transaction data is often the largest part of their stored data. Unfortunately part of the story is that transaction data is very hard to analyze and is often the least analyzed — leaving its value locked.

There are literally millions of variables that exist in transaction data but with only a manual and labor intensive process available to analyze these variables, only the surface is scratched of all other information insight available in the data. With data spiders, the manual effort of “discovering” predictive variable is automated allowing for the full information value of transactional data to be exposed. For marketers, this means that this trove of transactional data becomes actionable and can be used to increase the effectiveness of marketing efforts.

Data spiders, not only can a comprehensive view of predictive variables be generated, but with the evolution of these predictive variables transaction data changes can also be traced. For the first time, data spiders allow companies to not only understand the full range of predictive characteristics of their customers, but also allows them to stay close to customers as they evolve their preferences — a dream come true for marketers.

Rahul Asthana is director of product marketing for Fair Isaac’s Enterprise Decision Management Software and Solutions where he is responsible for working with organizations to identify, define and develop advanced analytic solutions that combine predictive modeling, optimization technology and decision strategies to incrementally improve and refine business’ tactics and results. For more information please contact Dr.Asthana at rahulasthana@fairisaac.com or by visiting www.fairisaac.com/edm Fair Isaac