One of the techniques we use in our industry is the center
of some controversy in the news recently. Specifically, the use of data mining
Datamining Scare,” The Wall Street
Without taking sides in this controversy, I find it interesting that data mining is being used in many applications today including trying to look for the “bad guys” (again, without commenting on the intentions of this program which is NOT what this blog is about.)
For too long the classic example of data mining was from the retail industry: on Saturday nights guys’ shopping baskets include both beer and diapers. Even Wikipedia, the online free encyclopedia, defines data mining with the beer and diapers example in the second paragraph.
Although I am sure the beer/diaper connection was a useful market basket analysis for that retailer, data mining has come a long way since then and is being used in many industries. Companies continue to data mine all sorts of demographic and consumer purchase data to determine who might buy what and under what conditions like the beer and the diapers example. Data mining is also used for fraud detection for your credit cards. More importantly for many of us, data mining is also being used in medical research and clinical trials to determine what might cure or lessen the impact of diseases.
Data mining has been advancing over the years. You may have seen it called artificial intelligence, predictive analysis or other names. Regardless of what people call it, the cost and resource intensiveness has decreased, putting it within the reach of many more applications over the years. It was once the province of only the most deep-pocketed enterprises (like the federal government, oops let’s stay off that topic!) but now is a cost-effective technique for many applications in companies of all sizes.
But, just like the tip of the iceberg, there is something to remember about any great data mining application. The data. Data warehousing and enterprise applications that are built on relational platforms have enabled these data mining applications. Whether they’re mining for information about diaper purchases or phone records, it’s ultimately all just data.