Data Doghouse - performance management, business intelligence, and data warehousing

Data Integration Myth #10: Emerging technologies will solve all our data integration problems.

New technologies aren't a shortcut for data integration. Whatever application integration tool you use, you still need to integrate data the old-fashioned way -- by talking to business users, profiling data sources, defining transformations, establishing data quality and consolidation rules, and documenting everything. EAI and EII may be new technologies that help you in a lot of ways, but they don't address basic data integration issues.

Data Integration Realities

Whether you like it or not, data integration has to happen. It's your choice to plan carefully and develop an architecture so it takes place efficiently and effectively. Anything less will lead to higher costs for your business - more data gathering, hunting, reconciliation and analysis, as well as the cost of basing business decisions on inconsistent, inaccurate, noncompliant information.

There is no free lunch when it comes to data integration. Everyone hopes it can be solved faster, cheaper and better. Silver bullets look great in demos and proofs-of-concept, but enterprise solutions are not 30-day fixes. The people promoting quick solutions are sincere, but most have never dealt with real-world enterprise problems. Most implementations are for stovepipes that don't involve enterprise data integration.

The good news is that these tools may be part of your solution, but the reality is that they are not the solution.

Data Integration Myth #9: The vendor's analytical package providing CPM is going to solve all our problems.

Vendor packages may offer the allure of off-the-shelf solutions that take almost no work and instantly provide the business with the answers to all their questions. Packaged solutions appease fears and satisfy the desire for instant solutions - and the demo looks great! However, be sure to read the fine print, such as the qualifiers that everything works fine only if all your data is clean, all the data is included in the supported sources and your business analytics match the prebuilt models.

If we've learned anything from recent history, however, it's that these packages are not simply buy-and-use projects. A great deal of customization is required to implement and deploy these packaged solutions. For these packages to really be effective, you need to perform data integration. These solutions are great as a front-end application for your data integration effort; use them to sell the business on the need for data integration. Don't let people oversell the packages - they're awesome, but only when you have the data.

Data Integration Myth #8: The ERP vendor's data warehouse package is going to solve all our problems.

Just because you're already using a vendor's ERP system doesn't mean that their data warehouse package is going to be a quick add-on. If you had to customize their ERP solution, you'll also have to customize their data warehouse package.

Remember, their data warehouse package is an ERP-centric solution. First, how much of your data is not from the ERP system? If it is a significant amount, you will need to spend a lot of resources to extend their solution to include these other data sources. Otherwise, you are merely implementing another stovepipe, in this case centered on your ERP system. The reality may be that the ERP vendor's data warehouse package may become yet another data source for your overall data integration effort.

In addition, examine the toolset the ERP vendor uses to implement their data warehousing package. Does it match you current toolset? If not, there is a cost to integrate and maintain it, and also to train your people. Does your ERP vendor understand your business intelligence needs or mainly your operational reporting requirements? If it's only the latter, you have a lot of work to do. The ERP vendor's solution may be a part of your overall solution, but is not the solution.

Data Integration Myth #7: A dashboard is going to solve all our problems.

IT people tell me they're implementing dashboards, expecting them to expand the use of their data warehouse/data mart environments and unleash the terabytes of data that they have in been collecting over the last few years. They tell me they have all the data the business will ever need and the one key that has been lacking is the dashboard. You might need a dashboard, but it's not a silver bullet. If you don't make it part of an overall comprehensive architecture, you're squandering your budget. Plan your company's enterprise performance management (EPM) scheme first, and then worry about whether or not you need a dashboard.

Data Integration Myth #6: We can do this project without the help of the line of business.

You think so, do you? If you don't work with the business users, how are you going to know their real needs? How will you understand what their data really means and how they use it? They may have issues with data consolidation and manipulation that you couldn't possibly imagine on your own. Do you really know what they're doing squirreled away in their cubicles, staring at Excel spreadsheets? You should You can't solve a problem if you don't know it exists.

Data Integration Myth #5: Because this project will benefit the business users, there's no need to market it to them.

It's tempting to ignore this. However, the promise of business results is what gets a project staffed and implemented; therefore, an internal awareness and marketing campaign is essential. Even after the project is approved and budgeted, it's important to get business sponsors, users and IT staff excited and involved. Your data warehouse might be viewed as successful, but the budgets can still be cut because of the business perception that it is "good enough."

Meet with your business users and sponsors to determine the perception of your data warehouse program and future expectations. You may need to deliver presentations, workshops or training sessions that sell the business benefits of the data warehouse program in the context of your industry and company.

Data Integration Myth #4:If you build the "right" data integration solution, everyone will use it and the stovepipes will go away.

You can have a great architectural solution, but if it doesn't meet the business users' exact needs, they won't give up their trusty stovepipes. You can fight the stovepipes, ignore them or understand them. You should try to do the latter and help leverage your solution to benefit the business users. There may be things you missed or have not gotten to -- that's okay, but try to figure out how your solution can be expanded to avoid creating more stovepipes. Also, recognize that spreadsheets are going to be used by the business as a primary analysis tool. They should be part of the enterprise solution.

Data Integration Myth #3: The CEO and CFO are getting all the information they need.

Sure they are, but at what cost? When one of them asks a question, their staff and others run off, gather data, manipulate it and produce a spreadsheet. Is the information accurate? With the recent passage of the Sarbanes-Oxley Act, companies have no choice but to push for transparent data -- can you trace it back to its source and explain all its derivations? How much time was spent hunting, gathering, reconciling and manipulating the data versus analyzing it?

Data Integration Myth #1: We can't afford a data integration solution

I hear large corporations admit they spent millions of dollars on data integration projects such as data warehouses, data marts, enterprise resource planning (ERP), and corporate performance management (CPM) in the 1990s. However, their business users are still gathering data and using spreadsheets for reporting and analysis. Smaller companies say the software is too expensive and requires more IT resources than they have.

The simple fact is that your business is making decisions every day based on whatever information it has. To get that information, people are bypassing the data warehouse and spending their time searching for data, gathering, consolidating and reconciling it, and guessing that it's accurate.

Those data integration investments did not yield the single version of the truth; they gave you many conflicting versions of it. The issue isn't that you can't afford a data integration solution - it's that you can't afford not do it the right way.