Data Doghouse - performance management, business intelligence, and data warehousing

People, Process and Politics - We All Hate Data Silos, So Why do They Happen?

Why are data silos so prevalent when we really ought to know better? We know that it’s risky to have inconsistent data. We strive for the “single version of the truth” along with our IT and high tech vendors.

So, with all this awareness, why does almost every company end up with data silos?

There are two causes that I’ve seen in my consulting work. First, companies typically organize BI and DW related projects on a tactical and project-by-project basis. This makes data silos very, very likely. Second, these tactical projects are designed without an overall information architecture blueprint. This absolutely, positively guarantees a silo. You “can’t get there from here” if you don’t know where you are trying to go.

There are several reasons why otherwise smart people create projects on a tactical basis without an overall architecture:

Funding - Projects are funded by a group representing an organization or business process. These groups are focused mainly on their own information needs and delivering business value in the context of those needs. Building an enterprise-wide solution is not in their minds nor in their charter.

Confusion - Industry jargon creates the impression that things are different, when they’re actually similar. There are many similarities between CRM (customer relationship management), SCM (supply chain management), budgeting and forecasting, performance management and balanced scorecards. Many people do not see that each has data, data integration and business intelligence that can be shared across projects.

Technology - People tend to treat different technologies as different projects and applications. ETL (extract, transform and load), SOA (service oriented architecture), EAI (enterprise application integration) and data virtualization are considered different and treated as separate entities rather than different methods to implement data integration.

Disconnects - Data warehousing and enterprise applications are treated as separate worlds. Even though the business person may need reporting from operational applications and the DW, different projects are created to implement reporting from each of these data sources. And often IT splits its ERP and DW people into separate groups, further perpetuating silos.

No Architecture  - Finally, without an information architectural blueprint how is anyone to know that these projects should be connected? If you split up building a house amongst various contractors and specialists such as electricians and plumbers without a blueprint and without these groups talking to each other it is unlikely the house gets built correctly. It’s the same with a DW or BI project. You need the blueprint (information architecture) - especially if the projects are set up tactically and on a project-by-project basis. This is called accidental architecture and it creates data silos.

Data silos result in poor quality data that is often inconsistent, wastes resources and time on overlapping and redundant projects, costs more money to build and maintain, and ultimately results in the business not getting information in a consistent, comprehensive and current manner. Data silos also encourage the creation of data shadow systems or spreadmarts, further exacerbating the problems.

How do we get out of this? Stay tuned for more posts in the People, Process and Politics series.

People, Process & Politics: Data Governance

For those of us in high tech, it’s easy to fall into the trap of concentrating too hard on the products, technology and architectural aspects of solutions. I’ll admit I’m a bit of a nerd and many people in IT feel comfortable with technology too. It also goes beyond IT; in today’s society people often assume technology is going to solve their problems.

There is a tendency, especially in the US and Europe, to believe that technology can solve the difficult problems that we face. Technology, especially in corporate settings, is viewed as the cure-all or silver bullet that overcomes all the obstacles that have been encountered with data in the past.

But the problem with our comfort level and faith in technology is that often the critical success factors of a solution lie with people and policies. And often there’s a little bit of politics thrown in to keep things interesting. Very interesting…

So, if you are embarking on or trying to revitalize enterprise information, you’ve got to put the people and the policies on your front burner. There are several areas that you need to consider:

•    Data Governance
•    Program & Portfolio Management
•    Integration Competency Center (CC)
•    BI Competency Center (CC)
•    Skills Assessment & Training

Let’s start with data governance.

I see it with my clients and you surely see it in your business: you’re striving to better use data to improve the top line by increasing customers, revenue, and profit.  Or, increasingly important in today’s economy, you’re examining operational performance to impact the bottom line by reducing costs. Add in compliance and privacy drivers and there is a compelling business case for enterprises to manage their data as a corporate asset.

Many of us have heard the “single version of the truth” pitch from enterprise application vendors or from people trying to justify business intelligence (or performance management or MDM or SOA or…) initiatives. These pitches all involve a massive dose of technology and products to solve the data problems you have never been able to in the past. Do not get lulled into the product hype or the silver bullet solutions.  CapEx (capital expenditures) give people the feeling that progress has been made, but the investment will be wasted without the proper amount of people, process and politics.

Regardless of what tools you use, you have to define, transform, measure, monitor, manage and then act upon it. Despite their best intentions, IT does not own data;  rather, the business does. Business responsibility for defining the data is essential to obtain trusted and actionable information.
Data is created and then transformed as it flows through various business processes throughout an enterprise. Ownership and the responsibility to define the data and its transformations are political. IT often is wary of the politics, but there is no way around it to be successful. Business ownership can get IT out of data politics and into their proper roles of enablers and stewards for corporate information. 

If you get over the political hurdle then IT has to avoid the “Goldilocks and the three bears” syndrome. The governance processes cannot be either too hot (complex) or too cold (not comprehensive enough). They need to be just right. It will likely take a few iterations to get the processes to be constructive rather than obstructive.

Getting the right people involved in data governance is another stumbling block people encounter. Too often the people involved are too senior and too removed to guide the processes. The other extreme is that the business people involved are at too low a level in the organization to have enough authority to make decisions that others will follow. This, like processes, may also take a few iterations to get right.

Data governance is neither simple nor a one-time event. Do not underestimate time commitments by both business and IT people. Too often governance is thrown on top of workers’ existing workloads. Failing to allocate sufficient time people to work on data governance is recipe for project failure.

It’s people, process and politics that make the shiny tools succeed.

Business Intelligence (BI) & On-Demand Software Indexes: Sprinting Ahead of the Pack

Market headlines have noted the milestone of the S&P 500’s year-to-date (YTD) performance moving into a positive gain with Tuesday’s market gains. That is terrific news but the Business Intelligence & Data Warehouse (BI/DW) Index and On-Demand or SaaS (software as a service) software index have posted approximately 20% YTD gains (as of yesterday’s market close.)

These indexes have been swept up in the general surge of high technology stocks with the NASDAQ up 11.2% YTD and software companies, as represented by the iShares S&P North American Technology-Software Index Fund, are up 16.3% YTD. However, both the business intelligence and on-demand software indexes have outpaced these high tech benchmarks.

Business Intelligence (BI) is not a luxury

Business intelligence (BI) firms (this includes data warehousing (DW), data integration and performance management categories) are performing better than the overall market because their products and services continue to be in demand in tough economic times. The recession and IT spending cuts are reducing BI firms' growth rates, but companies still need to monitor, measure and manage their businesses.

In good times BI is oriented towards expanding business by obtaining new customers or expanding what a customer buys. In tough times, orientation shifts towards operational efficiencies and cost cutting with BI being one of the enablers of those actions. IT budget reductions do impact new projects in almost every category of software, but BI has moved from being a “nice to have” to a critical enabler of many business initiatives within corporations today.

Business intelligence has become mainstream in Fortune 500 companies and is “crossing the chasm” in the small-to-medium size businesses (SMB). Many of these potential BI customers are faring better than their larger brethren. This provides an expanding market during tough times for BI vendors.

From an investor perspective, the index is dominated by software firms that generally have no debt, generate strong cash flows and receive steady maintenance income. This separates them from many industries that are impacted by the financial implications of the credit crunch. BI vendors’ customers may be impacted by these financial issues, which will reduce spending but it does not directly impacted the business operations of the BI firms themselves.

The Laggards

All BI firms are not fairing as well as others. The groups that are at the bottom of the performance rankings are: industry research firms and high tech titans.

The industry research firms, Gartner (IT) and Forrester (FORR), performed well (relatively speaking) last year as investors felt their business shielded them from the recession and IT spending cuts. This year though as these firms continually reduced their IT spending forecasts the reality set that their business was indeed correlated to their customers’ IT spending plans. If IT customers are postponing or cancelling major projects advisory consulting seems to be a nice-to-have. The research firms are also hit with some IT vendors are reining in their marketing spending. Finally, IT conferences are seeing a drop-off in attendance and vendor sponsorship further crimping revenue and prospect lead generation.

There are two groups of high tech titans that are also at the bottom of the performance rankings year-to-date: consumer-centric and enterprise application vendors.
HP (HPQ) and Microsoft (MSFT) are feeling investor concerns about retrained consumer and enterprise business spending. PC-related hardware and software related sales including printers sales are tempering growth at both firms with both using cost-cutting to maintain profits. Reductions or delays in business spending on server and storage sales are an additional overhang for HPQ.

SAP and Oracle (ORCL) both have solid maintenance revenue and professional services income streams but enterprise application growth is linked with the “big deal” that is at best delayed and often postponed (cancelled) until next year. Oracle does have the database side of its business that is not as dependent on the large enterprise deal. 

What's Next

The question is does the bull market or bear market bounce continue or take a breather? We are entering into a traditionally slower part of the calendar year for major purchases so major upside surprises are not likely to be prevalent until later in the year. The stocks may continue to trade on momentum or general market sentiment but fundamentals do not yet justify a major move from these levels.