Data Doghouse - performance management, business intelligence, and data warehousing

Suggestion #5 for the analytical sandbox: solutions must be cost- and resource-effective

When creating analytical sandboxes for business users, keep the following design principle in mind to help ensure you  provide the right environment for your enterprise:

Solutions must be cost- and resource-effective - All enterprises need to operate within budgetary and resource constraints, whatever that means for their size and industry. The solution should be to be right-sized to meet the enterprise’s data and analytical needs along with the resources and skills that will sustain the solution.

To read four more suggestions for analytical sandbox architecture plus detailed architecture options and advice, see the white paper Analytics Best Practices: The Analytical Sandbox.

Suggestion #4 for the analytical sandbox: sufficient infrastructure must be available for conducting business analytics

When creating analytical sandboxes for business users, keep the following design principle in mind to help ensure you  provide the right environment for your enterprise:

Sufficient infrastructure must be available for conducting business analytics - The infrastructure for an analytical sandbox includes:

• Processing, such as PCs and servers (physical, virtual and cloud)
• Storage (physical and cloud)
• Integration capabilities (physical and virtual)
• Self-service BI tool(s)

This infrastructure must be scalable and expandable as the data volumes, integration needs and analytical complexities naturally increase.  Insufficient infrastructure has historically limited the depth, breadth and timeliness of analytics as business people used their PC and spreadsheets to fill shortfalls.

To read four more suggestions for analytical sandbox architecture plus detailed architecture options and advice, see the white paper Analytics Best Practices: The Analytical Sandbox.

Suggestion #3 for the analytical sandbox: the business analyst needs to be “in control”

When creating analytical sandboxes for business users, keep the following design principle in mind to help ensure you  provide the right environment for your enterprise:

The business analyst needs to be “in control” - IT has traditionally managed the data and application environments. In this custodial role, IT has controlled access and has gone through a rigorous process to ensure that data is managed and integrated as an enterprise asset. The time has arrived when business analytical elite need to assume data ownership, get access to data from across the enterprise, and augment that data with other data that that they feel is appropriate -- all of their own volition. This does not mean they should abandon data governance and data quality efforts, rather, they should leverage them in the proper business context, i.e. when recurring, production-quality information is necessary.

To read four more suggestions for analytical sandbox architecture plus detailed architecture options and advice, see the white paper Analytics Best Practices: The Analytical Sandbox.

Suggestion #2 for the analytical sandbox: time-to-solution must be fast and disposable

When creating analytical sandboxes for business users, keep the following design principle in mind to help ensure you  provide the right environment for your enterprise:

Time-to-solution must be fast and disposable - Today’s competitive business environment and fluctuating economy are putting the pressure on businesses to make fast, smart decisions. Analysts using the sandbox need to be able to gather the data, combine it, analyze it, and then act upon the resulting insights -- fast. The analytical elites can no longer accept analysis that is delayed by days, weeks or even months as they wait for their requests to make it through BI and data-integration backlogs. Tools, data and infrastructure need to be architected to ensure that ad-hoc analysis can take place when it is needed by the business.

To read four more suggestions for analytical sandbox architecture plus detailed architecture options and advice, see the white paper Analytics Best Practices: The Analytical Sandbox.

Designing the analytical sandbox: data across the enterprise needs to be accessible and timely

When creating analytical sandboxes for business users, keep the following design principle in mind to help ensure you  provide the right environment for your enterprise:

Data across the enterprise needs to be accessible and timely - Business analytics is inhibited by the difficulty in accessing data across an enterprise and by the length of time it takes to get that data integrated. Business needs to operate and react to constantly changing conditions, so timely access to data scattered across an enterprise is necessary to make more informed decisions, even if the data in not “perfect.” The analytical sandbox needs to enable timely access across data silos and provide business people with an integrated view of the best data that is available at the time of analysis. This view will be a mix of physically or virtually integrated data to expedite time-to-analysis and avoid the productivity and error-prone trap of data shadow systems.

To read four more suggestions for analytical sandbox architecture plus detailed architecture options and advice, see the white paper Analytics Best Practices: The Analytical Sandbox.

Analytics Best Practices: The Analytical Sandbox

So this situation sounds familiar, and you are wondering if you need an analytical sandbox...

The goal of an analytical sandbox is to enable business people to conduct discovery and situational analytics.  This platform is targeted for business analysts and “power users” who are the go-to people that the entire business group uses when they need reporting help and answers. This target group is the analytical elite of the enterprise.

The analytical elite have been building their own makeshift sandboxes, referred to as data shadow systems or spreadmarts. The intent of the analytical sandbox is to provide the dedicated storage, tools and processing resources to eliminate the need for the data shadow systems.

The key components of an analytical sandbox are:

• Business analytics - contains the self-service BI tools used for discovery and situational analysis
• Analytical sandbox platform - provides the processing, storage and networking capabilities
• Data access and delivery - enables the gathering and integration of  data from a variety of data sources and data types
• Data sources – sourced from within and outside the enterprise, it can be big data (unstructured) and transactional data (structured); e.g., extracts, feeds, messages, spreadsheets and documents.

For more on analytical sandboxes, including architecture design principles and options, see the white paper Analytics Best Practices: The Analytical Sandbox

Does your business need an analytical sandbox?

Is your enterprises flooded with a deluge of data about  customers, prospects, business processes, suppliers, partners and competitors? Is data coming from traditional internal systems, cloud applications, social networking and mobile communications?

With this flood of new data comes the opportunity for business people to perform new types of analysis to gain greater insight into their business and customers.

The opportunity, however, comes with new challenges. Performing business analytics used to mean using pre-defined reports.

But now, with the flood of data and constantly-changing business environment, people don’t know what they need ahead of time, so pre-defined reports aren’t relevant. Instead, people need to make new queries based on what is happening right here, right now.  As a result, business analytics has to be “situational,” that is, it needs to respond to rapid changes in the business, economic and competitive environment.

The change in analytics means changes for IT. Traditionally, IT received detailed BI requirements and then created reports. Because this approach is not meeting the needs of discovery and situational analytics, we need a new approach.  

The answer is analytical sandboxes, a new paradigm that address the multiple-query challenges of situational business analytics and avoids the pitfalls of the makeshift data shadow systems.

For more on analytical sandboxes, including architecture design principles and options, see the white paper Analytics Best Practices: The Analytical Sandbox.

5 Principles of Analytical Hub Architecture - Part 2

Continuing the discussion on analytical hub design, here's the second part of my post on the architecture principles. If you missed the first two principles (1. Data from everywhere needs to be accessible and integrated and 2. Building solutions must be fast, iterative and repeatable) see this earlier blog post.

3. The advanced analytics elite needs to “run the show”

IT has traditionally managed the data and application environments. In this custodial role, IT has controlled access and has gone through a rigorous process to ensure that data is managed and integrated as an enterprise asset. The enterprise, and IT, needs to entrust data scientists with the responsibility to understand and appropriately use data of varying quality in creating their analytical solutions. Data is often imperfect, but data scientists are the business’s trusted advisors who have the knowledge required to be the decision-makers.

4. Solutions’ models must be integrated back into business processes

When predictive models are built, they often need to be integrated into business processes to enable more informed decision-making. After the data scientists build the models, there is a hand-off to IT to perform the necessary integration and support their ongoing operation.

5. Sufficient infrastructure must be available for conducting advanced analytics

This infrastructure must be scalable and expandable as the data volumes, integration needs and analytical complexities naturally increase.  Insufficient infrastructure has historically limited the depth, breadth and timeliness of advanced analytics as data scientists often used makeshift environments.

Read more about this in my free white paper on Analytic Data Hub design entitled Analytics Best Practices: The Analytical Hub.

5 Principles of Analytical Hub Architecture - Part 1

As I discused the other day in Why you need an analytical hub, enterprises need to spend time looking forward, rather than just backwards at historical data. The analytical hub is an important part of making that happen.

The analytical hub must be designed properly if it's going to allow data scientists to perform advanced analytics and predictive modeling.

In my white paper, Analytics Best Practices: The Analytical Hub, I present five design principles. The first two are below. I'll blog about 3-5 in a subsequent post:

1. Data from everywhere needs to be accessible and integrated in a timely fashion

Expanding beyond traditional internal BI sources is necessary as data scientists examine such areas as the behavior of a company’s customers and prospects; exchange data with partners, suppliers and governments; gather machine data; acquire attitudinal survey data; and examine econometric data. Unlike internal systems that IT can use to manage data quality, many of these new data sources are incomplete and inconsistent forcing data scientists to leverage the analytical hub to clean the data or synthesize it for analysis. 

Advanced analytics has been inhibited by the difficulty in accessing data and by the length of time it takes for traditional IT approaches to physically integrate it. The analytical hub needs to enable data scientists to get the data they need in a timely fashion, either physical integrating it or accessing virtually-integrated data. Data virtualization speeds time-to-analysis and avoids the productivity and error-prone trap of physically integrating data.

2. Building solutions must be fast, iterative and repeatable

Today’s competitive business environment and fluctuating economy are putting the pressure on businesses to make fast, smart decisions. Predictive modeling and advanced analytics enable those decisions to be informed.  Data scientists need to get data and create tentative models fast, change variables and data to refine the models, and do it all over again as behavior, attitudes, products, competition and the economy change. The analytical hub needs to be architected to ensure that solutions can be built to be fast, iterative and repeatable.

Read more about this in my free white paper on Analytic Data Hub design entitled Analytics Best Practices: The Analytical Hub.

Why you need an analytical hub

Enterprises are flooded with a deluge of data about their customers, prospects, business processes, suppliers, partners and competitors. It comes from traditional internal systems, cloud applications, social networking and mobile communications.

With the flood of new data comes the opportunity for business people to perform new types of analysis to gain greater insight into their business and customers.

Enterprises have been expanding their traditional BI footprint to provide much more comprehensive and timely reporting for their business. These investments are valuable, but they are limited to analyzing how a business has performed historically.  Looking beyond historical data, there’s a significant business opportunity in analyzing what the future may hold, e.g., predictive modeling, or examining customer behavior from sources outside the enterprise, e.g., social media. 

This shift to forward-looking analytics dictates changes both for the business and IT. Traditionally, IT received detailed data requirements, used ETL tools to extract data and load it into a data warehouse (DW), and then provided business people with read-only access to that data.  

It’s a long process – too long for people working with advanced analytics (we call them data scientists). They typically do not know all the data they need until they start modeling, and need great flexibility for processing data.

IT needs to change to a supporting role with an analytical hub, and relinquish control to the data scientists. IT needs to understand that data scientists are much more data savvy than traditional BI users, and can be trusted with data.

Read more about best practices for the analytical hub in this white paper.