Business Intelligence & Analytics

Business Intelligence & Analytics Tier 3 Performance IT Productivity

In the post Three Tiers for Measuring Business Intelligence & Analytic Performance, I described my views that Business Intelligence & Analytics contribute to organizational performance with three different levels of importance.

  • Tier 1: Business Metrics — Are we contributing to profitable revenue growth?
  • Tier 2: User Productivity — Are we impacting the user productivity?
  • Tier 3: IT Productivity — Are we more efficient in our IT operations?

In this post, I want to explore the third tier of metrics that answer the question ‘Are we more efficient with our IT operations?’.Productivity gains can be achieved by better serving the users, providing results quicker, or reducing IT spending. First; I describe the technical environment for Business Intelligence (BI), and then we can explore where efficiency gains might be achieved.

Business Intelligence & Analytic Technology and Process

Business Intelligence includes technology and an information process. The information process involves collecting and combining data and transforming it from raw data to ready-to-use for decision-making or performance management. The data — structured and unstructured — is collected from a variety of input sources, analyzed to address specific business topics, and then provided to the business process for use in decision making and management. The process includes many different types of technologies’ reporting, data warehousing, data quality, data mining, and performance management. Therefore, delivering the end results to the business requires integrating many different types of technologies, manipulating data, and providing the users with a selection of reporting and analytical options. Table 1 describes the some of technologies involved in BI.

Collect Analyze Use
Data warehousing Data mining Query and Reporting
Extract-Transform-Load, ETL Statistics Online-Analytical-Process, OLAP
Database Text mining Performance management
Operational Datastore Forecasting Analytical solutions
Operational Systems Excel Dashboard
Master data management Online-Analytical-Process, OLAP Portals
Data quality Visual Analytics Operational Systems integration
Data Modeling Excel
Social Media

Table 1 BI & Analytic Technology

In 2011 research on the use of technology in decision making, we discovered that most organizations use 5 BI technologies from 3 different vendors. (Miller, Queisser, Goettsche)

Where does the money go?

Business Intelligence spending is a sub-set of the IT spending and it consists of the labor, capital, and maintenance expenses for the technology platform and the information process. According to the 2012 TDWI BI Benchmark Report: Organizational and Performance Metrics for Business Intelligence Teams report, BI teams are 6 percent or less of the overall IT team size, and for more than 70% of the organizations BI capital spending and BI maintenance budgets are 10 percent or less of the overall IT capital spending and maintenance budgets.

That translates to BI spending and headcount of 2.7 Million USD and 29 people for an organization with revenues of 760 Million USD and 9,000 employees. BI spending, includes labor, hardware, software, networking and related infrastructure costs; BI maintenance includes the costs for labor, services, and licensing to support or further develop the BI environment.

Decision-making and management are dynamic activities. There are constant changes to source systems and IT technologies. These dynamics result in the need for periodic changes in ETL and reporting processes. Consequently, BI support is an on-going operational activity without a single start and stop date like other IT projects. In fact, a third of the time the data needed for reporting is not in the data warehouse.(TDWI) Several other facts related to BI are as follows.

  • The most prevalent roles in BI teams are the Extract-Transform-Load (ETL) and report developers. (TDWI)
  • Most organizations use 19% of their total resources for a new BI project on data integration. (Aberdeen)
  • 40% to 60% of data is unstructured (Gartner)
  • The teams spend 48% of their time in development and testing, 24% maintenance /change management, 15% support/training, and 14% in other activities. (TDWI)
  • The BI development teams handle support tickets for 49% of the organizations. (TDWI)

What are some performance measures?

As described in previous blog posts, the best metrics for the BI environments are those that demonstrate a clear connection between the BI solutions being delivered and their financial benefit to the organization.

Gartner recommend measuring IT spending changes to revenue changes to gauge the relationship between IT investments and business results. That is, if the business revenue is growing or declining at a specific rate, is the IT spending moving in the same direction? Is it leading the change? Is it lagging the revenue change? This measure can be used to evaluate the role that IT plays in business evolution.

Table 2 includes some potential performance measures.

Topic Measures
IT relationship to the business
  • IT spending change compared to revenue change
Servicing the users
  • Number of training sessions, workgroups, events
  • Meeting services levels (Turn-around time for requests)
  • Number of consulting days/engagements to support BI usage
  • Increase in customer satisfaction
  • Number of queries/reports executed by the users
  • Number of employees licensed to use a bi tool
  • Number of employees running a reports at least once a week
  • Number of queries/reports run by a customer or supplier
  • Number of queries/report executed daily
Adaptability of BI & Analytic environment
  • Duration per project
  • Time to add a new source of data to the data warehouse
  • Time to building a complex or dashboard report
  • Time to changing a hierarchy
  • Time to adding a new column to a report
  • Time to creating a new dashboard
IT Operations
  • IT costs as a percentage of operating revenue
  • Cost per user
  • Budget/chargeback objectives
  • Hardware Costs
  • Software Costs
  • Employee satisfaction — reduced overtime/more value add work
  • Number of help tickets received each month

Table 2 Example BI & Analytic Performance Measures

The increases in measures for servicing the user provide indicators to understand if IT is offering the right services and supporting to the business. Measures that judge the adaptability of the BI environment help to understand if the team is applying an sufficiently agile software development methodology, if the technology is easily adaptable, and if the technical debt (maintainability) is in check. Increases in these measures would be an indicator of problems or opportunities for improvements. The IT Operations measures must be balanced with the organizational objectives. That is, a decrease in software costs but a corresponding increase in employee overtime, would be a negative indicator.

Becoming more efficient

There is no silver bullet to becoming more efficient with your IT operations in regards to business intelligence. Some of the previously recommended methods like ‘standardize BI to a single vendor’ have proven to reduce IT costs of operations, but increase shadow IT operations — IT systems and solution that are built and used without the IT organizations knowledge or approval.

The following are some examples of BI projects that improved the IT performance.

Hardware migration: The organization wanted to cut down its IT budget by 13% (20 million). The business analyst also face performance issues as they did not receive priority for the computing power that they needed. One strategy chosen was to move part of the analysis and reporting system from the MVS to an NT server and to start using automated tool for ETL management. This strategy reduced the operating cost for the mainframe by reducing the millions of instructions per second (MIPS) and the resulting licenses costs. In addition, the analysts got their own decentralized operating environment.

Data quality processes. The goal was to improve customer communication and avoid operational problems. The problem resulted from poor quality in a large, diverse, distributed customer database. The project delivered a web-based solution that provides data quality scores from a high-level customer view down to specific customer data attributes. The business rules are organized by account managers, screen navigation is based upon the organizational hierarchy and alters notify managers of business rule violations.

Automating data processes. This project focused on delivering a solution to address credit card fraud at the point of purchase. The bank was detecting fraud through a semi-manual process involving a number of Business Analysts. This was time consuming and resulted in some fraud events being detected well after the event. The solution was focused on automating the steps to a daily automated process, from extracting data out of the relevant source systems, processing the data, and then automatically applying the fraud rule logic and generating daily alerts.

Modernize data integration. The company’s objectives was to improve the quality of analysis, bring products to the market faster, and improve the quality, cost and speed of the product development process to maintain and improve competitiveness. The BI project was to standardize the customer’s data, analysis and reporting programs. The business operational process was standardized by another part of the project. The following was achieved:–Organize the overall flow of analysis operation–Standardize the data and programs–Build an integrated platform for information sharing (data integration, accessibility improvement, centralized management of data and programs and regulatory compliance, which enables efficient recycling and progress management)

For getting started, track a few of the measures and benchmark your organization’s performance. Research reports from IT analyst such as TDWI, Aberdeen, IDC, and Gartner offer a selection for statistics that can be used to evaluate performance of similar organizations. Once the baseline is established, then select your next course of action based on the areas of weakness.

In our research on Business Intelligence Competency Centers [2], we have noted that organizations that implement a BICC or similar entity, have reported benefits with servicing the business and reducing delivery times and BI costs.  Figure 1 provides a view on the reported benefits. (Miller, Queisser, Gottsche)

Figure 1 BICC Benefits

Examples of actions to reduce the Total Cost of Ownership for BI & Analytic environments and increase performance include those described in Table 3.

Topic Actions
IT relationship to the business
  • See the blog posts on Tier 1 and Tier 2 performance
  • Implement a BICC
Servicing the users
  • Provide a selection of BI tools for different user profiles
  • Assign different levels of computing power based on user profiles
  • Have a capability to provide data when needed on an ad hoc basis
  • Encourage communities of practices
  • Establish methods to gain feedback from the users
  • Encourage communities of practices
  • Provide training programs, user groups, certification programs, and consulting to support the users in understanding the BI tools, the methods and techniques, and ways of interpreting the results
Adaptability of BI & Analytic environment
  • Use modern, standard software to automate data integration and data quality processes
  • Consider alternative BI deployment strategies — e.g., on-premise, hosted/Cloud, appliances — to offer additional functionality
  • Use agile software development methodologies
IT Operations
  • Review software licensing and deployment options for suitability, including per user, per server, concurrent, subscription, open source, or data model pricing options
  • Review hardware options to reduce processing windows, increase computing power, and improve price performance; consider appliances, grids, massively parallel systems, in-memory options

Table 3 Example Performance Improvement Actions

Achieving an efficient IT operations while continuing to service the business is a very hard thing to do. My advice is focus on the business, empower and enable the users to service themselves, and seek innovations, modernizations, and automation as a way to improve IT efficiency.


Hatch, David. Managing the Total Cost of Ownership for BI: The four hidden costs and how to avoid them. Analyst Report, AberdeenGroup, 2009.
Miller, Gloria J., Queisser, Thomas, Goettsche, Thomas. Mobilizing Technology and Teams for Better Decision Making, May 2011
Potter, Kurt, Michael Smith, Jamie K. Guevara, Linda Hall, and Eric Stegman. IT Metrics: IT Spending and Staffing Report, 2011. Analyst Report, Gartner, 2011.Stoddler, David. Achieving Greater Agility with Business Intelligence. TDWI Best Practices Report, TDWI Research, First Quarter 2013.The Data Warehousing Institute. 2012 TDWI BI Benchmark Report: Organizational and Performance Metrics for Business Intelligence Teams. TDWI Research, Renton, WA: The Data Warehousing Institute, 2012.


[1] The sample size was 410 participants.
[2] The sample size was 62 participants.