IBM Cognos TM1 & Cognos 8 BI

Introduction: What does the improved integration between Cognos 8.4 BI and TM1 have to offer

IBM Cognos TM1 is a relative new addition to the IBM Cognos software stack, following Cognos' acquisition of Applix in 2007, just prior to its own acquisition by IBM. The core TM1 solution is a real-time, interactive multidimensional OLAP database with in-memory and write-back functionalities, and is therefore perfectly suited to support all sorts of Performance Management (PM) applications.

Although IBM Cognos have not released any official news or statements about this yet, it can be expected that TM1 will become the core multidimensional engine in the next major release(s) of the Cognos platform, both for OLAP reporting/analysis purposes, as for financial analytic applications like planning, budgeting and forecasting (P, B & F) and consolidation. As such it could increase scalability for the CPM suite and become a mean for further integrating the CPM applications between them and within the BI platform as a whole.

Today, TM1 9.4 can already be used as a data source in Cognos 8.4 BI. This means all Cognos 8 Studio's, including Report, Analysis & Metrics Studio can access TM1 data. Moreover, TM1 includes accompanying application products that support financial and operational performance management capabilities or components.

In this insight we will try to explain the TM1 architecture and the added value of integrating TM1 with Cognos 8 BI.
 

TM1 OLAP engine

TM1 is a multidimensional database engine, based on a considerably different architecture than the current Cognos' PowerCubes (Powerplay), the main two differences being that TM1 is a memory-resident OLAP database and TM1 offers interactive read/write functionality.

The in-memory functionality allows for faster and more scale-able OLAP than traditional disk-based ROLAP (Relational) or MOLAP (Multidimensional). A complex relational query, in the most common limitation, can't be answered in real time, and the data explosion problem makes disk based multidimensional implementations with medium to large data volumes difficult to impossible.

RAM access speeds are up to 1 million times faster than random reads on disk. Modern hardware evolutions, like the advent of 64-bit chips, have paved the way for memory-resident solutions offering better performance (reduced user wait times), more functionality, and in most cases at a lower cost. (Source: Monash, Memory-centric data management).

The read/write functionality means that the cube technology can also be used to capture data from the end-user. This is not really useful in a reporting environment and traditionally also unwanted in a classical BI architecture, but is obviously something that is key in a Planning, Budgeting and Forecasting solution, in Simulation applications or in e.g. a Scorecard Application to capture KPI target values.

One might think of the combination of RAM-based, read/write technology as "spreadsheet like” interactivity, except that the data is off course held in one central location and that the solution is able to support massive data volumes. All calculations/aggregations are done on-demand and in memory (results are of course cached and reused to answer repetitive queries). In spite of the ability to cache, most, if not all, other BI products require pre-calculation in order to provide reasonable performance. When a change is made, the pre-calculation has to be repeated, resulting in delays. TM1's approach is to calculate everything on demand, at very high speed, thus reflecting the result of changes immediately, without any downtime for the user (zero-latency).

This architecture makes dynamic analytics (what-if analysis) possible. It allows users to develop complex models based on business data and rules that reflect various derived business performance measures, including projections and forecasts of future business activities. Users can change an input value and immediately view the effect this change has throughout the entire model.

The model interactivity is extremely useful for P, B & F solutions, but also for complex business applications such as risk analysis and activity-based costing/management. It can be used to understand the implications of -for instance- shifts in selling or purchase prices, customer profitability modeling under multiple scenarios, product mix optimization, spotting customer churn patterns, …
 

TM1 user interfaces and application toolkit

Next to the core OLAP engine, IBM Cognos TM1 provides a number of components that make up the user interface and application toolkit.

Architect and Perspectives (an Excel Add-In) are the client applications that let you create, manage, and edit TM1 databases. They are used to build typical multidimensional components like dimensions, cubes and cube views.

Figure 1 : TM1 Server Explorer
 
Dimension elements and cube data can be loaded via the Turbo Integrator Processes. Turbo Integrator can extract and transform data from other cubes, relational databases, Microsoft Excel and flat-files and can be build and executed from within TM1 Architect or Perspectives. It's considered a best practice to load data from an integrated dimensional data-warehouse. Next to loading data, Turbo Integrator is able to maintain the model via functions and script. It can for instance clear data in a cube, move data from cube to cube, create dimension subsets, or save the TM1 data to disk. Chores allow for the creation of batch files that integrate a number of Turbo Integrator processes. These may be scheduled for automatic execution and run off-peak hours.
 
Figure 2: Turbo Integrator Process loading data into a cube
 
 
The modeling language that implements the business logic is called Rules. It can be used to centralize the logic of custom calculations (e.g. a Variance measure representing the difference between Actual and Forecast values), override the default rollup behavior of hierarchical calculations (e.g. Weighted Averages) and share data between cubes (e.g. a conversion rate lookup in a separate cube in order to report on different currencies).
 
Figure 3: Rules Editor defining calculations and lookups
 

TM1 also includes capabilities designed for P, B & F requirements like workflow and spreading. A Workflow process reflects a series of tasks integrated with security in order to support typical Planning, Budgeting & Forecasting processes, like "submit” and "review”. Spreading provides rapid data entry at any level within a cube and allocates this input data to the leaf level elements in proportion to other data elsewhere, for instance "previous year” values or values in a lookup cube containing relative spread patterns.

Microsoft Excel is typically used to develop, author and navigate TM1 applications. A new feature in TM1 9.4 is Active Forms. These allow for standard Excel formatting in a worksheet and to view and update the live TM1 cube data directly in Excel. Action buttons enable navigation to for instance other worksheets and deliver Turbo Integrator process integration. Furthermore, server-side zero suppression improves end-user Responsiveness. Capabilities like workflow and spreading are directly supported from within the excel interface.
 
Figure 4: TM1 Active Form
 
 
TM1 Web is the platform that allows to access these Excel-applications via a thin-client browser environment like Internet Explorer. Web users can also access cube views, or various slices of a cube. Moreover, maintenance tasks such as running chores and processes, or managing security, can be accessed from within TM1 Web.
 
Finally, TM1 Executive Viewer (acquired by Applix from the Dutch company TemTec in 2006) is an end-user reporting tool designed for ad-hoc analysis and visualization. It can be deployed in both thick-client and thin-client modes, and offers capabilities such as pivoting, drilling, charting and conditional formatting. There is clearly an overlap between the Executive Viewer and Cognos 8 Business Intelligence functionalities. Executive Viewer will be positioned in the mid-market edition of TM1, for customers choosing not to integrate with Cognos 8 BI.
 

TM1 Integration with IBM Cognos 8 BI

IBM Cognos 8 Business Intelligence can enhance the deployment of a TM1 implementation by using standard Cognos features like query and reporting, dashboards, analysis, scorecards, scheduling, distribution and alerts. Framework Manager integration enables consumption of TM1 cube data via IBM Cognos 8 BI. This is shown in the diagram below.
 
Figure 5: Cognos 8.4 & TM1 9.4 integration
 
 

Portal integration puts TM1 Web (or Executive Viewer) content in a larger business intelligence context. These are also referred to as Cognos Connection Portlets or TM1 Fragments.

The real-time in-memory facilities of TM1 can be used to input data in TM1 Web fragments (Cognos Connection) and have the data updates immediately reflected by the Cognos 8 BI components. In other words, Planning, Budgeting and Forecasting or other PM applications, developed in TM1, can be accessed from within your BI portal and the BI components can access the TM1 data in real-time.

IBM Cognos 8 security integration within TM1 enables single sign-on and centralized security management.

Cognos 8 BI has very advanced front-end tools for analyzing and reporting on multidimensional data-sources like TM1. For example, Report Studio introduced Express Author Mode in version 8.3, next to the Professional Author Mode which is even more advanced. This "crosstab reporting” mode enables business analysts to report in an intuitive way on multidimensional data sources. For those familiar with Powerplay PC, Express Author Mode corresponds to the functionality of Reporting Mode. Analysis studio, already available since the launch of Cognos 8, on the other hand is more comparable to Powerplay Pc's Explorer Mode.
 

Conclusion

TM1 is definitely a valuable extension to the IBM Cognos product stack. The Cognos 8 BI suite offers traditional Business Intelligence features like Query, Reporting, Analysis, Dashboarding, Scorecarding, Alerting and Scheduling. TM1 is a robust multidimensional engine and application platform for typical financial PM applications like Planning, Budgeting and Forecasting. It is also able to support rapidly changing financial and business analytics that require a real-time response, like "What-if” scenario modeling. In this way the promise of agile Performance Management is achieved.

Although TM1 has the potential to enrich and complement the current IBM cognos platform, it isn't just there yet. TM1 Architect and Perspectives aren't what you can call good GUI tools. There are several little bugs, new windows opening one over another, standard actions that require you to write code. Moreover, the integration of the products is far from complete at this moment. Combining the capabilities of TM1 with Cognos 8 BI and existing financial applications like Cognos Planning (Modelling, Workflow, Contribution) and Controller will form the basis of a complete and integrated Performance Management offering. In an upcoming release we might already expect Cognos Planning front-end tools to be powered by the TM1 database.

Today, TM1 9.4 can be used as a native data connection in Cognos 8.4 BI. This means that the BI front-end tools can already query the TM1 data engine. In this way, TM1 might be positioned as an alternative to boost the performance and scalability problems of traditional multidimensional data-sources like DMR (Dimensionally Modeled Relational), Cognos Powerplay PowerCubes or Cognos Planning models.