IM - Data Integration

Testing ETL projects - used for Extracting data, Transforming it according to reporting requirements and Loading it into the data warehouse - is very complex because of many reasons. Typically huge amounts of data are handled and data comes often from multiple sources that need to be combined. Sometimes, the data is loaded incrementally, sometimes it needs to be reloaded in full. These ETL-flows can also apply all kinds of complex calculations and transformations on the data.

Introduction Go to Google and type in the words "Why do BI projects fail ?".

Dit recent gepubliceerd boek biedt een bijzonder volledig overzicht van de materie van Performance

SQL Server Data Quality Services (DQS), new since SQL Server 2012, is a knowledge-driven data quality product.

Microsoft SQL Server Data Tools is introduced with SQL Server 2012 and is the successor of the Business Intelligence Development Studio (BIDS). It is a, Visual Studio based, integrated development environment, used for all your BI-related development.

IBM InfoSphere Business Glossary is a software solution providing an enterprise data dictionary. It enables the creation, classification and management of the organizational vocabulary and business definitions to provide better alignment between business and IT users.IBM Business Glossary :

IBM Information Analyzer helps you quickly & easily understand data – its structures, formats, relationships and quality issues. Information Analyzer provides inherit methodology to analyze large volumes of data, with pre-built information queries that help you quickly understand how the data is constructed, to pinpoint quality challenges, and to plan integration processes with higher confidence.

IBM InfoSphere DataStage is a leading Extraction-Transformation-Loading (ETL) tool, offering parallel processing and integration for high volumes of data sources and target applications. DataStage includes large library of built-in transformations and an internal programing language (BASIC) to support simple-to-highly complex transformation logic to the data, as it goes through it.IBM InfoSphere DataStage:

DB2 Connect is a combination of application interfaces (APIs) and network infrastructure components that allow one to make SQL requests on a Windows, Linux, or UNIX machines against databases running on IBM System z (mainframe) and System i (AS/400). In other words, the data resides on the mainframe while application code that uses the data is deployed to distributed systems like a Windows laptop or a server.

Data integration is the collection of processes and tools, strategies and philosophies by which fragmented data assets are aligned to support business goals. Data integration from a technical point-of-view covers topics such as data federation, virtualization and service-oriented-architecture, but in the context of Performance Management or Business Intelligence, it is usually synonym to ETL (Extract-Transform-Load) or ELT-related activities.