elementary for BI & PM Migration Projects

Introduction

In information technology, migration is the process of moving from the use of one operating environment to another operating environment. Moving to a different environment is considered a migration because it involves taking steps to ensure that current applications continue to work in the new environment and making sure that new features are exploited. Migration can involve moving to new hardware or new software or both. Migration can be small-scale, such as migrating a single system, or large-scale, involving many systems, new applications or a redesigned network and webserver architecture.

Migrating a Business Intelligence environment is a complex undertaking nowadays, given the fact that it touches a lot of different technical components. Therefore it requires a lot of analysis and investment prior to execution. A typical BI environment involves business models, data models, ETL (needed to transform and organize the data into useful information), a target data warehouse, data marts, OLAP analysis and reporting tools.

As Business Intelligence matures, companies who have adopted BI already quite some time are increasingly faced with planning major Business Intelligence migrations. A good migration plan is a key factor in a successful migration project. Often companies embark on a migration without decent preparation & planning, which might be a recipe for disaster. A well-run BI migration starts with a business intelligence / performance management migration plan that helps to quickly determine complexity and potential risks in the migration project. This insight will discuss some of the essential issues that may be encountered in a BI & PM migration effort and a phased approach to the project.
 

Define business reasons for migration

In order to help scoping and defining the migration project, answering some essential questions can be helpful. Below some examples of questions which will offer important information needed to define a migration plan and its business context.

  • What applications are present in the current architecture ?
  • What functional front-end solutions (reporting, analysis, scorecarding, dashboarding.) exist in the current version ?
  • To which user communities have they been made available ?
  • What can be learned from existing usage statistics ? To what extent are the current end-user applications being used (correctly) ?
  • Have there been performance issues that also need to be addressed in the migration ?
  • What have been the business pains that have been solved by deploying the current applications ? Are they still relevant today ?
  • Do the business users know what the functional advantages are, offered by the migration (new functionality within the software release, performance gains offered by new hardware;) ?
  • Are the fundamental differences of the new application clear to the users, including maybe some functionality they might be lost in the transition ?
  • Which is for the company the most important new functionality implemented in the new release and does it matter to the organization in terms of end-user functionality, administration & maintenance ?
  • What are the data sources and connections used ?
  • Is the current application well-documented ?
  • Is the meta-data documented ? (cubes, metadata layers, sources)
  • Is the existing architecture documented, including platform, database type, web server, hardware and security setup.
  • What is the portal strategy of the company ? Is the portal part of the migration process ?
  • Will end users access the old and new environment simultaneously during the migration ?
  • What is the timing incl. any potential deadlines for the migration ?
  • What kind of security setup is currently used and are there any changes to be implemented in the migration ?
  • Is there a test (development) production environment available and how will the migration affect each of these environments ? In case there are no test and development environments, should the migration be used to set those up ?
  • What are the different stages identified in the migration process ? Preference for an incremental (by subject area) migration or "bing bang" full migration ?
  • How long will the previous environment stay in use after migration (2 environments to support, synchronization of data, metadata & security,) ?
  • Are the needs for documentation and training known ?
  • Is there a timeline for roll-out ?
  • Last but not least, migration projects are usually (because of the effort involved) an excellent moment to critically evaluate the existing reports and cubes. If the BI environment has been along for a couple of years, it may be a good opportunity to re-evaluate the business requirements prior to re-building all of its components. Has the existing application been revised together with the business community prior to migration ?


Project Management milestones in a best practise approach

The answers to the above questions will help defining the migration plan. Making use of the elementary methodology for BI & PM Migration Projects the migration plan will be divided into 7 phases or milestones. Working with a methodology during a migration project is aimed at making people work in a more effective way. It ensures to spend more time working towards the goals of the projects and less time on figuring out the next steps. elementary is designed to deal with every aspect of project delivery at any stage in the project lifecycle. It will organize the migration project in a number of phases, each of them with clear steps, roles, activities, techniques, documents and deliverables. Everyone involved in the project, at any time, knows where they are in the project, what has been delivered and what are the next steps.

Initiation Phase

This phase contains all preparatory and mainly project management related activities. The project structure, communication channels, decision process, stakeholders, development guidelines, risk handling procedures and much more is established.

Analysis

In this phase, most of the questions listed above need to be answered. Unless the migration effort is estimated to be very small, this phase will require a substantial amount of effort in order to ensure a successful migration. A first step requires documenting the scope of the migration project in a functional analysis. A functional analysis is intended to be read and understood by management or people who are not closely involved in the migration project. The management of the organization will take decisions based on this document.

  • Analyze business requirements (tip: use the migration questionnaire above). Start a functional analysis with a summary of the business requirements and objectives.
  • Give an overview of the current & future products that will be migrated.
  • Make a summary overview of what is new or different in the new version to support the decision to migrate. Any disadvantages or side-effects of migrating should also be explained.
  • Make an inventory of current BI applications/reports. This gives an indication of the testing effort.
  • It is also important not to overlook the required training of end-users and administrators. This should be planned in well ahead, so the required support resources have the correct skills once the migration project is complete.
  • A duration or time estimation is essential in a functional analysis. The planning needs to be detailed and split into the different environments. For each task the number of days, number of FTE's and responsible team member needs to be known.
  • Define complexity, risk and priority in the migration applications.

An important step is to perform a full risk analysis on the migration plans. This risk analysis is crucial and mandatory to any large scaled migration project, since it facilitates predicting and controlling any potential issue during migrations.

An overview of risks can be shown in what is called a risk matrix. Below an example of the risk matrix which is a "Prince2" Project Management technique. This method allows determining the probability and impact of all potential risks by scoring each risk with a value.
 
Impact:Rate the impacts to business and or applications when the risk occurs.
Probability:How likely is it that the risk occurs? Is there a big chance or relatively low?
Proximity:How close by is the potential risk to the migration?
Owner:Who is the owner of the risk?
Countermeasure:A countermeasure is an action, process, device, or system that can prevent or mitigate the effects of threats.
Mitigation actions:An action or task that will be implemented when a risk is triggered during any phase of the project.
 
Some examples of risks which can be put in the risk matrix:
 
IdDescriptionCategoryImpactProbabilityProximityOwnerCounter measureMitigation Action
1No security system is implemented on the new environment.Security      
2There is no additional hardware environment that can serve as failover during the migration process.Infrastructure      
3There is no database software installed and configured on the new hardware environment.Install & Configure      
4There is no third party software (e.g. Internet Information Server) installed and configured on the new hardware environment.Install & Configure      
5On the new environment are no repositories available yet.Install & Configure      
6There are still previous versions of the software installed on the servers.Install & Configure      
7Nothing is installed and configured on the new environment yet.Install & Configure      
8Reports have not been tested after the migration to the new environment.Test      
9Reports have not been modified after the migration to the new environment.Test      
 
Furthermore, a risk matrix usually also identifies the outcome should the risk occur and potential countermeasures and mitigation actions.

Specify

During the technical analysis, the architecture and the different kind of migration scenarios are discussed with the involved parties. In this phase, it is important to specify the new architecture and migration process. As migrations typically do not occur overnight, it is necessary to take the necessary precautions to keep the old and new environment live next to each other for a certain period of time.

  • Describe the architecture and infrastructure of the current environment and the new environment. Make a list of all the servers and the software components installed on the different servers. A schema of the current and new architecture can help to understand the impact of the migration.
  • Design the migration process for every environment. Take following actions into account for all environments when preparing the migration:
    • Make arrangements regarding a new repository
    • Work out a backup and recovery scenario of the current environment
    • Verify all security settings and make sure there is easy access to the installation software and patches.
    • Take a copy of manual edited files. Place them back after the installation or modify the new installed files with the same changes. Also check if templates, style sheets or other pre-defined developments are used.
    • Describe the configuration and security settings that need to be executed on the different servers.
    • If data needs to be migrated (e.g. BI tool repository), define a migration path.
    • Create a comprehensive test plan.

Build/Migrate

The specifications described in the previous phases are transformed and executed into a system that meets the customer requirements. Migration of metadata and reports is done in the Build phase as required. In case migration is not feasible, some components may need to be rebuilt. Some software vendors offer migration tools to facilitate the automatic migration process. These migration tools can provide benefits such as:

  • Analyze application components and component relationships
  • Shorten migration timeframes by eliminating manual steps
  • Simplify the migration process via migration wizards

With the use of a migration tool, an upgrade report is generated which will identity what has not been migrated automatically. This results in the need for manual migrations and fine-tuning. If some applications cannot be migrated automatically they might require significant manual work. Based on the findings during the analysis phase and the desired functionality in the new environment manual work needs to be performed to obtain a correct working environment.

Test

Acceptance testing generally involves running a suite of tests on the completed system. Each individual test or case, exercises a particular operating condition of the user's environment or feature of the system, and will result in a pass or fail outcome. The test environment is designed to be identical to the anticipated user's environment. These test cases must each be accompanied by test case (input) data or a formal description of the operational activities to be performed (intended to thoroughly exercise the specific case) and a formal description of the expected results. Fine tuning and/or optimization is conducted after each round of testing to achieve the desired performance levels. Following successful testing, the new environment is released for production. Alternatively, further enhancements are executed.

Acceptance tests are usually created by business users and expressed in a business language. These tests are created ideally through collaboration between business customers, business analysts, testers and developers, however the business customers are the primary owners of these tests. The objective is to provide confidence that the delivered migrated system meets the business requirements. The principal purpose of acceptance testing is that, once completed successfully, and acceptance criteria are met, a sign off on the system for final delivery can take place.

Deploy

Upon successful testing the migrated application is ready to go live. A synchronization of the data/reports is necessary to all other environments. All environments need to be in sync with each other. Technical training, end user training, configuration management and roll-out are crucial steps in this phase.

Once the migrated applications are deployed make sure there is still a post-migration support team available. Additional business and user needs may appear. Issues may involve system configuration or optimization, or the application configuration parameters may require fine tuning.

Evaluate

After the migration, the objectives of the migration are evaluated together with the end-users. The aim of this lessons learned meeting is to identify, share and use the experiences of the project in order to keep improving the migration process going forward.
 

Conclusion

Migration of BI environments is a complex process. A successful migration requires careful planning & experience in both source & target platforms.

It is important that expectations are set right up front on what options and approaches are available for the migration project. Making use of a methodology, like the elementary methodology for BI and PM Migration Projects, a well-prepared migration plan will help you to achieve this. It is essential to involve the business user community in any migration project to obtain all the necessary information. This information will help you to work out the migration plan with its different subphases. Those phases give you the opportunity to work in an effective way and to make sure that everybody understands and approves the migration and its impact. Once the migration is done, make sure post-migration support is foreseen.