As in any project, also in Business Intelligence and Data Warehousing projects, a time exists for discovery of what is wanted and/or needed followed by a time for building what results from the requirements. In typical project management best practices, this discovery process is done through a phase called "requirements analysis”. Normally requirements analysis is a part of the "Analyze” project phase as depicted in the element61 project methodology elementary™ underneath.
While requirements analysis tends to be a very broad topic, this insight will specifically focus on gathering the information requirements which are resulting in a conceptual information model. After all, the information model is the beating heart of any Business Intelligence solution. Its added value largely depends on the quality and the depth of the dimensions & the facts and their corresponding attributes & measures. On the other hand, it also goes without saying that the information model will dictate other key activities such as interface building, Extraction-Transformation-Loading (ETL), creation of a physical model and last but not least front-end BI modeling & reporting.
Requirements, characteristics and tracks
Before diving into the actual information requirements gathering details, it is best to set the scene by explaining what is meant by "requirements” and how they can be used; what are the characteristics of requirements; which requirements tracks can exist and where is the conceptual information model to be found?
Business Intelligence requirements
When building a Business Intelligence solution, it is important that all (business) requirements are gathered upfront and agreed upon (read: accepted), both by the business and the parties which will establish the required solution.
Requirements define the ‘what' of a certain solution:
- What the solution must do to add value for the business. Functional requirements which define the capabilities of the solution.
- What the solution must be to add value for the business. Non-functional requirements which define the characteristics, properties or qualities that the end-result must possess. They define how well the solution performs its functions.
- What limitations there are on the choices that designers & developers have when implementing the solution.
Requirements can be used:
- To record all business needs, translated into measurable and testable requirements
- To understand what to build and/or to deliver
- To guard the scope and protect against scope creep and as such protect the initial project plan and budget set forth
- As a base checklist versus the change requests which can and will show up, both during the project and post-project phases
- During a consecutive feasibility study
In the process of gathering requirements, it is important to use the right set of templates. These templates can provide an extensive checklist of possible requirements areas which may or may not be applicable, and as such enable a good head start. Another purpose of these templates is to secure all requirements which have been previously discussed (in a number of business sessions, with different business parties) are captured correctly and completely.
When creating requirements it is important to also gather the right "characteristics” of each requirement – call it metadata if you will.
Each requirement, which is part of the solution to be, requires a unique identifier together with a clear description, the rationale for the requirement, the corresponding owner and the beneficiaries. Unique identifiers for all requirements will enable a better communication between all parties.
A requirement always starts out as being valid, however during the course of the project a requirement can become obsolete. In that case, it is important to keep the requirement but change its status to obsolete. In this way, time spend on requirements that became obsolete can be justified. Finally a priority needs to be attached to each requirement, indicating whether the need is essential, conditional or optional.
Here the so-called "MoSCoW” concept can be used. In this concept, the capitals stand for
- M = must have this
- S = should have this if at all possible
- C = could have this if it does not affect any other requirement of being delivered
- W = won't have this time, but would like to have in the future
It is important that next to the obvious business requirements, proper attention is given to non-business requirements. Therefore 3 tracks should be distinguished throughout the entire requirements analysis life cycle.
Business track (main focus) – WHAT
What information is required in relation to the business needs of the organization ? Topics here should cover: conceptual information model (dimensions, facts & their relationship), scope of data, security, reporting requirements …
Technical track – MEANS
What are the surrounding requirements? How will the technical solution look like?
Topics here should cover: BI-tool requirements, availability, performance, data retention, frequency, data rules, data quality, interfacing …
Organizational track – HOW
How will the solution be managed? In which way is the solution established and put into the organization ?
Topics here should cover: Service level agreements, learnability, operability, monitoring, communication, support, standards …
Approach to gathering requirements towards a conceptual information model
As stated in the previous section, the main focus should be on the business track serving the number one objective: the conceptual information model. In this paragraph an explanation is given on the content of such a model. Next to that, a possible approach is presented on gathering the business requirements in function of that conceptual information model using a mix of both interview and workshop techniques.
Conceptual information model
The information requirements define the specific data items that must be included as part of the solution. This is consolidated into a conceptual view of the data model.
The objective here is to establish the following:
An overall conceptual information model. This is a graphical representation of business information needs in support of the business objectives of the organization.
Dimensions & fact definitions. First cut definitions of the business items that will be used within the solution together with their inter-dependencies.
A business delivery roadmap. This is prioritized roadmap on top of the information model.
The information model concepts set forth are based on the dimensional modelling approach by Ralph Kimball.
A 4-step dimensional design process can be used to detail the requirements.
- Which business process is supported
- Level of detail (grain)
- Dimensions (common) – at entity level & first cut definitions
- Measures – fist cut definitions
Deliverables here are :
The business process matrix which gives an overview of the combination of all relevant dimensions with all business processes within a certain subject area. A check mark in a cross-section points to a relevant relationship. Given the risk of duplicate data, more ETL development work, more upload time, different labelling and different terminology, it is important to use business processes here instead of departments.
The information matrix gives an overview of the combination of all relevant dimensions with all facts within one single business process. The cross-sections indicate the granularity or level of detail. A check mark in a cross-section points to a relevant relationship.
Typically this matrix will be detailed more towards its final conception.
- When the dimension levels will be detailed further, a checkmark might be replaced by a certain dimension level which might be more applicable than the most atomic detail of that particular dimension in relation to a certain fact, or will remain a checkmark indicating the lowest level as defined within the dimension.
- This matrix (likewise for the business process matrix) can also be used by replacing the check marks by priorities, availability, source complexity or cost and as such revealing other interesting viewpoints.
The dimension sheet covers all dimensions which are the axes or context by which the numeric information, contained in the facts, is analyzed. The most important dimensions & attributes - names & definitions should be known at this stage. If possible the following information should be added: name, definition, hierarchies, volume (of the data), examples, source, owner, distinct values …
The fact sheet covers all facts (or KPIs) which contain the numeric information upon which the reporting will be done. These measures are kept at the intersection of the different dimensions and are also reported as such. The most important facts - names & definitions should be known at this stage. If possible the following information should be added: name, definition, version (actual, budget, forecast), delivery frequency, additive, calculation, source, owner, supporting business process, availability …
Various approaches exist for gathering information requirements. Based on a number of real-world lessons learnt, it basically every time boils down to a combination of iterative interviews and workshops. This does not mean that there is no room for other techniques such as: brainstorming, questionnaires, prototyping, observations, use cases or other. In the end however, there is no surrogate for having a direct participation between the analyst(s) and the (internal) customers and/or the (internal) customer amongst each other.
A possible approach for gathering the business requirements in function of the conceptual information model can be depicted as underneath. This approach can be used for large scale projects; however when used within smaller exercises or depending on the context, certain elements can be left out or should be dealt with in a more pragmatic way.
Business Preparation. The most important part of this sub-phase is used for interviewing the business sponsor in terms of agenda & objectives. All meetings – both interviews & workshops – will be set up within a given timeframe. Next to that, this sub-phase will be used by the facilitators to get acquainted with the business in general and the organization in particular. Here also all required templates will be prepared.
Senior Management Interviews / Departmental Interviews. Identification of high level (actual and future) business needs.
Business Needs Workshop (1). Here a workshop is held, preferably according to the earlier mentioned guidelines. Here all participants need to share which information is currently used or asked from them, be it on a regular or ad hoc basis. Is the information asked for met, not met, or only partly met? What future questions can be expected? Here the information matrix comes into play.
IT Interviews. This concerns interviews with the main information creators. This information will be used later in the Gap Analysis phase. Here all relevant sources are from a high-level perspective verified on their fit-for-purposes in terms of availability, complexity, quality, known issues, etc. This can happen partly in parallel with the business sub-phases.
Domain / Sub-domain grouping. Once the deliverables (information matrix …) have been captured, the needs can be grouped into meaningful domains and split again into sub-domains which will be the basic building blocks of the conceptual information model. Possibly here the entire group of workshop participants can be split up according to the domain groupings.
Detail Needs Workshop (2). Based on the previous sub-phases, a more detailed understanding of all relevant business questions should be captured (which viewpoints, for how long, how recent does it need to be …). In this session, the most important instrument are the dimension & fact sheets.
Gap Analysis. This sub-phase will give insight in the delta's between the requirements and the high level data source analysis. This will provide information on the business requirements versus there feasibility at this moment resulting in a reality check. (Optionally, a roadmap can be established for bridging the gap.)
Prioritization. The data warehouse will be established by subject area. Typically an incremental approach is used here. Based on of the parameters - previously defined - which may influence priorities, these priorities are set, the various domains & sub-domains will be set out on a timeline to obtain a business information roadmap.
Interview versus workshop
Finally we would like to share some best practices, both for interviews and workshops:
Always start within a project context
The interview / workshop introduction should always cover the project & session objectives, next to a brief overview of relevant data warehouse & business intelligence concepts
Be sure to have clear sponsorship
Time-box in order to tightly manage the agenda and make sure that all relevant topics are dealt with equally
Go for an optimal mix of participants
o Approximately 85% business (presenting their wishes) & 15% technical people (putting the wishes in the correct perspective)
o Horizontal mix – in terms of having stakeholders from all relevant departments
o Diagonal mix – in terms of having profiles ranging from C-level or just underneath moving down to the more operational people
o Include as much as possible relevant subject matter experts
Establish as soon as possible a common business language. Define terms and concepts so all participants have a common understanding
Since not everything can be established at once - decide early on what defines the priorities in terms of an incremental roadmap. This can be a combination of business value, complexity, scope width, reusability of existing components ...
Provide feedback as soon as possible
Each information need is best stated as a business question. Example: total actual volume sold by month, by market, by product type
Although interview sessions certainly have their benefits, in this type of exercises, workshop sessions are definitively key towards a final conceptual information model :
Having all people together, preferably off-site
The group of participants should have the proper decision rights
Communication occurs directly between participants
All participants are equal and involved à parallel information
Feedback is provided immediately
Find a proper balance in ‘parking' topics
Next to a white board & a flip chart, a best practice is to use an on screen display of the information matrix, dimensions & facts sheets for immediate feedback when filling them in.
The success of any Data Warehouse & Business Intelligence solution is determined at the beginning of the project lifecycle, when analysts face the business to gather their requirements. Poor gathering of business requirements is one of today's main Data Warehouse & Business Intelligence project failures. One of the key aspects within all those business requirements is the information model. It requires a good preparation and experienced facilitators & analysts with the right skill set to discover the true business needs in this area. But above all, it requires a specific approach for gathering the needed requirements.