Today, most big data users are not data scientists. They are business users who don’t want to mine through everything. They want a simple guided experience that only gives them information that is relevant and contextual to them. But while they want to look at various slices of a big data repository, they may not always know which slices they want to look at.
So how does Qlik handle this? You may already be familiar with how Qlik empowers both the technical and non-technical user to fully explore information, gain new insight and understand the whole story within their data. Qlik also has a global network of over 1.700 partners including many that focus on big data technologies such as Cloudera, AMS or Google BigQuery.
For big data situations the power of Qlik builds upon the application of multiple techniques. Different data volumes and complexities are best met using different methods or combination of methods. Sometimes one method is sufficient. In other cases, it could be multiple methods working together. So, let us review each method, including Qlik’s newest technique “On Demand App Generation”, abbreviated by Qlik as ODAG.
Figure 1 – Qlik’s big data techniques
In-Memory
First is the Qlik Indexing Engine or QIX, Qlik’s patented in-memory data indexing technology. Some Qlik customers find that the inherent capabilities of Qlik’s indexing engine meet their big data requirements as Qlik can compress data down to 10% of its original size.
Figure 2 – Qlik Sense example app featuring patented QIX engine technology
Segmentation
Segmentation is the process of dividing one Qlik application into multiple applications. For example, instead of having one giant app with information on the entire world the user segments the data into regional apps for the Americas, EMEA and APAC.
Figure 3 – Segmentation example
Chaining
Chaining refers to linking of multiple Qlik apps together while maintaining some sense of state or selection that the user has made prior to linking. Segmentation and chaining can also be utilized together by first segmenting the data in subject specific views and then chaining these separate views to each other.
Figure 4 – Chaining example
Direct Discovery
Next is Direct Discovery. A hybrid approach that combines the Qlik in-memory data model with external data that is queried on the fly. Most tables are still loaded into memory, but extremely large tables are accessed using Direct Discovery. However, it should be pointed out that Direct Discovery is only applicable in basic situations where the user wants to query data from a single table with near real-time data access.
Figure 5 – Direct Discovery example
On Demand App Generation
And finally, On Demand App Generation. Many times users don’t initially know which slice of data they want to analyze in more detail. What they need is a method to quickly scan the entire data source for potentially interesting sections that require a more detailed look. On Demand App Generation consists of two different apps. Initially a user is given a selection app where he or she picks from a shopping list of particular subsets of data such as a time period, customer segment or geography. This selection then triggers the immediate generation of a purpose-built app that only contains detailed data related to their selection.
Figure 6 – On Demand App Generation example app
The user is now free to explore the selected detailed data in any direction. A detail is not interesting, simply go back to the selection app and select again. The user can now investigate as many different slices as he or she wants without the need to develop a new app each time.
The promised benefits of big data will not be realized until there is a way for businesses to easily analyze data. I believe that everyone in an organization should be empowered to drive value from big data, not just the data scientists. Users should be given the freedom to fail fast and easily change direction as they explore. At this stage, I believe Qlik is a secure investment as the platform has the scalability and flexibility to adapt as the big data landscape continues to rapidly change.
