A Lambda architecture is a data-processing architecture combining both batch- and (real-time) stream-processing methods. The advantage of this dual architecture is the ability to handle massive amounts of data while maintaining real-time monitoring. As a result, it is generally used as a basis for big data architectures.
- Batch layer aimed to
- keep and secure the master raw dataset (historical and latest data)
- provide pre-computed views (in batch) on business-relevant aggregations and metrics.
(One can compare this layer to the conventional DWH layer currently available in BI)
- Speed layer designed to deliver fast: i.e. to delivered real-time data streams which have low latency requirements
- Serving layer designed to interface with the end-user and consuming from both the batch and speed layer.
A serving layer can be seen as a dashboarding/reporting layer aimed to handle both batch reporting as well as real-time reporting
click to enlarge