What is Infrastructure as Code?
Setting up a Data Platform in the Cloud means spinning up various Cloud Components and configuring them all to work together as a Modern Data Platform.Infrastructure as Code means that we script all configurations and creations of all platform components. This means that, rather than creating them manually by clicking in the User Portal, we automate the creation, configurations and updates to any Data Platform component.
Figure: Infrastructure as Code is code which scripts the creation of VMs or other Cloud Components
Why Infrastructure as Code?
Imagine you have a Data Platform with a Data Lake, various Event Hubs, a Big Data Cluster, a Docker registry and that all configurations were manually configured through the User Portal. How would you ever deploy this setup to Production?
Infrastructure as Code reduces time and provides security to your Data Platform setup.
Major benefits include:
•Ability to track all changes on Infrastructure level
•Faster deploys from Dev to Production given no manual actions are required
•Ability to instantly deploy elsewhere - e.g. create an Acceptance environment
Which technologies allow this?
Infrastructure as Code is provided by all major Clouds including Azure, AWS and Google. To do Infrastructure as Code, you can
- either use the syntax specific to a Cloud (every Cloud has its own infrastructure syntax)
- or use a generic language such as Terraform
element61 has experience with the following tools:
Azure Resource Manager allows you to create Azure resources using a JSON-template. It allows the creation, update and deletion of resources.
Azure Command Line Interface (CLI) is the command line plug-in to create, update and configure resources through command line code in either Bash, cmd or Linux.
Terraform is an infrastructure provisioning tool created by Hashicorp. It allows you to describe your Infrastructure as Code, create “execution plans” that outline exactly what will happen when you run your code, build a graph of your resources, and automate changes with minimal human interaction. Terraform uses its own domain-specific language (DSL) and is thus cloud-agnostic allowing you to automate infrastructure stacks from multiple cloud service providers simultaneously and integrate other third-party services.
When to use Infrastructure as Code?
We recommend our customers to use Infrastructure as Code for any project or setup, small or big. It saves time and secures your setup to be future proof.
Our Data Engineers at element61 are trained to use Infrastructure as Code and can help you and your colleagues in absorbing this best-practice.
Contact us to get started!