What is Infrastructure as Code?
Setting-up a Data Platform in the Cloud means spinning up various Cloud Components and configuring them all to work together as a Modern Data Platform. Infrastructure as code means that we script all configurations and creations of all platform components. This means that, rather than creating them manually by clicking in the User Portal, we automate the creation, configurations and updates to any Data Platform component.
Figure: Infrastructure as code is code which scripts the creation of VMs or other Cloud Components
Why Infrastructure as Code?
Imagine you have a Data Platform with a Data Lake, various Event Hubs, a Big Data Cluster, a Docker registry and that that all configurations were manually configured through the User Portal. How would you ever deploy this set-up to Production?
Infrastructure as Code reduces time and provides security to your Data Platform set-up.
Major benefits include:
•Ability to track all changes on Infrastructure level
•Faster deploys from Dev to Production given no manual actions are required
•Ability to instantly deploy elsewhere - e.g. create an Acceptance environment
Which technologies allow this?
Infrastructure-as-code is provided by all major Clouds including Azure, AWS and Google. To do Infrastructure as Code, you can
- either use the syntax specific to a Cloud (every Cloud has its own infrastructure syntax)
- or use an generic language such as Terraform
element61 has experience with the following tools:
Azure Ressource Manager allows you to create Azure ressources using a JSON-template. It allows the creation, update and deletion of ressources.
Azure Command Line Interface (CLI) is the command line plug-in to create, update and configure ressources through command line code in either Bash, cmd or Linux.
Terraform is an infrastructure provisioning tool created by Hashicorp. It allows you to describe your infrastructure as code, creates “execution plans” that outline exactly what will happen when you run your code, builds a graph of your resources, and automates changes with minimal human interaction. Terraform uses its own domain-specific language (DSL) and is thus cloud-agnostic allowing you to automate infrastructure stacks from multiple cloud service providers simultaneously and integrate other third-party services.
When to use Infrastructure-as-Code?
We recommend our customers to use Infrastructure-as-Code for any project or set-up small or big. It saves time and secures your set-up to be future proof.
Our Data Engineers at element61 are trained to use Infrastructure-as-code and can help you and your colleagues in absorbing this best-practice.
Contact us to get started!