DevOps

Service offerings

Technologies

What is DevOps?

DevOps is a set of practices which automate the processes between software development (Dev) and IT teams (Ops). The objective of using DevOps is to shorten the development lifecycle and have frequent updates of tools and application in line with business priorities. 
 
DevOps isn't a software nor a framework; it's a mindset or way-of-working. The processes which DevOps helps automate and the practices it suggests are the following:
  • Coding –  Taking a DevOps mindset means collaborating on code development and reviews
  • Building –  supporting continuous integration tools and builds
  • Testing –  embedding continuous testing tools that validates code written 
  • Packaging – providing an artifact repository or application pre-deployment staging
  • Releasing – supporting release approvals and continuous deployment
  • Configuring – providing infrastructure configuration including infrastructure as code support
  • Monitoring – enabling applications performance monitoring

Why is DevOps important for a Big Data Platform?

When setting-up a Big Data Platform we use code to provide infrastructure and define data transformations. Similar to application development, this code needs to be build collaboratively, be tested, be released periodically. DevOps perfectly applies to Big Data and AI developmet.

element61 implements DevOps in all its Data Platform implementations providing

  • the tools and set-up for collaborative code development, automated testing, infrastructure-as-code and continuous integration and deployment.
  • the training of your team in using and benefiting from DevOps

What tools support DevOps?

DevOps isn't necessarily supported by 1 tool. Depending your the tools you already use (e.g. Github or Bitbucket) you might need to add tools or you might already have all the components you need.

When building an Azure Data Platform, we recommend to use Azure DevOps for organization with no DevOps yet available. For all other scenario's tools like CircleCI are most popular.

DevOps

Azure DevOps services is DevOps toolbox providing Code versioning, Agile tooling, Build and Release functionalities, Test plans and Azure Artifacts.
It has a strenght in (1) providing an end-to-end toolkit including all DevOps best-practices (e.g. use of .yaml-files), (2) tight integration with Azure

Getting started

Data Engineers at element61 are experienced in using DevOps in all projects. Contact us to get started or get some advice