9 Best Practices for a Mature Continuous Delivery Pipeline

Continuous Integration(CI) is a software engineering practice which evolved to support extreme and agile programming methodologies. CI consists of best practices consisting of build automation, continuous testing and code quality analysis. The desired result is that software in mainline can be rapidly built and deployed to production at any point. Continuous Delivery(CD) goes further and automates the deployment of software in QA, pre-production and production environments. Continuous Delivery enables organizations to make predictable releases reducing risk and, automation across the pipeline enables reduction of release cycles. CD is no longer an option if you run geographically distributed agile teams.

MSys has designed and deployed continuous integration and delivery pipelines for start-ups and large organizations leading to benefits like:

• Automate entire pipeline – reduce manual effort and accelerate release cycles
• Improve release quality – reduce rollbacks and defects
• Increased visibility leading to accountability and process improvements
• Cross-team visibility and openness – increased collaboration between development, QA, support and operations teams
• Reduction in costs for deployment and support

A mature continuous delivery pipeline consists of the following steps and principles:

1. Maintain a single code repository for the product or organization

Revision control for the project source code is absolutely mandatory. All the dependencies and artifacts required for the project should be in this repository. Avoid branches per developer to foster shared ownership and reduce integration defects. Git is a popular distributed version control system that we recommend.

2. Automated builds

Leverage popular build tools like ANT, make, maven, etc to standardize the build process. A single command should be capable of building your entire system including the binaries and distribution media (RPM, tarballs, MSI files, ISOs). Builds should be fast – larger builds can be broken into smaller jobs and run in parallel.

3. Automated testing for each commit

An automated process where each commit is built and tested is necessary to ensure a stable baseline. A continuous integration server can monitor the version control system and automatically run the builds and tests. Ideally, you should hook up the continuous integration server with Gerrit or ReviewBoard to report the results to reviewers.

4. Static Code Analysis

Many teams ignore code quality until it is too late and accumulate heavy technical debt. All continuous integration servers have plugins that enable integration of static code analysis within your CD pipeline or one can also automate this using custom scripts. You should fail builds that do not pass agreed upon code quality criteria.

5. Frequent commits into baseline

Developers should commit their changes frequently into the baseline. This allows fast feedback from the automated system and there are fewer conflicts and bugs during merges. With automated testing of each commit, developers will know the real-time state of their code.

Integration testing in environment that are production clones Testing should be done in an environment that is as close to production as possible. The operating system versions, patches, libraries and dependencies should be the same on the test servers as on the production servers. Configuration management tools like Chef, Puppet, Ansible should be used to automate and standardize the setup of environments.

6. Well-defined promotion process and managing release artifacts

Create and document a promotion process for your builds and releases. This involves defining when a build is ready for QA or pre-production testing. Or which build should be given to the support team. Having a well-defined process setup in your continuous integration servers improves agility within disparate or geographically distributed teams. Most continuous integration servers have features that allow you to setup promotion processes. Large teams tend to have hundreds or thousands of release artifacts across versions, custom builds for specific clients, RC releases, etc. A tool like Nexus or Artifactory can be used to efficiently and predictably store and manage release artifacts.

7. Automate your Deployment

An effective CI/CD pipeline is one that is fully automated. Automating deployments is critical to reduce wastage of time and avoid possibility of human errors during deployment. Teams should implement scripts to deploy builds and verify using automated tests that the build is stable. This way not only the code but the deployment mechanisms also get tested regularly.

It is also possible to setup continuous deployment which includes automated deployments into production environments along with necessary checks and balances.

8. Configuration management for deployments

Software stacks have become complicated over the years and deployments more so. Customers commonly use virtualized environments, cloud and multiple datacenters. It is imperative to use configuration management tools like Chef, Puppet or custom scripts to ensure that you can stand up environments predictably for dev, QA, pre-prod and production. These tools will also enable you to setup and manage multi-datacenter or hybrid environments for your products.

9. Build status and test results should be published across the team

Developers should be automatically notified when a build breaks so it can be fixed immediately. It should be possible to see whose changes broke the build or test cases. This feedback can be positively used by developers and QA to improve processes.

Every CxO and engineering leader is looking to increase the ROI and predictability of their engineering teams. It is proven that these DevOps and Continuous Delivery(CD) practices lead to faster release cycles, better code quality, reduced engineering costs and enhanced collaboration between teams. Learn more about MSys Technologies’ skills and expertise in DevOps/CI/Automation here.

Get in touch with us for a free consulting session to embark on your journey to a mature continuous delivery pipeline – email us at [email protected]