6 Fundamentals of Chef Workflow that you should know

Chef Workflow is a tool built by Chef for Continuous Delivery of applications and infrastructure. It provides facility for automated testing and deployment.
Chef Workflow has a shared pipeline model. Every change has to go through some predefined phases of the pipelines prior to getting released. These phases are customizable. So when you push your changes to chef-workflow, your code is tested automatically, reviewed (which is done manually of course) and then delivered.

1. Chef Workflow Pipelines

A pipeline is a series of automated and manual quality gates that take software changes from development to delivery.
Chef Workflow pipeline is made up of 6 stages: Verify, Build, Acceptance, Union, Rehearsal and Delivered.
The diagram below explains it:

Each project has associated Verify, Build and Acceptance stages. Verify and Build stages perform tests on the source code.
Union, Rehearsal and Delivered are a part of Shared Pipeline. They test the releasable artifacts.

2. Project Pipeline

It includes the stages which are unique per project. A developer has control only till the “Project Pipeline”. Here you push your change, someone reviews and approves it and then the code is shipped to the Shared Pipeline.

• Verify Stage:The verify stage runs automatically when someone submits a change. It is made up of various phases like:
  • Lint:Identifies stylistic problems in your source code
  • Syntax:Checks that the code can be parsed
  • Unit:Runs unit tests
• Build Stage:When a change is approved, Chef Workflow merges the change into the pipeline’s target branch and triggers the Build stage. Build stage again runs lint, syntax and unit phases from Verify stage. This is because your branch may have moved ahead since the Verify stage ran on this change.

Build stage has some additional phases:

• Quality:Runs additional test suites. Some tests are too time consuming. They can be put in Build phase instead of the Verify phase
• Security:Security tests as well as functional test suites can be added here
• Publish:Produces the potentially releasable artifacts and makes them available for rest of the pipeline

• Acceptance Stage:Till Build stage the pipeline was analyzing the source code. From the acceptance stage onwards, it starts analyzing the artifact produced in Build stage. As the name suggests, Acceptance is the stage where the team decides whether the change should go into production or not.

There are 4 phases in Acceptance stage:

• Provision:Provision infrastructure needed to test the artifacts
• Deploy:Deploy the artifacts to your infrastructure
• Smoke:Run smoke test. They should be short running
• Functional:Run functional tests to assure that changes are meeting the business requirements

3. Shared Pipeline

“Shared Pipeline” is automated by Chef Workflow. It runs the test cases for every stage, tests the cookbook/application by VM provisioning and if all stages are passed then the code gets merged into the desired branch.

• Union Stage:A project usually doesn’t work independently. It has dependencies on several other projects too e.g. one cookbook can have dependency on several other cookbooks. The purpose of Union stage is to analyze the impact of your change on the whole system. Here tests are performed with interactions between the interdependent projects. At times, your change may pass the Acceptance stage and fail during the Union Stage. In this case a discussion is required to find the right fix. The fix may be in your project or in some other dependent project. Phases of Union stage and the remaining stages are same: provision, deploy, smoke and functional.
• Rehearsal Stage:This stage is triggered if all phases of Union stage pass. The purpose of this stage is to gain confidence in your change. It repeats the same process as of Union stage in a different environment. It’s like a pre-production environment.
• Delivered Stage:It is the final stage and its definition can vary according to one’s requirements. It could mean deploying your changes and making them live, or publishing a set of artifacts for the customers.

4. Phases

Each pipeline stage consists of some phases. The diagram below explains the phases in each stage:

These phases are customizable. What happens in each phase can be defined in the Build cookbooks. Each phase is configured with a recipe in that cookbook. Build cookbooks can also be used to define what kind of artifacts to build and where to store them.

5. Chef Workflow Components:

The following diagram shows components involved in Chef Workflow:

The build cookbooks reside on Chef Server which decide what happens in each phase. Each build node is registered with the chef server and the phase jobs run on them. It’s better to have 3 build nodes so that lint, syntax and unit phases can run in parallel.
For each deploy-able stage of chef workflow (acceptance, union, rehearsal and delivered), there is a web accessible server where you can verify your changes pushed through pipeline. Server names are as per the stage i.e. Acceptance, Union, Rehearsal and Delivered.

6. Infrastructure:

• Workstation Server:This is your working environment. Here you clone a project from delivery server and push your changes through chef workflow pipelines.
• Delivery Server:This is like a github repo. This is where you clone your projects from and push your changes into.
• Chef Server:All the build cookbooks for different phases are hosted on the chef server. These cookbooks are run on the build nodes registered with Chef Server