A Continuous Delivery pipeline is a (mostly) automated software tool chain that takes delivered code, builds it, tests it, and deploys it. This simple concept gets complicated by tool chain realities: no one vendor does everything that needs to be done in the pipeline, and new solutions are evolving every day.
To make sense of the CD pipeline tool chain, I have taken a close look at the market and have identified a set of tool categories. I’m sure I’ve missed something, and you may not agree with my categories, and in either case I would like to hear from you! You can reach me on Twitter (@ksbittner). If you think the categories sound right, I’d like to hear that, too. This is your chance to help define the continuous delivery tools market.
Continuous Delivery Tools & Technologies
Continuous Delivery is a process by which source code is built, deployed to testing environments, test, and optionally deployed to production environment using a highly automated pipeline. Many different kinds of tools need to be brought together to automate this process. The tool categories described below provide the building blocks of the automated Continuous Delivery process.
Continuous Delivery Management
Continuous Delivery management tools manage the software delivery pipeline for an application, including all related components and services. They utilize a model of the application to understand component/service dependencies, including build-time and run-time dependencies. They orchestrate the application delivery workflow from code check-in to deployment of the application into one or more production environments, and provide transparent visibility into the state of an application in the delivery pipeline. For composite applications, they manage the multiple pipelines that deliver the components of the application.
Version management tools provide the ability to store separate versions of things, including the ability to manage variation through branching and merging. Version management tools are also often called source code management tools; we have used a more encompassing term because they are often used to manage much more than just source code.
Continuous integration is the process in which all changes to a software baseline are built and tested each time code is checked in. As a best practice – and in order to implement continuous delivery – the CI process must be automated.
Content management systems (CMS) manage intellectual property that is often served up by applications. Some sophisticated CMS tools also provide the ability to build applications, eliminating or reducing the amount of application code that is required to deliver a particular capability.
Build automation tools are software applications that automate the compilation and integration of executable software components from source code. In the context of continuous delivery, build automation tools are invoked from the Continuous Integration process/tool.
Environment management tools automate the provisioning and management of environments based on versioned configuration information. They are able to create a freshly configured environment from stored configuration information, or to modify an existing environment to match a defined configuration.
Artifact repository tools are used to store and manage binary objects that are used to build software. In some cases these binaries represent software for which the organization does not have the source code, as is the case with third-party libraries, or open source software for which the source code is maintained elsewhere. In other cases these binaries are for commonly used components that do not need to be rebuilt from source code every time.
Static analysis tools analyze source code for variation from a set of defined coding standards. They are used to find errors in the code, including architectural anti-patterns and security vulnerabilities. They are often used in conjunction with a manual peer code review processes to identify potential problem areas that require further manual inspection. They can also be used to find opportunities for refactoring of common code from more than one code line.
Unit testing tools are used to verify that the API offered by a component or service conforms to agreed specifications and that the component (or unit) works correctly.
Service virtualization tools simulate the behavior of a system or service by providing an alternate implementation of that system or service’s APIs. The simulation provides exact functional parity and typically also simulates the load and performance characteristics of the system/service.
Test management tools manage the software testing process, typically including managing test cases, test automation artifacts and test results.
Test Data Management
Test data management tools create and manage test data. They typically provide tools for creating anonymized test data from production data. They also typically create test data sets that not only provide randomized data, but also ensure that test scenario edge cases will be executed, ensuring complete test coverage with the minimum data sample size.
Automated Functional Testing
Similar to Unit testing, automated functional testing (AFT) uses APIs to test the integrated functionality of the application, moving beyond unit testing that ensures that each component (unit) does its job. AFT also often includes automated UI-based testing.
Perormance/load testing tools verify the scalability and performance of a system by subjecting it to a simulated workloads and transaction mixes.
Application Release Automation
Application release automation tools are used to deploy one or more software components to a target environment to speed and streamline the process, reducing manual errors and to provide the ability to easily roll back a deployment if it fails or is deemed unsatisfactory.
Application analytic tools are used to capture data about application usage, and user preferences and behavior from the run-time environment. Analytic tools are typically embedded in applications by developers at the time code is written.
Lifecycle integration tools tie together tools from multiple vendors into a single integrated tool chain, ensuring data consistency and integrity. They also provide reporting and visibility across the integrated tool chain.