Continuous integration (CI) and continuous delivery or deployment (CD) are key DevOps practices that provide teams with rapid feedback on their latest code changes and help improve code quality. With a good CI/CD process in place, software development teams can move toward more frequent releases, thereby delivering more value to users and benefiting from their feedback.
The rising importance of CI/CD is evident as 44% of developers now confirm regular usage of CI/CD tools, with a significant 22% having adopted a new tool within the past year. This surge in popularity has led to an overwhelming array of options in the CI/CD tool market, making it challenging to identify the perfect fit for your team’s unique requirements.
Source: Developer Ecosystem Report 2022
To help you navigate this overwhelming landscape of available tools, we present the insights gathered from over 29,000 developers in the State of Developer Ecosystem 2022 report. This comprehensive report covers various aspects of software development, including identifying the most-used CI tools in both personal and professional contexts.
In this post, we’ll go through the most popular CI tools on the market today to help you choose the right one for you and your team.
What is a CI tool?
A CI tool is a platform that automates the process of building, testing, and publishing your software. CI tools integrate with your version control system so they can fetch the latest changes from your repository.
Most CI tools consist of a central server (often known as a build server) and one or more build agents (or runners) that run on separate machines. The build server provides a UI for configuring your build and test pipelines, stores the details of each job, and initiates each pipeline run.
The build, test, and other tasks are distributed to the build agents by the build server. If multiple build agents are available, multiple pipelines can run at the same time, and tasks from the same pipeline can be run in parallel.
You can find out more about CI/CD in our comprehensive CI/CD guide.
What kinds of CI/CD tools are there?
When it comes to choosing the best CI/CD platform for your team, there are several factors to bear in mind.
Open-source
Whether a CI/CD tool is open-source or a commercial solution is an important consideration. While open-source software is free to use, the only support will be what is provided by others in the community. If you discover a bug or would like a new feature added, you either have to wait for someone else to address it, or you can contribute the change yourself. Whichever way new code is contributed, it may take time for the changes to be reviewed and accepted by the project maintainers.
By contrast, commercial platforms usually include SLAs for technical support, and they deliver regular updates containing new features and bug fixes. Many commercial CI tools offer a free tier, but – unless yours is an open-source project – there’s usually a restriction on the number of concurrent builds you can run or the number of build minutes available.
Cloud-based vs self-hosted options
The move toward using continuous integration and continuous delivery tools in the cloud is in full swing, with more than half of developers (51%) having transitioned to using cloud-based CI tools, and 42% having adopted cloud-based CD tools.
Source: Developer Ecosystem Report 2022
Cloud-based or SaaS CI/CD tools typically include a web UI for managing your build pipelines, with the build agents or runners hosted on public or private cloud infrastructure.
With a cloud-based solution, there is no installation or maintenance required. Once you sign up, you have immediate access to a CI server and can start running your builds on cloud-hosted agents without delay. Some SaaS options also allow you to run builds on your own build agents at a lower cost.
Self-hosted options allow you to choose where both your build server and build agents are deployed: in a private cloud, on on-premises hardware, or in public cloud-hosted infrastructure. The trade-off is that you are responsible for managing installation and upgrades to the CI server and your build agents. With on-premises deployments, you also need to consider scaling and availability of the server and agents.
Integrations
A CI/CD pipeline touches every stage of the software development process. As such, your CI/CD platform needs to be able to integrate with each element in your toolchain, including your VCS system, issue trackers, team messaging platforms, and deployment environments hosted in the cloud or on-premises.
The integrations available vary by CI tool. Some CI systems are part of an existing ecosystem or are restricted to particular source control systems or VCS hosting services. Others offer the flexibility to integrate with tools from multiple providers and give you the option to extend the platform with plugins or APIs.
Configuration
Configuring your automated CI/CD pipelines includes everything from specifying the trigger that will initiate each pipeline run to determining the behavior in the event of a failed build or test. These settings may be configured using scripts or from a UI.
While using the UI can be easier for those new to CI/CD and for non-developers, scripts allow you to adopt a configuration-as-code approach with the logic stored in version control. Some tools also give you the option to generate scripts from the UI, giving you the best of both worlds.
The best CI/CD tools in DevOps according to our State of Developer Ecosystem report
The report ranks the 17 most-used CI tool options according to the responses of over 29,000 developers worldwide.
Source: Developer Ecosystem Report 2022
We’ve prepared a breakdown of the most popular CI tools to help you choose the best option for your team and project. Continue reading for a more detailed overview of each tool in the list.
| CI Tool | Open Source | Hosting | Free version | Build agent license pricing | Supported platforms |
| GitHub Actions | No | Cloud | Yes | Execution units included with each tier. Additional hosted runner minutes start at $0.008 (for Linux). | Linux, Windows, and macOS |
| Jenkins | Yes | Self-hosted | Yes | Free | Linux, Windows, and macOS |
| GitLab CI | No | Cloud and self-hosted | Yes | Build units included with each tier. Additional units to run builds on shared pipelines start at $10 for 1,000 minutes. | Linux, Windows, macOS, and Docker |
| Azure DevOps | No | Cloud and self-hosted | Yes | 1 pipeline included free. Additional pipelines start at $15 per month (self-hosted) or $40 per month (cloud-hosted). | Linux, Windows, and macOS |
| CircleCI | No | Cloud and self-hosted | Yes | Build minutes included with each tier. Credits can be traded for build minutes, users, and additional network and storage. | Linux, macOS, Windows, GPU, ARM, and Docker |
| Travis CI | No | Cloud and self-hosted | No | Limit on concurrent jobs varies by tier. Unlimited build minutes with any tier. | Linux, macOS, and iOS |
| Bitbucket Pipelines | No | Cloud | Yes | Build minutes included with each tier. Additional minutes start at $10/month for 1,000 minutes. | Linux, Windows, and macOS |
| AWS CodePipeline / AWS CodeStar | No | Cloud | Yes | Pricing per pipeline. AWS storage incurs additional costs. | Linux, Windows, and macOS |
| TeamCity | No | Cloud and self-hosted | Yes | On-Premises: 3 build agents and unlimited build time included in the free tier. Additional agents start at $239 per year Cloud: from $45/month for 3 committers. | Linux, Windows, macOS, and Docker |
| Google Cloud Build | No | Cloud | Yes | Serverless platform with charges per build minute. | Docker |
| Bamboo | No | Self-hosted | Yes | 1 remote agent included in the basic price. Pricing for 5 agents begins at $640 per agent per year. | Linux, Windows, macOS, and Solaris |
| JetBrains Space | No | Cloud and self-hosted | Yes | Cloud: 2000 build minutes are included in the free plan. Paid plans start with 4000 minutes, and allow purchasing extra resources – $8 per month for 1000 minutes. On-Premises: 3 concurrent Automation workers are available on the free plan, and from 10 – on paid plans. | Linux, Windows, and macOS |
| Drone | No | Cloud and self-hosted | Yes | Not available | Linux, Windows, and ARM |
| AppVeyor | No | Cloud and self-hosted | Yes | Maximum of 2 concurrent jobs with Premium plans. | Linux, Windows, and macOS |
| GoCD | Yes | Self-hosted | Yes | Free | Linux, Windows, and macOS |
| Codeship | No | Cloud | No | $75 per month for each concurrent build with CodeShip Enterprise. | Docker |
GitHub Actions
The new number 1 overall is GitHub Actions, which remains the most popular tool for personal projects. Using Actions, developers can initiate workflows from events in GitHub, such as pushing to a branch or opening a pull request. As you might expect, GitHub Actions is limited to source code hosted in GitHub repositories.
Pricing: Free for public GitHub repositories. For private repositories, a number of hosted runner minutes are included depending on your pricing plan. Additional hosted runner minutes are also available for purchase, starting at $0.008 per minute (here and below the $ stands for US dollars).
Jenkins
By far the most popular of the open-source CI/CD tools, Jenkins has previously topped the league table of most popular CI/CD tools overall, and it remains the most popular for professional use.
Jenkins is a well-established platform with an active community providing support and maintenance. You’ll find integrations with all major version control systems, together with a wide range of community-supported plugins to customize your Jenkins server.
Pricing: Free to use. The only costs are those associated with running your infrastructure.
GitLab CI
GitLab CI integrates with the rest of the GitLab platform to provide automated build, test, and deployment pipelines. Although primarily designed for GitLab-hosted repos, it can also be integrated with GitHub, Bitbucket, and other Git servers. In addition to the hosted build runners, you can run builds on your own runners at no extra cost.
Pricing: Free tier includes 400 compute minutes per month for running builds on shared pipelines (with one unit buying one minute of execution time on the cheapest runner). Additional units can be purchased starting at $10 for 1,000 units. Paid tiers begin at $29 per user per month.
Azure DevOps
Climbing the rankings from last year is Microsoft’s Azure DevOps Pipelines, which is part of the DevOps suite of products. DevOps Pipelines supports both cloud-hosted and on-premises build agents and provides out-of-the-box integrations for deployments to all major cloud computing providers.
Pricing: Free for open-source projects with up to 10 parallel build jobs. The basic plan is free for the first 5 users, then $6 per user per month with one free pipeline. Additional pipelines for concurrent build jobs start at $40 per month (cloud-hosted) or $15 per month (self-hosted).
JetBrains TeamCity
TeamCity allows you to build code from any source on any infrastructure and get the results quickly. With support for a long list of platforms and frameworks, built-in pipeline optimizations and test parallelization, and integrations for every part of your development toolchain, TeamCity combines speed and flexibility with a sleek UI and full support for configuration as code.
With TeamCity On-Premises, you can choose to run builds on local servers, in a private cloud, or in public cloud infrastructure, with a combination of platforms as required. Deep integrations with major cloud providers allow you to scale your pipeline capacity on demand while keeping costs under control.
A rich plugin ecosystem ensures you can customize your deployment with minimal effort, while the REST API gives you the freedom to integrate your CI tool into your existing toolchain.
TeamCity Cloud delivers all of TeamCity’s powerful functionality as a fully-managed solution. There are no limits on the number of concurrent build jobs, and you can add your own build agents to retain control over your build infrastructure.
Pricing: TeamCity On-Premises Professional includes 3 build agents and unlimited build time for free. Additional build agent licenses start at $299/year.
TeamCity Cloud subscriptions begin at $45 per month, including 24,000 build credits.
CircleCI
CircleCI is a cloud-based CI/CD platform suitable for Git repos hosted in GitHub, GitLab, or Bitbucket. It offers extensive execution environment support together with a web-based UI so you can monitor the progress of builds in real time.
Pricing: Free tier includes 6,000 build minutes per month. Paid tiers begin at $15 per month for 5 users.
Travis CI
Billed as the original cloud-based CI/CD solution, Travis CI offers out-of-the-box continuous deployment to a wide range of platforms together with a number of built-in integrations. A self-hosted solution is also available for Enterprise customers.
Pricing: Begins at $64 per month for consecutive builds and unlimited minutes.
Bitbucket Pipelines
Atlassian’s Bitbucket Pipelines is an add-on to Bitbucket Cloud, allowing users to trigger automated build, test, and deployment tasks on each commit, push, or pull request. Bitbucket Pipelines integrates natively with the rest of the Atlassian suite, including Jira and Trello.
Pricing: Free tier includes 50 build minutes per month. Paid tiers begin at $15 per month for up to 5 users.
AWS CodePipeline / AWS CodeStar
With AWS CodePipeline, developers can trigger builds and run tests on source code from both AWS-hosted repositories and third-party locations, initiate custom functions with AWS Lambda, and then deploy changes to a range of AWS services. If your entire toolchain is in AWS, then AWS CodeStar provides build, test, and deployment capabilities.
Pricing: One free pipeline per month with the AWS Free Tier.
Google Cloud Build
Cloud Build from Google Cloud is a serverless CI/CD platform that allows you to build source code from hosted Git repos and to deploy directly to Google Kubernetes Engine, Cloud Run, App Engine, and more.
Pricing: 120 free build minutes each day. Additional build minutes start at $0.003.
Bamboo
Atlassian’s Bamboo integrates with Bitbucket Data Center, Jira, and Confluence to provide a highly available and resilient self-hosted CI/CD platform. Bamboo is aimed at large corporations looking for enterprise-grade software.
Pricing: Begins at $1,200 for 12 months with 1 remote build agent. Free for registered non-profit organizations.
JetBrains Space
JetBrains Space is a complete software development platform that provides a fast and reliable CI/CD experience with auto-scaling Linux VMs.
Having everything in one place, Space tightly integrates Automation jobs and deployments into chats, code reviews, issues, and other Space features, allowing for faster and more efficient software development.
The Space Git flow allows you to achieve a stable main branch and better code quality by configuring quality gates for merge requests, via either Space Automation or integration with TeamCity.
Set up Automation jobs using the Kotlin DSL, which combines a simple declarative way of describing general cases with the full power of the Kotlin programming language for advanced workflows.
Pricing: The Space Cloud free tier includes 2,000 build minutes per month. Paid tiers start at $8 per active user per month with 4,000 build minutes.The Space On-Premises free tier includes 3 concurrent Automation workers. Paid tiers start at $25 per user per month with 10 workers.
Drone
As a cloud-native CI tool, Drone is primarily designed for running builds in Docker containers and Kubernetes clusters. That said, virtual machine and local build server options are being added and can be used in a non-production context. A community plugin ecosystem provides a number of common integrations.
Pricing: Free for open-source projects, students, and startups. Enterprise Edition plans begin at $300 per month for up to 10 users.
AppVeyor
AppVeyor is a Windows-based CI/CD tool with support for Git, Mercurial, and Subversion repos. AppVeyor build agents run in virtual machines in the cloud or on on-premises infrastructure. With their off-the-shelf pricing plans, individual build jobs are limited to 60 minutes.
Pricing: Free for open-source projects. Plans for private projects begin at $29 per month.
GoCD
The only other open-source CI tool in this list, GoCD, is supported and maintained by a smaller community than that of Jenkins. A limited ecosystem of plugins provides some customization options.
Pricing: Free to use. The only costs are those associated with running your infrastructure.
CodeShip
CodeShip from CloudBees is a cloud-hosted CI tool designed for small and growing teams. It supports Git repos (both self-hosted and common VCS hosts) and provides integrations with common development tools.
Pricing: CodeShip Basic begins at $49 per month for a single user. A Pro version is also available.
How to choose a CI/CD software tool for your pipeline
When choosing a CI/CD platform, consider both your current requirements and how your project is likely to evolve in the future.
Support for your chosen VCS system, programming languages, build platforms, and testing frameworks is a must. However, also bear in mind any plans you may have to migrate to other options in future. Can you combine source code from multiple repos across different version control systems? If you move to a new framework or extend your suite of automated tests, is the CI tool likely to support it? How much freedom is there to customize your pipelines to your workflow or extend the platform’s functionality?
We asked some of our developers at JetBrains how they suggest choosing a CI/CD tool. This response sums up what they had to say:
Choosing a CI/CD platform can depend on many variables.
Are you using a big monorepo but want to make sure your CI/CD only starts for changes in a specific directory?
Which version control systems are in use, and does the tool support them all?
Does the tool quickly support newer technologies, or does it at least let you “shell out” (run a custom CLI tool) to handle your requirements in such a situation?
Having a maintainable CI/CD definition is also important. CI/CD as code is my personal preference, where the definition can be scripted using a programming language like Kotlin instead of a declarative format like YAML.
With CI/CD as code, you can use development tools to refactor and debug build definitions, making it easy to troubleshoot.
Ultimately, choosing a CI/CD platform should be done based on your current and anticipated future needs.
– Maarten Balliauw, Developer Advocate, JetBrains
Why do organizations choose custom CI tools?
Building an in-house CI/CD platform allows you to create a solution tailored to your requirements, and 9% of developers reported using a custom CI tool in 2022.
There are several reasons why some teams decide to create their own solution instead of relying on tools available on the market. One is that an in-house tool can be tailored to the team’s specific needs, processes, and workflows more intimately than any other. It can also be designed to be better integrated into complex systems or meet specific requirements.
However, developing your own CI tool entails ongoing development support and updates in the long term. Additionally, the time required for developers to build and maintain the custom platform needs to be factored in.
How can you improve your CI/CD process using CI tools?
Using a CI/CD platform makes it much easier for development teams to automate their CI/CD process so that builds, tests, and other tasks take place automatically, either based on activity in the VCS or following a regular schedule. CI tools provide configuration and integration options so you can build an automated pipeline much more quickly than if you tried to script everything from scratch.
Furthermore, a CI tool should collect the results of every pipeline run, making it easy to monitor the health of your codebase and measure the impact that a regular build and test process has on code quality.
As your team and codebase grow, a CI tool will allow you to scale your build capacity and run jobs in parallel to ensure continued rapid feedback on the latest code changes. CI tools integrated with other developer tools can be invaluable in helping you deliver quality software.
If you’re looking for ways to get stable builds and improve code quality, JetBrains offers a complete and well-integrated flow with a unified UI – the Space Git flow integrated with TeamCity. Space provides you with Git hosting, code reviews, and quality gates, while TeamCity offers a CI/CD pipeline. Read this article to learn more.
Conclusion
An automated CI/CD process has the potential to improve your software development process dramatically. By providing teams with rapid feedback on their latest work, a CI/CD platform helps you improve code quality while accelerating delivery.
When choosing the best CI/CD tools for your team or organization, looking at the most popular options on the market is a good place to start. However, it’s essential to compare functionality against your own requirements and consider how your projects are likely to evolve in future.
When choosing a CI/CD solution, parameters like language and framework support, performance, and breadth of integrations – from version control systems and cloud infrastructure to issue trackers and messaging platforms – are all vital to consider. Learn more about continuous integration best practices in our CI/CD guide.
What CI tools do you use in your organization or for personal projects? Let us know in the comments below.