DevOps & DevSecOps new players: GitLab CI&CD, GitHub Actions, BitBucket Pipelines

Introduction

Technology is growing rapidly which drives improvements in ‘development technologies’ as well. We see new features in programming languages, inference engines enhancements, new capabilities, optimizations an so on. And this is quite normal and need as well. There is only one reason behind ‘these improvement and enhancement’ is ‘Ease in life’. Software development stakeholders like product owner, DevOps, architects, developers are also contender for making their life easy. Additionally, it is quite common that new challenges are also added to developers very frequently like security, performance bottleneck, expansion etc. which also needs to managed properly with limited human and compute resources. These all needs requires enhancement in ‘Development Automation’ as well.

We know “DevOps is a practice which composed of Process, People and Toolset and which combined the software development and IT Operation together to achieve high quality, rapid software development with high use of automation”.

With the growing need and challenges, DevOps toolsets are also being evolved. These toolsets have been provided by different vendors and each vendors is striving to deliver cutting edge of their DevOps products which breaks the barrier of traditional technologies and toolset.

We will discuss few of the emerging toolsets which are growing rapidly in DevOps and Software development field.

To understand more about DevOps, please follow article: https://siddhivinayak-sk.medium.com/continuous-integration-continuous-deployment-ci-cd-with-jenkins-sonarqube-nexus-docker-and-45a309bde131

Edge DevOps Tools vs Legacy Tools

There are several tools / toolsets are available which provide solution for setting up DevOps/DevSecOps environments.

Traditionally, there was huge hold of Hudson/Jenkins tools for CI & CD automation and Subversion (SVN)for source code repository. But now, these are being treated as legacy tools because it serves a one or two purpose only. Example, SVN can only manage the source code with legacy feature support however modern Source Code Management (SCM) system provides many features along with source code management which not only helps in DevOps practice setup but also helps in bug/incident management, test management, project & spring management for example GitLab.

Since Agile and Scaled Agile Frameworks (SAFe) are taking places over traditional software development approaches like SDLC waterfall model there is a need for a complete platform which provides following:

• Source Code Management (SCM) — where source code can be stored and accessed whenever required
• Continuous Integration (CI) Practice — which provides code compile, build, unit test run, package etc. of software project source code
• Security Practices— Security is major concern from various perspectives like data storage, access grants, dependency check, code security, platform security, data security etc. It has huge scope and created an essential space along with DevOps called “DevSecOps”
• Continuous Delivery (CD) Practice— when software product is packaged successfully, it needs delivery to the target environment which may be on premise or Cloud environment.
• Feature, Bug & Event/Incident Management — a development project also needs feature, Bug and Incident management tool so that these can be managed effectively for the project
• Project Management — a sophisticated tool required for Agile based project management which have capabilities to plan, create Sprint Iterations along with estimation, burndown, dashboarding, reporting capabilities
• Documentation — documentation is a need which begins from project start. A sophisticated document management is need so that all project related documents can be stored and accessed as per need and grant.

These features are being provided by using various toolsets from same/different vendors. We can understand these needs using DevOps practice diagram:

Figure 1: DevOps processes mapping with toolset

We can see here, Jenkins is a build automation tool which does not have SCM capabilities solely.

Now question is, “When an DevOps/DevSecOps requires such practices regularly for development why a single platform is not providing these features”.

There are many DevOps tools vendors tried to answer the question by creating a single platform which can be utilized for most of the DevSecOps practices without having additional toolsets. Some of the such tools are:

• GitLab
• GitHub
• BitBucket

and so on.

These new tool / software platforms are providing/provided features which works like complete suit of tools used for software development. Many of these tools are not enough mature yet, but continuously being enhanced with features to make this goal achieved for single platform for complete DevSecOps and project management platform.

Note: Since these new tools like GitLab provides feature of various tools as SUIT of tools, we can refer it as a complete platform.

“What benefits we can get with single platform for complete DevSecOps?” It is a very necessary question because we already have may open source or paid tools to achieve the DevOps/DevSecOps practices. Answer is, “to reduce the development cost”. How we reduce cost, please follow the benefits for these new tools.

Why we should adapt/move to such platforms? There are various factors which we can consider:

• Compute Cost — these platform has been designed with latest technologies & support containerization and requires less compute resources as it works as single software for complete platform. Therefore, instead of installing various different tools, we have to install a single tool which reduces physical resource uses and save compute cost.
• Maintenance Cost — if there are multiple tools for providing same feature, we need separate effort for each tool for manage, maintenance, upgrade etc. If we have single software for various features, although it will increase complexity but it will reduce the touch surface which reduce the maintenance cost.
• Integrations with New Technologies — most of these platforms are either in development/incubation/developed recently which uses recent technologies and provides easy integration with latest technologies like build tools, security tools etc.
• Easy to Use — it is very obvious, if we have multiple tools, we need to learn these for our daily work. If we have only one, need to work on single platform which relatively reduces the complexity.
• Sophisticated for Agile/SAFe — these new platforms are highly curated for latest development models like Agile and Scaled Agile Framework (SAFe). This makes easy project management with advance features and Agile tools.
• High Performance — based upon points related to technology, complexity and containerization support; the performance is increased.
• Quality — these new tools also has quality improvement capabilities by using integrated plugins. It can be used for code quality, overall package quality improvements. High level of automation also is factor of quality improvement for these tools.

These benefits provides us ease, less effort, less resources and rapid development cycles which turns out as financial benefits for the software development which is required for any organization.

Let’s begin journey with these platforms in subsequent sections.

Tools and Usage

There are many Edge DevOps platform has been developing/developed by different software vendors which are being popular these days. Each platform provides features to support SCM, CD & CD Pipeline along with project management. Let’s discuss some of them in detail:

1. GitLab

GitLab is an organization which provides a platform called ‘GitLab’ which provides complete DevSecOps practice tools used to develop, secure and operate a software product.

It has been initially developed by Ukrainian developers in Ruby, Go and Vue.js. It follows open code development model where core functionalities are released as MIT open source license and other advance features are released under proprietary license. It provides hosting like on premise, on Public Cloud and GitLab hosted software as service.

GitLab platform provides below major features:

1. Project Management — A project/sub-project can be created, managed, member and activity management. A project group can be created for containing another sub-group or project.
2. Repository — Complete source code management based on popular SCM technology GIT. It has many advance feature than normal GIT e.g. Branch, Tag, Merge using web based user interface
3. Issue Management — It contains Issue creation and management along with Boards, Service Desk and Milestones. These features are used for Agile activities like backlog creation, spring iteration, boards etc.
4. CI & CD — It provides feature for creating pipelines for the source code. The pipeline follows a YAML based DSL for ease of writing pipeline. It is so easy, NO NEED TO LEARN ANY SCRIPTING. Provides advanced features like Job, Pipeline (consists of Jobs), Schedule etc.
5. Security & Compliance — It provide scan, discover, audit, configuration of the projects. Supports various type of security testing like Static Application Security Testing (SAST), Infrastructure as Code (IaC)Scanning, Dynamic Application Security Testing (DAST), Dependency Scanning, Cluster Scanning, Container Image Scanning, Secret Detection, API & Coverage Fuzzing.
6. Deployments — It can be configured to deploy on various type of environment including all major Cloud support. It also provides release management.
7. Package & Registries — Provides registry services like Nexus where image registry, package registry and infrastructure Registries are available.
8. Infrastructure — Connect with Infrastructures like Kubernetes or Terraform where source code can be used to create infra objects to deploy application.
9. Monitor — Provides environment monitoring tools where application can be monitored. It can be configured for error tracking and alert.
10. Analytics — It provides data about various projects, repositories, pipelines statistics which are created in the GitLab.
11. Wiki — A complete solution for documentation. Can create wiki pages with formatted content and attachments.
12. Snippet — Code snippets can be created for various documentation and sharing purpose.

Below is the home page of GitLab:

Figure 2: GitLab Home Page depicts ‘Menus and Features’

GitLab is highly mature platform for various backend and front end application development. It has their own concept of DevSecOps practice named “InnerSource” practice which is same as DevSecOps with some GitLab features.

It can be used very easily for Agile and SAFe based software development and there will not be any additional tool required except application deployment infrastructure.

GitLab provides free services for public projects with some basic features along with 50k pipeline runs for a year.

Link: https://gitlab.com/

2. GitHub

GitHub is also a platform which provides complete project management, source code management, DevOps practices and behaves as single platform for DevSecOps.

It is quite similar to GitLab where feature name is different e.g. GitLab Pipeline called here GitHub Actions. GitHub became subsidiary of Microsoft in 2018.

It provides SCM based on popular SCM tool GIT with web based user interface. Following are the major features provided by GitHub:

1. Code — It provides source code management where code can be added by developers, branches can be created, code merged from one branch to another branch. Provides basic editor for file on web based interface.
2. Issue — Provides Issue tracking and management system. Issue can be created with various labels and other metadata.
3. Projects — It provides project management where backlogs can be created, features can be created, table and boards used for Agile based development.
4. Wiki — Documentation feature where pages can be created for documentation.
5. Security — Provide security policy definition, security advisories, dependabot alerts, code scanning alerts.
6. Insights — It provides various data related repositories, dependency graph, Network, traffic etc. used for viewing purpose.
7. Collaborators — Provides feature to manage members.
8. Alerts and Notification — Provides alerts and notification based upon configuration and activity/events.

It provides various advance features for workflow customization with integrated tools called GitHub Apps.

Tags, releases, packages publish is also supported.

Below is home page of GitHub:

Figure 3: GitHub home page for repository

GitHub provides various commercial feature along with free features for public repositories. One can create repositories, project and pipeline free with some limitations.

GitHub is quite similar to GitLab but its features are evolving to cater the Agile based projects.

Link: https://github.com/

3. BitBucket

BitBucket is a product of Atlassian which earlier used only for Source Code Management (SCM) but now it has been evolved more by Atlassian and now it provides a platform where projects, repositories, CI & CD pipelines, deployment connection, issue tracking (with help of JIRA) etc.

It is also similar to GitLab & GitHub where pipelines are created with YAML configurations. Below are major features in BitBucket:

1. Source — View, create file etc. for source code management
2. Commits — List commits of branch
3. Branches — List branches for the repository
4. Pull Requests — Pull Requests list which provides review and merge code from branches
5. Pipelines — Yaml based configuration for pipeline steps creation
6. Deployments — Configure and deployed with pipeline to the environments like Cloud environment
7. Jira Issues — Issue tacking with Jira. BitBucket currently itself not providing issue tracking similar to GitHub or GitLab so it integrates with Jira
8. Security — It provides security integrations tools, scanning for opensource vulnerabilities and container scan
9. Downloads — Download source based upon tags branches
10. External Tools Integrations — Currently, BitBucket is evolving and new features are being added. Till the time Atlassian provides other tools integration with it like, Jira for Project Management and Issue Management, Confluence for document management.

Below is the home screen of BitBucket:

Figure 4: BitBucket home page

Currently, BitBucket is not providing complete solution for DevSecOps solely but seems Atlassian is evolving BitBucket with new features to make competitive with other alternative platforms.

Link: https://bitbucket.org/

Conclusion

With the time, the CI & CD automation tools are evolving to make single development platform for Agile and Scaled Agile Framework (SAFe) development approach.

With legacy DevOps & DevSecOps toolsets has hundreds of different tools needed to complete the DevSecOps practices. Now, different vendors are gearing up with new tools which provides complete suite of features for DevSecOps.

GitLab, GitHub and BitBucket are spreading rapidly because of its easy YAML based pipeline declaration. These are also providing integration with deployment environment like on premise and Cloud.

Single platform will require less compute resource and effort for management. These platforms are being developed using latest tech stack therefore it provides high performance and reduces overall cost for software development.

Links of Work

1. GitLab Project and Repository based on Java and Maven — https://gitlab.com/springapplicationgroup/springfirstproject
2. GitHub Project and Repository based on Java and Maven — https://github.com/siddhivinayak-sk?tab=projects&type=classic , https://github.com/siddhivinayak-sk/spring-reactor-reactive
3. BitBucket Project and Repository based on Java and Maven — https://bitbucket.org/my-sk-projects/product-service

References

• Build Your CI & CD Pipeline with Jenkins — https://siddhivinayak-sk.medium.com/build-your-ci-cd-pipeline-with-jenkins-2dc082162f86
• GitLab — https://gitlab.com/
• GitLab CI & CD Pipeline Configuration — https://gitlab.com/springapplicationgroup/springfirstproject/-/blob/master/.gitlab-ci.yml
• GitHub — https://github.com/
• GitHub Actions — https://github.com/siddhivinayak-sk/spring-reactor-reactive/blob/main/.github/workflows/maven.yml , https://github.com/siddhivinayak-sk/spring-reactor-reactive/actions
• BitBucket — https://bitbucket.org/

About the Author

Sandeep Kumar holds Master of Computer Application degree working as Java developer having 11+ years of working experience. He has experience design and development of enterprises applications in domains like education, content, laboratory, and banking; got various appreciation for his solutions. He secured Google Cloud Developer certificate and Azure Cloud Certificates.