Static + Dynamic Code Analysis with SonarQube

Introduction

Programming languages provides a way to write functional behavior using human readable script which further translated into the machine code and executed to perform the expected functionality. Programming language are just way to express the functionality in form of script, so it depends developer-to-developer (not mentioning compiler because it checks only syntax, what should be written to achieve a function output, is not compilers burden) how to instruct the using script for functionality because same thing can be said in many ways.

But here question is what is the optimum ways write a functionality using programming language. Based upon software development experience and learnings so far, developers are suggested to follow:

• Design pattern
• Standard Programming guidelines
• General coding ethics
• Code Review
• Code security and memory optimization etc.

While developing application, it is always a challenge to maintain the code quality. Because, there is always chance to write optimize code. But based upon learnings and recommendation some standard rules have been defined which need to followed while writing code. These rule may also vary language-by-language.

Addition to the how optimized code to be written by following coding guidelines and learnings, code must be scanned for security vulnerability else there may be chance of data leak for example, assume a developer who is writing code has illicit behavior towards business / organization, he/she may write code to pass the data to external database repository while processing the business logic and once he/she leave the organization, he/she may control application, threat to leak the sensitive data.

For code security risk and vulnerability a common list is formed which globally recognized and used to check already identified risk for vulnerability is available or not in given code. This list is called Common Weakness Enumeration (CWE) and governed by MITRE which is non-profit research and development group. MITRE partnered with SANS Institute to list top 25 CWE items called SANS25.

A similar list is provided by Open Web Application Security Project (OWASP) which is also a community-driven compilation of software vulnerabilities which are mainly be applicable for Web Applications.

There are other list which also considered for security like Common Vulnerability and Exposures (CVE) which may be caused by code related misuse as well.

Now, checking the code bases, analyze and identify the written code loop holes from various perspectives (especially from coding ethics, memory optimization, design pattern, deprecation, security and vulnerability perspective) is quite tedious. One way to do the code review but again it will require huge effort and chance to miss.

As solution, a term called static code analysis comes into the picture. There are many tools and utilities which provides feature of static code analysis for different programming language to identity various types of loop holes in code which may be correct for increasing code quality, performance and security.

Below are the few tools for static code analysis:

• SonarQube
• Embold
• Veracode
• Klocwork
• Codacy

The SonarQube which is product of SonarSource is popular solution among all and widely used in software development industry.

About SonarQube

SonarQube (Community Edition) is an open source static + dynamic code analysis platform developed by SonarSource for continuous inspection of code quality to perform fully automated code reviews / analysis to detect code smells, bugs, performance enhancements and security vulnerabilities for 20+ programming languages. It also provides reports on duplicate code, coding standards, unit tests & coverage, code complexity (cyclometric / cognitive complexity), bugs and vulnerability based upon SANS and OWASP list.

It also records historical metrics which were captured with older version of same code base and form the pictorial graphs and drill-down screens which helps developers in fixing those highlighted items.

SonarQube also provides tools / plugins which integrates with different build tools (e.g. Ant, Maven, Gradle, MSBuild, CI Tools like Hudson/Jenkins, Atlassian Bamboo etc.) and perform the code analysis and push the output to SonarQube for further analysis and presentation purpose.

SonarQube analysis covers below dynamic aspects based upon the rules defined into it:

• Test and Code Coverage
• Memory Error Detection
• Faulty Localization
• Invariant Inference e.g. give expression will always true / false
• Concurrency Error which may cause race-condition, resource / memory leak
• Rule based performance analysis
• Security vulnerabilities

Below are the list of static code analysis items which are detected in SonarQube:

• Errors in code / program
• Recommendation on code based upon deprecation or performance aspect
• Metrics computation which give a comparative analysis how well new code is

SonarQube has below menus:

• Project — It provides a dashboard which lists the projects for which analysis data has been posted into SonarQube. It also graphical representation and drill-down facility to go deeper or specific items details.
• Issues — It lists the issues observed into the projects which are further classified into types (Bug, Vulnerability and Code Smell) and also severity associated with it (Blocker, Critical, Major, Minor and Info). There are some other filters on which issued can be filtered.
• Rules — SonarQube runs the analysis based upon defined rules, these rules can be managed (enabled / disabled) by administrator based upon the need.
• Quality Profiles — Quality Profiles can be created based upon the programming language and nature of project where customized set of rules will e applicable.
• Quality Gates — SonarQube provide to define a gate based upon which project can be categorized as PASSED / FAILED. By default a gate is available called ‘Sonar Way’, custom gates can also be created and associated with projects.
• Administration — It provides various options for administrators e.g. Configuration, where list of scope, DevOps integration JaCoCo report path etc. can be customized. Security, it provides role based access management. Projects provides administration of Projects for which data captured into SonarQube like delete of projects. System, provides health statistics for different components used in SonarQube. Marketplace, provides list of plugins available and paid solutions from SonarSource.

Setup SonarQube

SonarSource Provides below products for static code analysis in Sonar Way:

• SonarLint — A plugin for IDE, need to just add plugin into IDE like Eclipse and it will allow to do the static code analysis and show the report at that time. It also highlight the effected code. SonarLint Link for Eclipse: https://www.sonarlint.org/eclipse
• SonarCloud — It is on Cloud version of SonarSource static code analysis product.
• SonarQube — It is on-premise, web-based static code analysis tool. It is mostly used in organization within CI pipeline.

SonarQube has below four offering from SonarSource:

1. Community Edition — Used mostly by organization as it does not require dedicated license. It is free and open source.
2. Developer Edition — Developed specifically for developers.
3. Enterprises Edition — Licensed and designed to meet the enterprises requirements.
4. Data Center Edition — Designed for Data centers for highly available

Note: Detailed comparison can analyzed from https://www.sonarqube.org/downloads/

On Developer machine developer can add SonarLint as plugin by following below steps:

1. Open URL: https://marketplace.eclipse.org/content/sonarlint
2. Open Eclipse / STS / Other Eclipse based IDEs and open Help -> Eclipse Marketplace
3. Drag ‘Install’ button from Web page and drop it to Eclipse Marketplace window. This will start discovering SonarLint plugin.
4. Click on Install Now and then click on Finish. It will ask you consent for installation and then install it in IDE.

Let’s setup SonarQube Community Edition:

1. Download and install Java 11 or later (Java 11 recommended LTS version, Oracle or OpenJDK) Download from: https://www.oracle.com/in/java/technologies/javase/jdk11-archive-downloads.html

2. Verify Java has been installed by running java command as

java -version
it should return output like (depends on version installed):
openjdk version "11.0.1-ojdkbuild" 2018-10-16 LTS
OpenJDK Runtime Environment (build 11.0.1-ojdkbuild+13-LTS)
OpenJDK 64-Bit Server VM (build 11.0.1-ojdkbuild+13-LTS, mixed mode)

3. Download SonarQube Community edition (if you have license/enterprises version, you can download respective version). URL: https://www.sonarqube.org/downloads/. It will download a zip file like: sonarqube-9.1.0.47736.zip

4. Extract the zip file, it will list the artifacts like:

5. Open conf/sonar.properties file and configure below properties

# Default database is used embedded H2
sonar.embeddedDatabase.port=9092
# Default JVM Option
sonar.web.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError
# Sonar host, for local localhost else can provide actual hostname
sonar.web.host=localhost
# Sonar port
sonar.web.port=9000
# Sonar CE JVM option
sonar.ce.javaOpts=-Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError
# Sonar Elasticsearch JVM option
sonar.search.javaOpts=-Xmx512m -Xms512m -XX:MaxDirectMemorySize=256m -XX:+HeapDumpOnOutOfMemoryError
# Sonar search port - on which Elasticsearch is running on local
sonar.search.port=9001
# Sonar search host - on which Elasticsearch is running
sonar.search.host=localhost
# Sonar data and temp path
sonar.path.data=data1
sonar.path.temp=temp1

SonarQube has internally four sub-systems: web-application, compute-engine, database and search engine

SonarQube internally uses a database to store its data, by default H2 database is used. But it can be configured based upon choice and SonarQube supports (Oracle, PostgreSQL and Microsoft SQLServer).

SonarQube internally uses Elasticsearch for indexing data and searching when users search from SonarQube web portal. Normally, it runs with SonarQube, but it can be separated. By default, it sticks with SonarQube and runs when SonarQube started.

Web application is mainly accessed by the users and web application internally uses the compute engine. Although, it runs internally, but can be configured on custom ports.

6. Open bin directory, it has binaries / startup script for Linux/Mac/Windows

7. Open respective directory based upon OS and start below script based upon OS:

• StartSonar.bat for windows
• sonar.sh for Mac or Linux

Wait for few minutes, as SonarQube starts its components sequentially. Finally your console looks like:

It means SonarQube is running.

8. Launch web browser e.g. Chrome and open http://localhost:9000/

By default, SonarQube creates an administrator user with below details:

Login: admin

Password: admin

After, first login you can create many users and setup viewing permission.

Use Cases

As discussed in previous section, SonarQube can be used for different programming language static + dynamic code analysis. Below are few use cases with different programming languages:

Java with Maven Build Tool

Prerequisite:

• JDK must be installed
• Maven must be installed and configured

Below are the steps required for SonarQube analysis with Java based project:

To perform SonarQube analysis with Java and Maven, need to add below plugins:

<plugin>
   <groupId>org.sonarsource.scanner.maven</groupId>
   <artifactId>sonar-maven-plugin</artifactId
   <version>3.4.0.905</version>   
</plugin>

Additionally, SonarQube internally uses JaCoCo reports for code coverages to JaCoCo plugins must be available:

<!-- Please define exclusion rules properly. If exclusion not required remove it. -->
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>${jacoco.version}</version>
    <executions>
     <execution>
      <id>prepare-agent</id>
      <goals>
       <goal>prepare-agent</goal>
      </goals>
     </execution>
     <execution>
      <id>report</id>
      <phase>prepare-package</phase>
      <goals>
       <goal>report</goal>
      </goals>
      <configuration>
       <excludes>
        <exclude>**/*com/sk/config/*.*</exclude>
       </excludes>
      </configuration>
     </execution>
    </executions>
    <configuration>
     <systemPropertyVariables>
      <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
     </systemPropertyVariables>
    </configuration>
   </plugin>

[Note: Check surefire plugin inclusion / exclusion rules, surefire plugin specified explicitly]

Define the SonarQube properties in properties section:

<properties>
  <jacoco.version>0.8.5</jacoco.version>
  <sonar.sources>src/main/java</sonar.sources>
  <sonar.host.url>http://localhost:9000/</sonar.host.url>
  <sonar.tests>src/test/java</sonar.tests>
  <sonar.login>admin</sonar.login>
  <sonar.password>admin</sonar.password>
</properties>

Now, run below command:

mvn sonar:sonar

It will invoke SonarQube analysis and post results into the SonarQube. After completion of maven command, it will list project analysis on SonarQube dashboard as:

Please refer work on GitHub for details: https://github.com/siddhivinayak-sk/jwt-openid-oauth2.0-keycloak-kerberos-ntlm/tree/main/spring-boot-jwt

Note: In case of any issue, exception, please investigate SonarQube credentials and URL defined in properties and check if given URL is accessible.

NodeJS with Angular Project

Prerequisite:

• NodeJS and NPM
• NG CLI

Below are the steps code analysis:

Install Sonar plugin and Karma SonarQube unit reporter:

npm install -g sonar-scanner --save-dev
npm install -g karma-sonarqube-unit-reporter --save-dev

Note: Karma is test suite for Angular projects which must be configured so that it can run tests and create reports which further published to SonarQube.

Modify karma.conf.js and add SonarQube unit reporter:

plugins: [
      require('karma-sonarqube-unit-reporter')
    ],
    sonarQubeUnitReporter: {
      sonarQubeVersion: 'LATEST',
      outputFile: 'reports/ut_report.xml',
      overrideTestDescription: true,
      testPaths: ['./src'],
      testFilePattern: '.spec.ts',
      useBrowserName: false
    },
    reporters: ['sonarqubeUnit'],

Change in package.json and add sonar under scripts:

"scripts": {
"sonar":"sonar-scanner"
}

Create a properties file for providing SonarQube configuration:

sonar.host.url=http://localhost:9000
sonar.login=admin
sonar.password=admin
sonar.projectKey=my-angular-app
sonar.sourceEncoding=UTF-8
sonar.sources=src
sonar.tests=src
sonar.exclusions=**/node_modules/**
sonar.test.inclusions=**/*.spec.ts
sonar.typescript.lcov.reportPaths=coverage/lcov.info
sonar.testExecutionReportPaths=reports/ut_report.xml

Run below commands to perform test and Sonar analysis:

ng test --code-coverage --watch=false
npm run sonar

This command will do analysis and push output to SonarQube like:

Please refer work on GitHub for details: https://github.com/siddhivinayak-sk/node-angular-front-end

.NET with C#

Prerequisite:

• .NET SDK
• Install sonar-scanner-msbuild

Generate token in SonarQube:

• Open SonarQube -> Login with Administrator -> My Account -> Security -> Generate Token
• Provide token name
• Click on generate button, it will generate a token which will further used in dotnet sonar-scanner/SonarScanner.MSBuild.dll

Run below command to perform Sonar analysis:

dotnet test
dotnet /sonar-scanner/SonarScanner.MSBuild.dll begin /k:MyProject /d:sonar.host.url='http://localhost:9000' /d:sonar.login='c87723110deab90be1520f7554dd3b85b7b82427'
dotnet build
dotnet /sonar-scanner/SonarScanner.MSBuild.dll end /d:sonar.login='c87723110deab90be1520f7554dd3b85b7b82427'

After running these commands, code will be analyzed and publish output to SonarQube like.

Note: In reference project, we have build C# project on Linux environment. Please see Dockerfile.build and Jenkinfiles which creates environment and run the analysis. The argument host.url is passed on which SonarQube is running.

Please refer work on GitHub for details: https://github.com/siddhivinayak-sk/dotnetmicroservice

Note: SonarQube supports many programming language. We have added use case with Java, Node + Angular and .Net with C#. Based upon request, we will add more use case.

Conclusion

Maintaining the source code quality for project is a common requirement for software development. The code quality may refer to various aspects e.g. bug, security vulnerability, resource / memory leakage, deprecation, performance improvement and so on.

The manual code review with human resource is quite tough as it requires significant effort of expert developers and reviewers and still there is chance to skip few scenarios.

Static code analysis by using automated analysis platforms is one way which saves much efforts and majority of cases covered quickly.

SonarQube (CE) is most popular and widely used code analysis tool which provide rich set of feature for code analysis and provide the dashboards and charts to present data.

About the Author

Sandeep Kumar holds Master of Computer Application degree working as Java developer having 10+ years of working experience. He has experience design and development of enterprises applications in domains like education, content, laboratory, and banking; got various appreciation for his solutions including spot appreciation for Glassfish to JBoss migration project. He secured Google Cloud Developer certificate and participated into OCI trainings. He is a part of HCL-ERS platform as Sr. Lead developer.