Continuous analysis with Kiuwan and Visual Studio Team Services (VSTS)

Microsoft is not that company neglecting everything that has anything to do with the Internet anymore. They didn’t catch the Internet wave when they had the chance back in the day, and they’ve learnt from their mistakes.

Now they are in the forefront of the cloud wave. Besides the infrastructure and platform as a service they offer with Azure, they are doing a remarkable job with platforms for developers in the cloud and Visual Studio Team Services (VSTS) is their flagship product.

Not long ago I published a post on how to configure TFS 2017 to run continuous analysis as part of your builds. It’s been quite a success and we are getting a lot of feedback about it. Some of that feedback comes from people using the cloud SaaS flavor of TFS, Visual Studio Team Services (VSTS).

I figured it was worth to give it a try, so I did. The experience with the on-premise TFS helped a lot to get started, since the concepts are basically the same, the difference is that VSTS is fully hosted by Microsoft. Forget about the “pain” of installing and configuring the product, one of the benefits of a cloud services: everything in that respect is taken care of for you. You just have to go to click “Get started for free”, create your account and off you go (kudos to Microsoft for making it so easy). Of course if you already have a Visual Studio account use it to start your Team Services. You will be redirected to your own VSTS instance with your own URL (mine is


We are ready to roll.

What’s the difference between TFS and VSTS?

The concepts are the same: Projects, build definitions for them, work items, agents, etc. The user interface is functionally similar if not the same, but there are subtle differences that make VSTS cleaner and easier to use.

The agents is where the biggest difference lays. With FTS you were responsible to install them and configure them. With VSTS, that is not necessary, they have the concept of Hosted Agents. These are ready to use agents provisioned on the fly when necessary with everything you need pre-installed. Here you can find more about hosted agents with the full inventory of installed software for every type (yes, there are different types of hosted agents). We’ll take a closer look at these ones later on, now I would like to talk about Private Agents.

Private Agents are agents that you install and manage yourself in your infrastructure. They are in fact like FTS agents, but instead of configuring them to be connected to your in-house TFS server they are configured to connect to your VSTS account (you need a PAT generated in your VSTS account to configure them). Here is you can find the instructions to configure your private agents. It is very straight forward. With private agents you have full control on the software you install that is necessary for your builds like the Kiuwan Local Analyzer (KLA). So configuring a VSTS private agent to run Kiuwan analyses with the KLA, requires that you install the KLA in all agent machines, the same way as for FTS agents. You can find more about this in my FTS and Kiuwan integration article.

Now, hosted agents. To run Kiuwan analyses on agents, of any type, you need the KLA. But hosted agents are not machines you can login to and install software. What can you do then? The answer is simpler than you may think: Download and install the KLA as part of your build. This obviously adds another couple minutes overhead in your build but it is more than justified given the benefits you get by running a Kiuwan analysis.

Once, the differences are understood creating and configuring build definitions to run automated Kiuwan analyses in a walk in the park.

Automated Kiuwan analysis with VSTS step-by-step

1. Install the KLA in the private build agent machines (if you want to use them)

If you are going to use only hosted agents you can skip this.

Installing the KLA is pretty simple and straightforward, just take the downloaded directly from your Kiuwan account and unzip it in a directory of your choice. That directory /KiuwanLocalAnalyzer will be your Kiuwan home (sometimes we call it agent home, to avoid confusion with the TFS agent I’ll stick to Kiuwan home). In Windows machines the KLA installation is done just unzipping. For unix based machines there are a couple of little things to do:

  • Go to your <Kiuwan home> directory and run a chmod command for all *.sh files to add execution permission (find -name “*.sh” -exec chmod +x {} \;)
  • Now, go to <Kiuwan home>/bin directory and create a symbolic link named agent pointing to (ln -s agent).

To be able to invoke the KLA CLI in every machine without the installation path, add the <Kiuwan home>/bin directory to the system path.

Now the steps to configure the build definitions. They will run in the agents you want depending on the agent queue you assign to the definition. All steps here apply for private and hosted agents unless explicitly noted.


2. Select an existing project in VSTS or create a new one

In VSTS every project must have one or more Version Control repositories associated to it. This way the build agents take care of stracting the source code automatically. In VSTS the repositories are hosted and you can choose from Git and Microsoft TFS repositories.


For existing projects, most probably the repositories will be configured and the code should be already there. For new projects you can add the code to version control from Visual Studio (provided you’ve connected Visual Studio with your VSTS account, of course).

The repositories you configure here will be the ones that new Build definitions will use by default.

3. The next step is to configure a new build task

that will run the Kiuwan analysis in a project existing build definition, or create a new build definition to include the Kiuwan analysis task.



Click the build definition edit link to add new build tasks.

4. Define the process

When you create a new build or you are editing an existing one, the first thing is to define the process. Which basically boils down to define the agent queue that will be used to run the build. This is the key to decide if your build will run in a hosted agent or a private agent.

Here you can see how you can pick from a list of hosted agents or private ones (default).



5. Define build specific variables

Before you jump to define the tasks to run in the build, you have to define a couple of variables that will be available for the build agent (any build agent) at execution time to hold your Kiuwan credentials necessary to run the analysis. Look for the variables section in the top Build definition submenu.


Using the add variable link, create the KiuwanUser and KiuwanPasswd variables (you can choose other names of course). Just make sure click the lock icon to the left of the password variable value to hide it and never displayed.

6. Let’s configure a new build task

Go back to the task configuration, first option in the top Build definition submenu, to configure a new build task. but first things first. You can see that the first thing is to download the source code.


As you can see in the above screenshot there are different repositories supported, starting from the one configured in the project, but VSTS projects can have their code hosted somewhere else. This is a very flexible feature.

Another cool feature is that you can define build phases, and each phase can assigned to a different agent queue (the default is to inherit from the build definition queue). This is not supported in TFS 2017 and again adds another level of flexibility to manage your SDLC.

Depending on the build definition template you selected to create your build, you will have some pre-configured tasks. In the screenshot above you have the tasks for a Visual Studio build grouped in Phase 1. Click the + sign at the right of the phase to add a new task. The tnew task we are going to creta is different depending on the type of agen the build is going to run on.

For Private Agents

If the build is going to run on a private agent (remember this depends on the agent queue configured in the build definition or in the phase), you need to select the Command Line task in the Utility section. We are assuming the KLA installed in the agent machines.


In this case, what makes sense is to run a Kiuwan analysis right after the Build solution task. You just have to click on the newly created task and drag it right after the desired task.

To configure it, first give it a name like ’Run Kiuwan Analysis’, the Tool to run will be agent (this is the KLA command). And the arguments passed to the Tool are as follows:

-n $(System.TeamProject) -s $(Build.SourcesDirectory) -l "VSTS build $(Build.BuildNumber)" --user $(KiuwanUser) --pass $(KiuwanPasswd) -wr

Notice that we are using System and Build predefined variables exposed by VSTS for the name of the application to analyze (it will be the project name), the directory where the source code is (it will be the checkout directory), and the label for the analysis (it will be the build number). Finally, we use the 2 variables we previously defined for the Kiuwan credentials. Using variables ensures consistency for every analysis and make them independent of the build agent machines that run them.


There are another couple of things you can configure in the task. The most relevant one is the Continue on error in the control options. Check it if you want to continue with the build even if the analysis fails (warning: that the analysis fails means that it could not run for some reason, it has nothing to do with the results of the analysis. Conditioning a build based on analysis results is a different matter that require extra tasks).

This configuration will run a Kiuwan Baseline analysis. If you want to run a delivery analysis for a change request you can define another build configuration where the analysis task will have extra parameters for the KLA agent to run a delivery analysis. Refer to the Kiuwan Local Analyzer documentation for more information on the available parameters.

For Hosted Agents

If the build is going to run on hosted agents, the task configuration is different. Starting with the task type, now you have to select the PowerShell task again in the Utility section.


It still makes sense to run a Kiuwan analysis right after the Build solution task. So click on the newly created task and drag it right after the desired task.

The PowerShell task allows you to run a PowerShell script from a file (it has to exist in the agent machine*) or that you define inline in the task definition. Since we don’t have access to the hosted agent machine we are going to define it inline.

* NOTE: Something you can do is to have the PowerShell script to run in the source code repository for your project. This is not a bad a approach since you can maintain versions of the script and change the behavior of the Kiuwan analysis for different versions of your project. This a tendency we are seeing where build and deploy configurations are part of the application code.

Why a script this time? Because we need to do several things: Download the KLA, install it and run the analysis The easiest way to do this is with a PowerShell script.


Notice that the version of PowerShell to use is 2.* (although in preview it works fine and it is required because of some of the commands we need to use). Give the task a name like ‘Download and run KLA’, specify Inline as type and put the following script in the Script textarea.

# PowerShel script to run a Kiuwan baseline Analysis
Write-Host "Downloading KLA..."
Invoke-WebRequest -OutFile
Write-Host "Installing KLA..."
Expand-archive .\ -DestinationPath .
Write-Host "Running KLA..."
.\KiuwanLocalAnalyzer\bin\agent.cmd -n $(System.TeamProject) -s $(Build.SourcesDirectory) -l "VSTS build $(Build.BuildNumber)" -c -wr --user $(KiuwanUser) --pass $(KiuwanPasswd) supported.technologies=abap,actionscript,aspnet,c,cobol,cpp,csharp,html,informix,java,javascript,jcl,jsp,natural,objectivec,oracleforms,perl,php,plsql,powerscript,python,rpg4,ruby,transactsql,vb6,vbnet,xml exclude.patterns=**/KiuwanLocalAnalyzer/**,**/*.min.js,**/*.Designer.vb,**/*Reference.vb,**/*Service.vb,**/*Silverlight.vb,**/*.Designer.cs,**/*Reference.cs,**/*Service.cs,**/*Silverlight.cs,**/.*,**/Pods/BuildHeaders/**/*.h,**/Pods/Headers/**/*.h include.patterns=**/** encoding=UTF-8 timeout=3600000

First we use the Invoke-WebRequest command to download the KLA from Kiuwan. The we invoke Expand-archive to extract the KLA into the build working directory. And finally we execute the KLA CLI .\KiuwanLocalAnalyzer\bin\agent.cmd with the appropriate parameters to run the analysis.

Notice that in an inline script you can directly use the variables* exposed by VSTS and the ones defined in the build definition.

*NOTE: If you take the approach to have the script in a file in your repository you have to pass all these variables as arguments to the script and do the correspondent treatment to pass them to the KLA in the script.

After every build run all this is decommissioned and cleaned up. That’s why we need to download, install and run with every build.


7. The last thing is to define triggers to run the build on the project on given conditions.

The possibilities here are:


  1. Continuous integration (CI): Run the build when changes are checked-in in the repository (this will work when checking-in changes from Visual Studio)
  2. Scheduled: Run the build periodically based on a defined schedule.
  3. Gated Check-in. Run the build every time a developer tries to check-in code. If the build fails (i.e. the changes don’t pass a Kiuwan audit), the code is not checked-in.

Or any combination of the 3. This is a cool feature. The triggerconfiguration is independent of the type of agent that will run the build.

For testing purposes, you can always run your build clicking the Save & queue… link on the top right of the above screen.


Congrats! You have a new build definition associated to a project in VSTS that will run on private agents or hosted agents depending on how you configured it.

Run a build and check Kiuwan results

To run a build just que ti from VSTS. Alternatively, you can run a build from Visual Studio if you configured the appropriate triggers (CI trigger). This is the same as running a TFS build from Visual Studio (See my Kiuwan and TFS 2017 integration article). The only thing you have to be careful with is that you are connected to your VSTS account from Visual Studio.

This will be the output from a build run on a hosted agent:


And these are the results in Kiuwan after the run.


So there you go! Your applications continuously analyzed with Kiuwan using VSTS in the cloud.

Once you know how to configure a Kiuwan analysis as part of a VSTS Build definition the possibilities are endless. It all depends on how exactly you want to drive your software development process and your continuous integration pipelines.

And going beyond with the REST APIs from Kiuwan and VSTS you can build custom integrations to, for example, create work items in your VSTS projects from Kiuwan action plans. The sky’s the limit…