Installing and Configuring the Invicti Enterprise Scan TeamCity Plugin

TeamCity is a build management and CI server that allows users to run automated tests on the server after committing code but before going into production.

These instructions assume you are familiar with the basic concepts of Invicti Enterprise and TeamCity, such as Websites, Scans and Build Configuration.

You must also have:

• A Invicti Enterprise account
• A TeamCity account with Administrative privileges
• Knowledge of PowerShell or at least basic programming understanding

First, we will make some small settings on the Invicti Enterprise website:

• We will define the website we want to scan every night and note down our Invicti Enterprise API credentials which will be required in the following step (see Preliminary Actions in Invicti Enterprise).
• Next, we will write a PowerShell script to launch a new scan, query the status of the scan and download the scan report file.
• Finally, we will create a Build Configuration in TeamCity which will be executed nightly. To keep it simple, we will trigger the Build Configuration nightly. With a little bit of configuration change it can also be triggered just after a git commit.

Preliminary Actions in Invicti Enterprise

Regardless of whether you choose to run a Full Scan or an Incremental Scan, there are several things you need to do first:

• Add a website to Invicti Enterprise and verify ownership:
  • See Adding a Website in Invicti Enterprise.
  • Later, you will invoke some REST API methods. Just like any other REST API, the Invicti Enterprise Web API requires some credentials. You can find them in API settings.
• Run a Invicti Enterprise Scan (see Introduction to Scans and Creating a New Scan)
  • Find the Scan Id in the Scan Report – Incremental Scan (see How to Find the Scan ID of a Report)
    • For further details on the results of the scan, you can open a scan report in Invicti Enterprise from which you can extract the ScanId
  • The reason for launching Incremental Scans rather than Full Scans from TeamCity, is that Full Scans often take too much time, whereas Incremental Scans are completed much quicker.
• Access the Invicti Enterprise API Credentials, including User ID and Token (see How to Access Your API Credentials
  • Authentication to the API is via HTTP Basic Access Authentication:
    • When you make any calls to the API, you will need to provide your User ID and API Token in the HTTP Basic authentication header, in this format (where {XXX} is your Base64-encoded USERID:API-TOKEN)
      • Authorization: Basic {XXX}

Now, you are ready to proceed with one of the following:

• Full Scan
• Incremental Scan

Full Scan

First, ensure that you have completed the Preliminary Actions in Invicti Enterprise.

At the end of this tutorial, you will have a TeamCity Build Configuration which will be executed nightly. This Build Configuration will launch a scan by invoking the Invicti Enterprise Web API and will display the Scan Report in a Report Tab in TeamCity.

Invoking Invicti Enterprise Web API Using PowerShell

If you are not interested in how PowerShell script and Invicti Enterprise Web API works, and just want to achieve the end result, go directly to the Final Script, and replace the parameters with your own values in the $headers and $newScanBody variables.

Invicti Enterprise exposes a Web API that allows client applications to view and manipulate scan tasks.

To invoke the Invicti Enterprise API, we use PowerShell, because TeamCity is able to execute PowerShell scripts as a Build Step.

For further information about the Invicti Enterprise Web API, see API v 1.0 Documentation.

The script does the following:

• Launch a new scan
• Check the status every 10 seconds, until the scan is finished
• If the scan finishes successfully, download and save the Scan Report as an html file

We will invoke three different methods from the API:

• The first method creates a new scan
• The second method checks the status of the scan
• The third method downloads the Scan Report

Preliminary Steps

Before starting to write PowerShell:

1. Create a .bat file called scan.bat. Add the following command to the file:
   • pausepowershell -ExecutionPolicy ByPass .\scan.ps1

This is the PowerShell command that executes the script. This .bat file will ensure that we do not have to enter the same command over and over during tests

2. Create a second file named 'scan.ps1' in the same directory as 'scan.bat'. All the PowerShell commands will go into this file.
   • To keep the PowerShell script clean, you need to define some values at the top of the .ps1 file. Add the following line in the top of .ps1 file:
     • $baseUri = "https://www.netsparkercloud.com/api/1.0/scans/"

This is the base URI of the Invicti Enterprise Web API. All addresses we will be invoking will start like this.

• Then add the following under $baseUri line (replacing Your Base64 Encoded USERID:API-TOKEN with yours from your API Credentials):
  • $headers = @{
        "Authorization"="Basic [Your Base64 Encoded USERID:API-TOKEN]"
        "Content-Type"="application/json"
}

These are the HTTP header values which will be sent along with every request.

Launching a New Scan

To launch a new scan we will invoke the POST /scans/new method.

How to Launch a New Scan

1. Add the following code to this method in JSON format:

{
  TargetUri: 'http://www.example.com/',
  FormAuthenticationSettingModel: {
          IsEnabled: true,
          LoginFormUrl: 'http://www.example.com/login',
          Personas: [{
        UserName: 'username',
        Password: 'password',
        IsActive: true
     }]
  }
}

• It contains:
  • The URL of the website that we have defined in Adding a Website to Invicti Enterprise
  • Login information (login form url, username and password) to the web site that we want to scan
• If your web site does not require login you can simply remove FormAuthenticationSettingModel and only send the TargetUri
• If your web site uses another authentication mechanism you can check docs whether it is supported and how
• You can also find other parameters supported by this method in the API v 1.0 Documentation.
2. Add the following lines after $headers value in .ps1 file (where $newScanUri is the Invicti Enterprise Web API endpoint address for new scan launching method and $newScanBody is the data that will be posted to invoke this method):
   • $newScanUri = $baseUri + "new"

$newScanBody = "{TargetUri:'http://www.example.com/',FormAuthenticationSettingModel:{IsEnabled:true,LoginFormUrl:'http://www.example.com/login',Personas:[{UserName:'username',Password:'password',IsActive:true}]}}"

• Do not forget to replace TargetUri, LoginFormUrl, UserName and Password values with your own values
3. Add the following lines:
   • $newScanResp = Invoke-RestMethod -Uri $newScanUri -Headers $headers -Body $newScanBody -Method POST

     $id = $newScanResp[0].Id

   • This invokes the rest method located at $newScanUri by posting $newScanBody
   • If successful, the result will be a response holding the Id value of the Scan, which will be used for the other methods invocations we are about to make
   • We store its value in a variable called $id
   • For other values that are returned with the response, see API 1.0 Documentation.Docs.
4. Finally, the completed .ps1 file looks like this:
   • $baseUri = "https://www.netsparkercloud.com/api/1.0/scans/"
$headers = @{
"Authorization"="Basic [Your Base64 Encoded USERID:API-TOKEN]"
"Content-Type"="application/json"
}

     $newScanUri = $baseUri + "new"
$newScanBody = "{TargetUri:'http://www.example.com/',FormAuthenticationSettingModel:{IsEnabled:true,LoginFormUrl:'http://www.example.com/login',Personas:[{UserName:'username',Password:'password',IsActive:true}]}}"

     $newScanResp = Invoke-RestMethod -Uri $newScanUri -Headers $headers -Body $newScanBody -Method POST
     $id = $newScanResp[0].Id

5. You can now launch a new scan by running the scan.bat file. Script execution will stop immediately after launching the scan.

Querying Scan Status

1. Add this line under $newScanUri at the top of the .ps1 file:
• $statusUri = $baseUri + "status/"

This is the base uri that is used to query the status of the scan:

• It invokes the GET /scans/status/{id} method
• This method needs be invoked periodically until the scan finishes
• Eventually, the scan will have a Complete, Failed or Cancelled status
• We will create a loop that checks the status of the scan, until it has one of these states
2. Add the following lines to the .ps1 file:
   • $statusFinalUri = $statusUri + $id

     Do
{
   Start-Sleep -s 10

        $statusResp = Invoke-RestMethod -Uri $statusFinalUri -Headers $headers -Method GET

     } While ($statusResp.State -ne "Complete" -And $statusResp.State -ne "Failed" -And $statusResp.State -ne "Cancelled")

In this code, the following applies:

• $statusFinalUri holds the full address that is required to query the scan status
• The Do-While loop executes until the scan has one of the specified states
• Start-Sleep pauses script execution for 10 seconds
• The $statusResp variable holds the response returned from the method invocation
3. Finally, the complete .ps1 file looks like this:
   • $baseUri = "https://www.netsparkercloud.com/api/1.0/scans/"
$headers = @{
  "Authorization"="Basic [Your Base64 Encoded USERID:API-TOKEN]"
"Content-Type"="application/json"
}

     $newScanUri = $baseUri + "new"
$statusUri = $baseUri + "status/"

     $newScanBody = "{TargetUri:'http://www.example.com/',FormAuthenticationSettingModel:{IsEnabled:true,LoginFormUrl:'http://www.example.com/login',Personas:[{UserName:'username',Password:'password',IsActive:true}]}}"

     $newScanResp = Invoke-RestMethod -Uri $newScanUri -Headers $headers -Body $newScanBody -Method POST
$id = $newScanResp[0].Id

     $statusFinalUri = $statusUri + $id
Do
{
Start-Sleep -s 10

     $statusResp = Invoke-RestMethod -Uri $statusFinalUri -Headers $headers -Method GET

     } While ($statusResp.State -ne "Complete" -And $statusResp.State -ne "Failed" -And $statusResp.State -ne "Cancelled")

4. You can now launch a new scan by running the scan.bat file. The script will execute as long as the scan executes.

Downloading Scan Report

1. Add this line under $statusUri at top of the .ps1 file:
   • $reportUri = $baseUri + "report?type=ScanDetail&format=Html&id="

     $reportFileName = "index.html"

This is the base uri that we will use for querying status of the Scan:

• The Full Scan Report file downloads in Html format (for further information on other support report types and formats, see API Documentation
• $reportFileName will be used as the filename when saving the downloaded report file, but if you choose to give it a different name, store it somewhere safe, because we will be using this value in TeamCity configuration
• To query scan report file we will invoke the GET /scans/report/{id} method. We should only invoke this method if scan has finished successfully. A scan having “Complete” state means scan has finished successfully. So we need to check if $statusResp.State has this value.
2. Add this code underneath the Do-While loop in the .ps1 file:
   • if ($statusResp.State -eq "Complete")
{
$reportFinalUri = $reportUri + $id

     $html = Invoke-WebRequest -Uri $reportFinalUri -Headers $headers -Method GET -UseBasicParsing

     "$html" | Out-File $reportFileName
}

   • Here, we first check $statusResp.State if it has value of 'Complete' which indicates that scan has completed successfully.
   • $reportFinalUri holds the full address that is required to download scan report file.
   • After invoking method $html holds the HTML content of the Scan report. In other words this variable holds the downloaded data that should be written onto disk.
   • Finally we write HTML data into the index.html file using Out-File.
3. Now the completed .ps1 file looks like this:

   • $baseUri = "https://www.netsparkercloud.com/api/1.0/scans/" 
     $headers = @{
      "Authorization"="Basic [Your Base64 Encoded USERID:API-TOKEN]"
      "Content-Type"="application/json"
     }
      
     $newScanUri = $baseUri + "new"
     $statusUri = $baseUri + "status/"
     $reportUri = $baseUri + "report?type=ScanDetail&format=Html&id="
     
     $reportFileName = "index.html"
     
     $newScanBody = "{TargetUri:'http://www.example.com/',FormAuthenticationSettingModel:{IsEnabled:true,LoginFormUrl:'http://www.example.com/login',Personas:[{UserName:'username',Password:'password',IsActive:true}]}}"
      
     $newScanResp = Invoke-RestMethod -Uri $newScanUri -Headers $headers -Body $newScanBody -Method POST
     $id = $newScanResp[0].Id
     
     $statusFinalUri = $statusUri + $id
     Do
     {
      Start-Sleep -s 10
     $statusResp = Invoke-RestMethod -Uri $statusFinalUri -Headers $headers -Method GET
     } While ($statusResp.State -ne "Complete" -And $statusResp.State -ne "Failed" -And $statusResp.State -ne "Cancelled")
     if ($statusResp.State -eq "Complete")
     {
      $reportFinalUri = $reportUri + $id
      
      $html = Invoke-WebRequest -Uri $reportFinalUri -Headers $headers -Method GET -UseBasicParsing
      
      "$html" | Out-File $reportFileName
     }

4. You can now launch a new scan by running the scan.bat file. The script will execute as long as the Scan executes. and after executed is completed you should have the scan report file named as “index.html” next to .ps1 file.

Final PowerShell Script

Up to this point, for the sake of simplicity, we have focused on the task. However, when things go wrong, we need to be ready.

In the final script, we added some exception handling, logging and exit codes. At this point, the only thing you need to know about this extra code is that the messages that are written using the Write-host method, which you can view in the Build Logs tab in TeamCity. Others are simply for handling error cases.

The final script looks like this:

$baseUri = "https://www.netsparkercloud.com/api/1.0/scans/"
$headers = @{
 "Authorization"="Basic [Your Base64 Encoded USERID:API-TOKEN]"
 "Content-Type"="application/json"
}
  
$newScanUri = $baseUri + "new"
$statusUri = $baseUri + "status/"
$reportUri = $baseUri + "report?type=ScanDetail&format=Html&id="

$reportFileName = "index.html"

$newScanBody = "{TargetUri:'http://www.example.com/',FormAuthenticationSettingModel:{IsEnabled:true,LoginFormUrl:'http://www.example.com/login',Personas:[{UserName:'username',Password:'password',IsActive:true}]}}"

try
{
  $newScanResp = Invoke-RestMethod -Uri $newScanUri -Headers $headers -Body $newScanBody -Method POST

  $id = $newScanResp[0].Id

  $lastState = ""

  $statusFinalUri = $statusUri + $id

  Do
{
  Start-Sleep -s 10

  $statusResp = Invoke-RestMethod -Uri $statusFinalUri -Headers $headers -Method GET

  if ($statusResp.State -ne $lastState)
  {
  Write-Host $statusResp.State
  $lastState = $statusResp.State
  }
  } While ($statusResp.State -ne "Complete" -And $statusResp.State -ne "Failed" -And $statusResp.State -ne "Cancelled")

  if ($statusResp.State -eq "Complete")
  {
  $reportFinalUri = $reportUri + $id

  $html = Invoke-WebRequest -Uri $reportFinalUri -Headers $headers -Method GET -UseBasicParsing

  "$html" | Out-File $reportFileName

  exit 0
  }
  else
  {
  exit -1
  }
}
catch
{
  $errResponse = $_.Exception.Response    

  if (!$errResponse)
  {
  Write-host $_.Exception.Message
  exit -1
  }

  $errRespStream = $errResponse.GetResponseStream()
  $errRespStream.Position = 0

  $reader = New-Object System.IO.StreamReader($errRespStream)
  $responseBody = $reader.ReadToEnd();

  Write-host $errResponse.StatusCode
  Write-host $errResponse.StatusDescription    
  Write-host $responseBody

  exit $errResponse.StatusCode
}

TeamCity Configuration

Since you need a Build Configuration that will execute the PowerShell script each night to scan your website and then display the Scan Report in a Report Tab inside TeamCity, you need to configure the following options in TeamCity.

This tutorial assumes that you are already familiar with TeamCity and have some Projects on it. If you do not have any Projects in TeamCity, first read Jetbrains' Creating and Editing Projects documentation.

• First, you will need to get a TeamCity account with Administrative privileges. Confirm whether you have Administrative privileges by logging in to TeamCity and checking that you can see the Administration link on the top right of the screen. If it is not there, you do not have Administrative rights. Find the TeamCity Admin and request them.
• After logging in, click Administration, then select the Project in which you want to create your Build Configuration.

Adding a Build Report Tab

First we need to create a Report Tab, a Project-wide setting in TeamCity. This means if you add a Report Tab to a Project, any Build Configuration under this Project can benefit (not necessarily) from it.

How to Add a Build Report Tab

1. In TeamCity, in the left hand menu, under Project Settings, click Report Tabs.

2. Click + Create new build report tab. The Edit Report Tab Settings dialog is displayed.

3. In the Tab Title field, enter 'Invicti Enterprise'. This is the name of the tab that will be displayed in the Build Detail page (similar to other tabs such as Overview, Build Logs and Parameters).
4. In the Start page field, enter the name of the relative path from the root of the build's artifacts to a start page of the generated report. The default value: index.html. If you used this value for $reportFileName in Downloading Scan Report, you can leave it as it is. This value must have the same value as $reportFileName.
5. Click Save.

Creating the Build Configuration

Creating the Build Configuration is what does the actual job.

How to Create a New Build Configuration

1. In TeamCity, in the left hand menu, under Project Settings, click General Settings.

2. From the Build Configurations section, click + Create build configuration. The Create Build Configuration window is displayed.

3. The Parent project drop-down list is already set your project, NS Enterprise.
4. In the Name field, enter a name for the new build configuration.
5. As you type, Build configuration ID is automatically generated.
6. Click Create to create additional build configurations.

After creating the Build Configuration, TeamCity will redirect you to the Version Control Settings window.

General Settings

1. In TeamCity, in the left hand menu, click General Settings.

2. If the advanced options are hidden, click Show advanced options.
3. Some fields are already populated.
4. In the Artifact paths field, enter 'index.html' (or the value you used for $reportFileName in Downloading Scan Report).
5. Leave all other default options as they are.
6. Click Save.

Triggering the Build Configuration

Now we want to configure when the Build Configuration will run. To keep it simple, these instructions are for a scheduled trigger, which will run the configuration nightly. You can easily configure this to run after a git commit.

How to Configure the Build Configuration

1. In TeamCity, in the left hand menu, click Triggers.
2. Click + Add new trigger, then click Schedule Trigger. The Schedule Trigger dialog is displayed.

3. If the advanced options are hidden, click Show advanced options.
4. Change the Date and Time options, including time zone, to your required settings.
5. Leave all other default options as they are.
6. Click Save.

Creating a Build Step (PowerShell)

The final step is to give the PowerShell script, that we have prepared in Invoking Invicti Enterprise Web API Using PowerShell, to TeamCity.

How to Create a Build Step

1. In TeamCity, in the left hand menu, under Build Configuration Settings, click Build Steps, then click + Add Build Step. The New Build Step window is displayed.

2. From the Runner type drop-down, click Powershell.
3. If the advanced options are hidden, click Show advanced options.
4. In the Step name field, enter a name.
5. From the Powershell run mode drop-downs, Version and Bitness, click the options that match the version of PowerShell installed on your TeamCity server.
6. From the Script drop-down, click Source code.
7. In the Script source field, paste the contents of the .ps1 file.
8. From the Script execution mode drop-down, click Execute .ps1 from external file.
9. Click Save.

Running the Build Configuration and Viewing the Scan Report in TeamCity

Now we have everything configured and ready to see some results. Our build configuration is scheduled run every night but we can also run it manually any time we want.

How to Run the Build Configuration

1. In TeamCity, in the left hand menu, click Build Configuration Settings.
2. Click Run.
3. From the Projects drop-down, select your project.
4. From the Overview tab, click on your newly-created Build Configuration.
5. In the Recent history section, you can view completed and running configurations.
6. Click on the completed execution. The Build details window is displayed, showing both . where you can see both the Build Log and Invicti Enterprise tabs.

7. Click the Build Log tab to view the progress of the build. After a completed Build Configuration execution (failed or successful) you can see the log messages that are written by the Write-host command in the PowerShell script.
8. Click the Invicti Enterprise tab (which will be visible if the PowerShell script execution completed successfully, the Report file is saved as index.html successfully and the Build Configuration has been successfully executed) you can view the Full Scan Report.

Incremental Scan

First, ensure that you have completed the Preliminary Actions in Invicti Enterprise.

At the end of this tutorial, you will have a TeamCity Build Step, which will be executed after a git push, and will launch a scan by invoking the Invicti Enterprise Web API. The purpose of this is to find any vulnerabilities caused by a new git push, before they are reflected to the production environment.

Invoking Invicti Enterprise Web API Using PowerShell

Invicti Enterprise exposes a Web API that allows client applications to view and manipulate scan tasks.

To invoke the API, we use PowerShell, because TeamCity is able to execute PowerShell scripts as a Build Step.

For further information about the Invicti Enterprise Web API, see API v 1.0 Documentation.

How to Invoke Invicti Enterprise Web API Using PowerShell

1. First create a file called scan.ps1. Add the following script to the file:
• $scanUri = "https://www.netsparkercloud.com/api/1.0/scans/incremental"

  $headers = @{
"Authorization"="Basic [AuthToken]"
"Content-Type"="application/json"
}

  $requestBody = "{BaseScanId: '[ScanId]'}"

  try
{
Invoke-RestMethod -Uri $scanUri -Headers $headers -Body $requestBody -Method POST
Write-hot
exit 0
}
catch
{
$errResponse = $_.Exception.Response

  if (!$errResponse)
{
Write-host $_.Exception.Message
exit -1
}

  $errRespStream = $errResponse.GetResponseStream()
$errRespStream.Position = 0

  $reader = New-Object
System.IO.StreamReader($errRespStream)
$responseBody = $reader.ReadToEnd();

  Write-host $errResponse.StatusCode
Write-host $errResponse.StatusDescription
Write-host $responseBody

  exit $errResponse.StatusCode
}

• Replace '[AuthToken]' with your Base64 Encoded USERID:API-TOKEN (How to Access Your API Credentials).
• Replace '[ScanId]' with the ScanId value, including the brackets (see How to Find the Scan Id of a Report).
2. This script invokes the /scans/incremental method, which launches a new Incremental Scan using the Invicti Enterprise Web API. It does so by sending the base scan's ScanId to the Incremental Scan. The code inside the catch clause is intended for exception handling and will help us if we have any issues about triggering the scan. Messages written using the Write-host command will be visible in the Build Logs tab of the Build Configuration window.
3. Open inside the standard command window (cmd.exe), run the script with the command to launch an Incremental Scan:
   • powershell .\scan.ps1 -ExecutionPolicy ByPass

TeamCity Configuration

Since, you need a Build Configuration with a PowerShell Build step that will trigger an Incremental Scan inside TeamCity using Invicti Enterprise Web API, you need to configure the following options in TeamCity.

This tutorial assumes that you are already familiar with TeamCity and have a Build Configuration that runs following a git push. If you do not have any Projects or Build Configurations in TeamCity, first read Jetbrains' Creating and Editing Projects documentation.

• First, you will need to get a TeamCity account with Administrative privileges. Confirm whether you have Administrative privileges by logging in to TeamCity and checking that you can see the Administration link on the top right of the screen. If it is not there, you do not have Administrative rights. Find the TeamCity Admin and request them.
• After logging in, click Administration, then select the Project and Build Configuration in which you want to add the PowerShell Build Step. The Build Configuration Settings window is displayed.

How To Add a Build Step

1. In TeamCity, in the left hand menu, under Build Configuration Settings, click Build Steps.
2. Click + Add build step. The New Build Step window is displayed.

3. From the Runner type drop-down, click Powershell.
4. If the advanced options are hidden, click Show advanced options.
5. In the Step name field, enter a name for the step.
6. From the Powershell run mode drop-downs, Version and Bitness, click the options that match the version of PowerShell installed on your TeamCity server.
7. From the Script drop-down, click Source code.
8. In the Script source field, paste the contents of the scan.ps1 file.
9. From the Script execution mode drop-down, click Execute .ps1 from external file.
10. Click Save.

Now that you have successfully added the PowerShell script to TeamCity, it will run following the completion of the Build Steps, and trigger an Incremental Scan on your website to see if the new build caused any vulnerabilities.