Scanning a RESTful API Web Service

This document is for:

Invicti Standard, Invicti Enterprise On-Premises, Invicti Enterprise On-Demand

REST (Representational State Transfer) is an architectural style that can be used to communicate with web services. REST has a lot in common with protocols such as SOAP. It is used as a communication mechanism between two applications, or between an application and an online service. Many mobile web applications communicate with a RESTful API at the backend in order to communicate with the online service.

You can scan a RESTful API Web Service in both Invicti Enterprise and Invicti Standard editions.

For further information, please see How to Ensure REST API Security and Invicti API Documentation.

Differences Between a Web Service and a REST API

Many web services rely on complex communication mechanisms such as SOAP, RPC and CORBA. REST uses the standard HTTP methods for all four CRUD (Create, Read, Update, Delete) operations.

Commonly Used HTTP Methods (Verbs) in REST API

This table lists the commonly used HTTP methods in REST.

Method	Description
POST	Create a resource
GET	Retrieve a resource
PUT	Change the state of a resource or update it
DELETE	Remove or delete a resource

The Challenges of Scanning REST API Interfaces

Unlike RPC and others, REST can be easily consumed and understood by users because of its simple structure. For example, many REST-based web services can provide a response in JSON or XML format. But this same benefit is what makes it very difficult for an automated web vulnerability scanner to crawl and attack.

Lack of Standards for REST

There is no consistent standard for REST API, as there is for WSDL and other similar protocols. Most RESTful web services have their own documentation, useful for developers but useless to automated web vulnerability scanners.

A number of projects aim to standardize the REST API:

Using Parameters in URLs

Another challenge automated scanners encounter when scanning RESTful web services for vulnerabilities is that REST APIs use parameters in URLs.

For example in the HTTP GET request below, 123 is a parameter and not a directory in the web application:

GET http://www.example.com/rest-api/products/123/

Invicti deals with this using heuristic URL Rewrite technology that can automatically identify and scan parameters in URLs. In a REST API things work a little differently.

Scanning a RESTful API Web Service for Vulnerabilities

There are three ways to scan a RESTful API. Each is outlined below:

Importing the Definition Files Manually

When you import an OpenAPI (formerly Swagger), WADL or WordPress REST API definition file, the Invicti web application security scanner will parse the definition file and create a link for every resource available in the API.

There are 2 methods to manually import the definition file: From Fileand From URL.

The From File option lets you import your document to Invicti . This requires you to import the file over and over again whenever you update your web service.
The From URL option lets you provide a link for the definition file, so you do not need to import it again to Invicti whenever you update your web service. For further information, see Importing links and API definitions.

Method 1: Importing the definition file from the file to Invicti

How to import the OpenAPI (formerly Swagger), WADL, or WordPress definition files from the file in Invicti Enterprise

Log in to Invicti Enterprise.
From the main menu, select Scans > New Scan.
From the Scan Settings section, select Links/API Definitions.

From the From File section, select Web Application Description Language (WADL).
From the opened window, select the schema file. Then, select Open.
Once the scanner imports all the schema you can see them in the list of Imported Links as seen in the screenshot.

Select Launch to start the scan.

During the import, URL Rewrite Rules will automatically be generated, so that every parameter in the RESTful API is scanned.

When importing a RESTful web service definition file in Invicti Enterprise, the URL Rewrite rules are not shown in the Start a New Scan page, but they will be reported in the Knowledge Base node, either once the scan is finished or during the scan.

How to import the OpenAPI (formerly Swagger), WADL, or WordPress definition files from the file in Invicti Standard

Open Invicti Standard.
From the ribbon, select New.
From the Start a New Website or Web Service Scan dialog, select Links/API Definitions, then double-click Web Application Description Language (WADL).

From the Import Links window, select the schema file. Then, select Open.
Once the scanner imports all the schema you can see them in the list of Imported Links as seen in the screenshot.

Select Start Scan.

Method 2: Importing the definition file from the URL to Invicti

How to import the OpenAPI (formerly Swagger), WADL, or WordPress definition files from the URL in Invicti Enterprise

Log in to Invicti Enterprise.
From the main menu, select Scans > New Scan.
From the Scan Settings section, select Links/API Definitions.
From the From URL section, select Web Application Description Language (WADL).
From the Add an URL dialog, enter the URL.

Select OK to import the definition file from the URL to Invicti.
Select Launch to start the scan.

When importing a RESTful web service definition file in Invicti Enterprise the URL Rewrite rules are not shown in the Start a New Scan dialogue, but they will be reported in the Knowledge Base Nodes once the scan is finished.

How to import the OpenAPI (formerly Swagger), WADL, or WordPress definition files from the URL in Invicti Standard

Open Invicti Standard.
From the ribbon, select New.
From the Start a New Website or Web Service Scan dialog, select Links/API Definitions, then double-click Web Application Description Language (WADL).
On the Web Application Description Language (WADL) dialog, enter an URL.

Select OK to import the definition file from the URL to Invicti.
Select Start Scan.

Automating the Discovery of RESTful APIs During Crawling

Similar to when scanning other web applications and services, authentication can be configured from the Authentication tab.

How to Automate the Discovery of the RESTful API During Crawling

The Invicti web application security scanner will automatically import, crawl and scan a REST API web service, if it is identified during a scan. Once the scanner identifies the definition file, it will automatically generate the URL Rewrite rules so it can scan all the parameters in the web service.

When the scanner identifies a RESTful API web service during a crawl it will also report it in the Knowledge Base node. This is what the REST APIs node looks like in the Knowledge Base section of the Technical Report in Invicti Enterprise.

This is what REST APIs look like in the Scan Summary Dashboard in Invicti Standard.

Should the scanner identify references to a RESTful API web service during the Crawling stage (via JavaScript or other means) but then fail to identify the definition file, once it has the necessary samples it will still try to heuristically create the URL Rewrite Rules to scan all the parameters.

Importing RAW HTTP Requests Manually

In cases where the OpenAPI (formerly Swagger) or WADL definition files are not available, or the RESTful API cannot be identified during the crawl of a web application, you can import the API’s links via RAW HTTP files.

You can capture the HTTP requests via a third party proxy tool such as Fiddler and import them before starting the scan. Here is a list of supported proxy files:

Burp Saved Items (*.xml)
Fiddler session archives (*.saz)
HTTP archive files (*.har)
Paros log files (*.txt)

Note that when using this method the scanner will not automatically generate the URL rewrite rules. Configure the URL Rewrite Rules manually.

Invicti supports Burp’s XML format.

How to Import RAW HTTP Requests Manually

The procedure for importing these tools manually is the same as that for importing SWAGGER or WALD (see Importing the Definition Files Manually).