Troubleshooting Inconsistent Web Security Scan Results
Invicti is a black box web vulnerability and security scanner. It works by emulating an actual attacker, sending a number of HTTP requests to a target website and waiting for the responses from the web application in order to identify any vulnerabilities or other security issues.
Invicti can send a large number of requests within minutes. Since it is all automated, sometimes there are factors that could affect the results of the scan, which also means two scans of the same website can have different results. Here is a list of things that can have an impact on your scan results. The next time you encounter inconsistent scan results, you can use this list to troubleshoot the problem:
- Server-side problems
- Connection problems
Server-side Problems
This table lists and explains the potential server-side problems, which may block the scanner’s requests or result in the web application not responding to the scanner’s requests. If the scanner does not get a response for any of its HTTP requests, it cannot determine if there is a vulnerability on the target web application, resulting in different scan results.
| Problem | Description |
| Server load problem | If the server is out of resources, the web application might not be responding to all the HTTP requests from the scanner, which will generate timeouts. This can happen because Invicti affects the web applications in different ways, so you might notice an unusual CPU load on the application server, database server, and other components. |
| HTTP Error 500 / Internal Server Errors | These problems occur randomly due to a session state problem, database server load or current CPU load, for example. |
| Session Killed | The session gets killed during the scan because of application memory recycle, server restart or error recovery features in the web servers, and the scanner can’t recreate the session. |
| Different caching algorithms | Depending on the caching system, like a reverse proxy, used on the web application, the same request might get different responses each time. |
| Web Application Firewalls, IPS and other similar protectıon mechanisms blocking the scanner’s HTTP requests | If there is a mechanism blocking Invicti’s HTTP requests, such as firewalls, IPSes, or a load-balancer, Invicti won’t be able to access the web application properly. These systems might prevent some specific attack payloads and headers Invicti uses, or limit the number of requests. In order to overcome this issue, you should allowlist your IP, or create a firewall rule to allow Invicti’s custom header, “X-Scanner: Invicti”. |
Connection Problems
This section lists the potential connection problems.
| Problem | Description |
| Reverse proxy or proxy connection failures | Sometimes, the target can be deployed on a local or private server that requires a proxy connection. Such servers don’t allow to reach out them without providing a proxy connection. Likewise, the machine that Invicti is installed in might need a proxy to connect to the internet. |
| Temporary internet connection failures | When Invicti is not able to access the website, the problem may arise from a temporary internet connection failure. The machine Invicti installed or the target server might be affected from these connection corruptions. |
| HTTP Timeouts (due to server-side or communication related problems) | If the target server has issues related to server performance, it might respond very slowly. If its capabilities are not enough for an automated tool, then probably it will no longer respond. |
| Load balancers | These will almost definitely cause problems and different scan results.
Ideally in such a setup, you should scan the website directly, by bypassing the load balancer. |
How Can You Improve the Scan Results?
If you are getting inconsistent scan results, you can try the following suggestions.
| Suggestion | Description |
| Decrease the Scan Speed | This is done by reducing the number of concurrent connections the scanner opens with the web application.
To decrease the scan speed open the Scan Policy Editor, navigate to the HTTP options and adjust the Concurrent Connections slider. You can also adjust the scan speed during the scan. |
| Monitoring the application and database server | This is done by making sure they are not under a heavy load.
If you notice a spike in resources, try to determine which pages the scanner is scanning, which might also help you solve an underlying problem. |
| Ensure that there is minimum interference between the Invicti scanner and target website. | Network components such as proxies, web application firewalls, network firewalls, intrusion prevention or detection systems can all slow down the connection and drop or block requests. |
Diagnosing the Problem
Here are some tips to help you troubleshoot the issue and solve it.
Check the HTTP Request and Response for the vulnerabilities that were identified in only one of the scans:
- Do all the responses look as expected?
- Do you always get the same response when you send the same request multiple times? You can test this by exporting the request to the HTTP Request Builder tool.
- Was the vulnerable link, file or parameter identified in the scan in which the vulnerability was not reported? You can check this by reviewing the Sitemap and Crawled URLs List (CSV) Report in both scans.
- Check the sitemap, which is populated during the crawling stage, and shows users all crawled pages and parameters. If something is listed in the Sitemap, Invicti will attack it. This ensures that all pages and parameters are covered successfully.
- Watch all requests by Viewing the HTTP Requests and Response of an Issue if you need to get a detailed view of the scan. If you notice that Invicti has missed a page or parameter, you can use the proxy feature to show these missing resources or import these requests.
If you are unable to resolve the issue, submit a ticket through out Help Center.