Why penetration testing tools don’t work as enterprise scanners

The need to scan web applications for vulnerabilities is now widely accepted, moving the focus from “do we need this” to “how do we do it.” Yet with security tool vendors all making superficially similar claims and using the same acronyms, there can be confusion around choosing the right product for the job. One common mismatch is taking a vulnerability scanner designed for manual penetration testing and trying to use it on an enterprise scale and with enterprise workflows. This can end in tears – and one reason it happens is tool bias.

How tool bias affects vulnerability scanner choice

All professionals have their specialized go-to tools that they know inside out and are happy to recommend if asked. Application security testing is no different, so if you ask a penetration tester about a good vulnerability scanner, they’re likely to recommend whatever they know and use for their manual testing. And while this could be an excellent product for penetration testing, it will likely fall short on multiple counts if you try to use it at scale as an enterprise scanner, if nothing else because it’s not designed to work in fully automated workflows.

Factors like familiarity and availability may also artificially narrow down the tool and vendor shortlist, with organizations more likely to go with what they know or have than to investigate what would work best. This could mean settling for a rudimentary scanner bundled with another security product or assuming that just because a vendor has a good pentesting scanner, their enterprise offering will automatically be just as effective. As with many things, convenience and upfront price can override more practical considerations.

Taking the upfront cost argument a step further, the widespread reliance on open-source or otherwise free tools in the ethical hacking community may lead to advice that you don’t need any commercial tools to scan for vulnerabilities. While this can be true for manual penetration testing, applying the same toolchain to vulnerability scanning in an enterprise setting will result in vast amounts of extra work to get security improvements that are modest at best. In a worst-case scenario, using a free scanner at an enterprise scale may generate significant costs due to the extra overhead of verifying and triaging findings, creating tickets, and communicating across teams without an efficient process in place.

Whatever the source of bias, penetration testing and enterprise-grade web scanning are two different use cases that grow even further apart as you scale up the number of scans, scan targets, and people involved in testing and remediation. To take just one difference as an example, the results from a pentesting scanner are intended for a security professional who has the skills and experience to weed out false alarms, identify the most likely issues, and manually dig deeper for the root cause. For an enterprise scanner, vulnerability reports might go directly to developers who don’t have the time or security skills to investigate and verify issues. Instead, they need precise technical information and guidance on fixing the core flaw.

Enterprise DAST must-haves

For automated use in enterprise scenarios, we now generally talk about dynamic application security testing (DAST) solutions rather than vulnerability scanners, and that distinction goes far beyond hitting the right acronyms. An accurate scanner is only the foundation for an enterprise-grade DAST to build all the management, scalability, and automation features required to operate in automated development workflows. Several capabilities of a DAST solution make all the difference in an enterprise setting, as illustrated by Invicti Enterprise:

• Accuracy good enough for automation: When a vulnerability report leads to an automatic developer ticket, false positives are a deal-breaker. Invicti handles this using proof-based scanning to automatically confirm the majority of serious vulnerabilities by safely exploiting them. Because exploitable flaws are definitely not false positives, they can go directly into bug tickets in the issue tracker.
• Integration into existing development workflows: Development organizations live and breathe issue trackers, so any security reports consumed by developers must go into these systems. Emailing vulnerability reports as PDFs or sending them as individual messages is a recipe for inefficiency and internal friction between teams.
• Immediately useful remediation guidance: Developers should focus on building innovative software, not clarifying vulnerability reports or pushing back on false alarms, so each security ticket should include complete practical information to fully fix the issue and prevent it from resurfacing.
• Scalability to scan lots of assets, often: Unlike the single scan performed to kick off a pentest or vulnerability assessment, scans in enterprise application environments can run into dozens if not hundreds a day, from scheduled full scans to single-page retests and everything in between.
• Reporting and visibility across environments: Each scan in an enterprise DAST is just one small part of a broader picture. To make sense of the thousands of vulnerability reports you could have in the system at any one time requires reporting and management features to keep track of the overall security posture, identify problem spots, monitor long-term trends, and plan future strategy.

Different tools for different purposes

To be clear, this is not about knocking any established pentesting tools – it’s about choosing the right tool for the job. For a penetration tester, a vulnerability scanner is expected to provide good starting points for manually investigating promising results within the scope of a single test. For dev teams, vulnerability reports from the company DAST are expected to show what security flaws need fixing – all while working at scale, automatically, and without slowing down the pace of development.

It’s also not an either-or proposition. Building DAST into your application security program means you can quickly and efficiently find and fix the majority of typical security vulnerabilities in-house as part of routine development and testing. When the penetration testers or bounty hunters step in, they can then look for more advanced issues and business logic vulnerabilities without wasting your time and money on the simpler stuff. This leaves you with more secure applications and also better value from manual testing – a win-win.