Why web asset discovery is a crucial part of your AppSec program

Tolga Kayaş on Enterprise Security Weekly #242

Appearing on episode 242 of the Enterprise Security Weekly cybersecurity podcast, Tolga Kayaş from Invicti’s Application Security Management team talked to Adrian Sanabria and Paul Asadoorian about the challenges and best practices around web asset discovery. Watch the full interview below and read on to learn why discovery should be a permanent part of any web application security program.

Attackers start by checking what you have – and so should you

Amazing though it may seem in this day and age, many organizations still don’t fully know what websites and applications they are exposing to the public Internet. Large enterprises routinely add hundreds of web assets a year, from business applications and marketing campaign sites to knowledge portals and web interfaces for desktop software. These can include cloud-native solutions, single-page applications (SPAs), static sites – but regardless of the technology stack, deployment model, and intended useful life, they all increase the attack surface.

Any serious attacker starts by performing careful reconnaissance using all available data sources to find promising targets and plan further action. To protect your systems from attack, you need to have at least the same level of information about your web assets. Simply put, you can’t protect what you don’t know about. While there are many approaches to application security testing, knowing what you need to test is a fundamental requirement before the first test even runs.

The risks of an incomplete asset inventory

Especially in heavily virtualized environments, it only takes a few clicks to spin up a new website or set up a new repository. You might even get multiple deployments hosted on the same web server with no obvious way of telling what is where. This is the development side of self-service shadow IT – the bane of IT admins the world over.

Without centralized control, large organizations can end up with thousands of assets dotted all over their web environments, many long-forgotten by their creators and left to their fate. These abandoned and unmaintained sites make prime targets for malicious actors, as they will often yield to old vulnerabilities that would be fixed in newer deployments and can be probed much more freely than production assets. Just as importantly, attackers are constantly honing their skills and techniques, so a site that was secure two years ago may now be wide open to a new attack.

The best-practice approach to minimizing this risk is to maintain a tightly controlled web asset inventory, complete with policies and processes to onboard, securely maintain, and offboard assets. In a perfect world, this should mean that deploying any public-facing website or application requires explicit approval. Once approved, the asset is added to a central inventory where it is covered by regular security testing. When it is no longer needed, it is removed both from the inventory and the Internet.

In the real world, few organizations are likely to have a waterproof asset inventory, so asset discovery is needed to find the assets that slipped under the radar. For those only now creating their central inventory, the first step will be finding everything they already have to establish a baseline. And for those without any central asset management, running discovery is the only way to determine what they have and what they need to secure.

Approaches to web asset discovery

Finding publicly accessible web assets is not a trivial task. The main two approaches are internal discovery, where you have access to file systems and configuration data, and external discovery, where you probe systems from the outside and query external data sources much like an attacker would. Each has its strengths, but ideally, both should be combined for the best results.

Internal discovery will often start by checking configuration data for your environments and security appliances. In the cloud era, many internal assets are not physically on-premises – they are still in the cloud, just not publicly available. If you have central control of your entire environment, you can automatically analyze configurations to discover many assets. For on-prem environments, you need to start by identifying security appliances and extracting asset information from their configurations. You can then go deeper and use standard network scans and similar methods to detect active IPs, ports, and HTTP services for investigation.

In contrast, external asset discovery methods are non-invasive and have the benefit of directly showing how much information about your systems is available to attackers. You can query public datasets and crawlers for your known mission-critical assets, certificate data, and DNS records. Internet search engines, both generic and specialized, are also a valuable source of open-source intelligence (OSInt) about your web assets. Finally, you can use third-party providers to run discovery for you – and this is where Invicti can make your life much easier.

Automated and systematic discovery with Invicti

Invicti features built-in systematic web application discovery to help you determine which sites and applications you need to scan for vulnerabilities. As soon as you create your user account and enter your company email, the discovery process starts automatically based on the company domain. You can further seed it with additional domains and web properties that you own, organization names from your SSL certificates, and known public IP address ranges.

You get your first results in minutes and the discovery service automatically repeats this search on a daily basis, even if you are not currently using the product or running scans. That way, you are automatically notified about any new assets that appear in your environment so you can add them to your inventory and decide if you want to scan them for vulnerabilities. This automated discovery service is included as standard with Invicti, but additional manual discovery services are also available separately as part of white-glove application security management.

Invicti’s guided success offering includes automated and manual discovery alongside security testing and management services to help customers get off to the best possible start with Invicti. Security is only as good as its weakest link, so taking this systematic guided approach can be far more effective than hoping that any gaps in discovery will be exposed through penetration testing or a bug bounty program. While it’s true that asset discovery will always be the first stage of any penetration test, you will get far more value by doing as much as possible internally and letting the external experts focus on more advanced cases.

Building discovery into a comprehensive AppSec program

Any web application security program worth its salt should start with asset discovery as a prerequisite for security testing. Without accurate discovery data, you don’t know your realistic attack surface, you can’t effectively prioritize testing and remediation, and you can’t even be sure how to define the scope of your penetration testing or bug bounty program. Invicti makes discovery easier by automating the entire process and seamlessly embedding it into the product. 

Invicti users get the benefits of automated external discovery straight out of the box, even before setting up and running the first vulnerability scan. When you combine this with guided success services to optimize the automatic process, add manual discovery, and customize vulnerability scanning for your unique environment, you can quickly get a complete and accurate picture of your overall web security posture – and immediately start improving it.