How to Measure Time to Value in Web Application Security

Ferruh Mavituna on Enterprise Security Weekly #178

First Things First: What Are You Trying to Measure?

Time to value is a big deal, especially in large organizations. But before you can measure improvements and determine if you’re getting value from your process, you need a baseline to compare against. To determine if your applications are becoming more secure, you need to know what those applications are, what their current security status is, what you’re doing to improve security, and how effective your efforts are. Simply put, you need to find all your vulnerabilities and start fixing them – and only then can you start measuring if and how quickly your security is improving.

Choosing Your Starting Point

A common approach in organizations with hundreds of existing websites and web applications is to identify and secure just the most critical ones using some kind of a manual process. That way, a handful of vital assets are secured while all the others remain potentially vulnerable (or even undiscovered). If you try to apply the same process across hundreds of assets, you are going to need many months just to get the first results. In most cases, this is not a practical solution.

For new applications, one approach is SAST (Static Application Security Testing), where security testing is done on application source code before it goes into production. However, this is only practical for new environments with complete control of the development and deployment pipeline. In real life, most organizations already have a large web footprint and need to secure dozens of existing web assets with a small team and often a limited budget. This is where DAST (Dynamic Application Security Testing) comes in. You can use an automated scanning solution like Invicti to help your team manage a large number of assets and issues.

Discovery is also vital for ensuring that you know your actual environment and web attack surface. In many cases, customers scan one website and discover that it connects to 50 web services and microservices. This is where Invicti’s discovery feature is invaluable, helping organizations find all the assets they need to secure.

You Have the Scan Results – Now What?

So you go for a DAST tool, run your scans, and you have your results. This is the first step to knowing what you need to fix, but to make a direct impact, these results need to be actionable. Many other DAST solutions leave customers with a list of potential vulnerabilities that must be checked by the security team before developers can fix them. With Invicti, you get the unique Proof-Based Scanning technology that shows you which results are 100% real and proven vulnerabilities, complete with initial priority assignment. This is a game-changer because now you know what to work on next.

Next comes the challenge of actually fixing the vulnerabilities. When we’re talking about many hundreds of web assets and potentially thousands of vulnerabilities, direct integration with developer tools is the only realistic way to go. This lets you bypass the performance bottleneck introduced by manual verification of every single issue by the security team. Invicti has very mature integration capabilities – it integrates with bug trackers and other workflow systems, so automatically proven vulnerabilities can go straight into developer tickets as issues to fix.

Instant Value with Automation and Integration

This is where we get back to time to value in web application security. With Invicti, you start with accurate scanning and discovery, identify and automatically prove vulnerabilities, and send developers accurate and actionable bug reports through their ticketing system complete with enough information to implement the fix. Then you automatically re-test the fixes to make sure the vulnerabilities are gone and that’s it: value.

The result? Immediate and measurable security improvements within days of starting the whole process rather than months of tedious work. Compared to manual processes, your time to value is so short it’s unbelievable. Our customers confirm that implementing our approach completely changes their perception of what is possible in web application security. Instead of spending many months chasing false positives and wasting time on constant back-and-forth exchanges between the security team and developers, they can cut away the inefficiencies and achieve real results fast.

Measuring Time to Improvement in Web Application Security

Talking to our customers, we see two main web application security metrics being used to measure time to value. The first is the classic time to fix (TTF) – simply measuring time from the initial vulnerability report to the final fix. This shows how agile the whole process is. Obviously, same-day is a better result for a critical vulnerability than 10 days or more.

Another popular metric compares vulnerabilities introduced to vulnerabilities fixed. This provides a long-term view of the overall trend: is your web application security improving or getting worse? With grouping reports, you can also break this down to compare trends across different teams or projects and identify areas than need to improve.

And on a final note, remember that before you can think about measuring value from vulnerability management, you need to know what you’re measuring. Even the best metrics won't help if they only cover a handful of known websites, so web asset discovery is a crucial first step. Get your asset management right first and only then can you get value from vulnerability management.