Improving web application security testing with the Pentagrid Scan Controller

We're back with another helpful Portswigger Burp Pro Proxy extension that you shouldn't miss if you do web application security analysis. This time the wasteful approach of Burp's feature "Actively scan all in-scope traffic" triggered the development of a new extension called Pentagrid Scan Controller, because there are several improvements possible and desirable.

For example, Burp's default scanner actively scans static resources on the server, which seems useless in many cases, so the extension will not do that by default. But the power of the extension lies in all the configuration, you can do yourself to improve the automatic active scanning by deciding what should be scanned and what shouldn't. It also takes the entire concept one level higher up, because you define what you think are interesting HTTP requests and what not. This allows the extension to show you a ranked view of requests and as a tester you now know where to spend the rest of your manual testing time after Burp's scanner ran. Another improvement is in the area of repeatability of requests. The extension will figure out if a requests is repeatable at all (because that's necessary to do proper scanning). If a request is not repeatable it will be made repeatable or ignored (you can see the decision in the UI), because there is usually no point in scanning non-repeatable requests.

You can head over to the Pentagrid Scan Controller github page, where the extension was published as open source. Visit the official Burp BApp store page of Pentagrid Scan Controller or load it directly inside Burp Pro via the BApp store. We recommend to watch the area41.io talk of Tobias Ospelt about improving web application scanning which explains the issues of Burp scanner, what the extension will improve and how you can use the extension. If you want to skip to the extension part directly, you can watch the talk from 24:20 min.

The extension version 0.1 already has many features, such as:

  • Scanning only in-scope items

  • Different deduplication techniques of items already scanned

  • Ignoring requests if the URL or the entire request matches a certain regex

  • Delays before repeatability checks or scans, which allows you to browse the website for a while without being disturbed. Additionally, if a request turns non-repatable after 10 seconds it's better not to scan it at all and let you know.

  • Scanning requests that are not repeatable (not recommended)

  • Building your own "interesting score" by URL file extension, HTTP status code, HTTP method and number of parameters. Define the score values.

  • Changing the heuristic keywords to figure out if a request is repeatable or not

  • Reimplemented JSON, XML and multipart injection. This was necessary, because extensions in Burp can currently not set a parameter value of JSON bodies through the official Burp API.

  • Added Non-Standard-HTTP header and URL path injection

  • Thread pooling to make sure Burp Proxy is never waiting for the extension (performance)

  • Make requests repeatable by using Hackvertor tags and show it in the UI. This means you can reuse the repeatable request in other tools such as the Burp Repeater.

  • The usual Burp extender API workarounds such as project-level storage.

  • Hiding items in the table

We're currently working on a new version of the extension where the following things will change:

  • New feature: Ignoring requests if the response matches a certain regex

  • New feature: Influence "interesting score" by response content-type (disabled by default as URL file extensions are usually sufficient)

  • New feature: To declutter the UI, most options will be hidden by default and there is a button to toggle advanced options.

  • New feature: The extension UI is divided into more tabs which group options together.

  • New feature: Divide repeatability reasoning and scan reasoning in the UI

We are also going to add additional scan capabilities (like ActiveScan++), for example:

  • Add non-standard HTTP headers as insertion points

  • Add URL paths as insertion points (often used as parameters in REST APIs)

  • Add Bearer Authorization HTTP header as an insertion point, this is important because Burp's new scanner checks for JSON Web Tokens (JWT) and is not yet checking the default "Authorization: Bearer ey[...]" location.

  • Add Basic Authorization HTTP header as an insertion point (username:password in base64)

  • Automated Smart Content Discovery as part of scanning, meaning automated Snallygaster checks are performed per directory.

Scanner options showing advanced smart content discovery options.

So stay tuned for the next release.