Professional

Running a full crawl and audit

• Last updated: August 30, 2024

• Read time: 4 Minutes

Burp Scanner can crawl and audit web applications from one or more start URLs. When scanning, it follows any links from these URLs into the application to map out content.

To run a full crawl and audit of a web application, click New Scan > Webapp scan on the Dashboard to open the scan launcher. The launcher has tabs that configure various aspects of the scan.

Step 1: Configure scan details

The Scan Details tab enables you to configure basic details of the scan, including the type of scan you want to run and the URL from which the scan should start:

1. Select a Scan type:

   • Crawl and audit.
   • Crawl.

2. Enter a URL into the URLs to scan field. This is the URL that the scan starts from. To enter multiple URLs, place each on a new line.

3. Select Protocol settings.

   • Scan using HTTP & HTTPS.
   • Scan using my specified protocols. If you select this option, make sure you specify the protocols in the URLs to scan field.

4. Optionally, use the settings in the Detailed scope configuration section to refine the scan scope. This limits the URLs that Burp Scanner can access during the scan.

5. If you want to isolate the scan, select Run isolated scan. Results from isolated scans do not appear in the Target > Site map, Target > Crawl paths, or Dashboard > All issues tabs. It can be useful to isolate a scan if you want to test scan configurations without impacting "live" scan results, for example.

Once you have specified scan details, select the Scan configuration tab.

Step 2: Select a scan configuration

Scan configurations are groups of settings that define how a scan is performed. You must select a scan configuration before you can run your scan.

The Scan configuration tab enables you to either select a preset scan mode or define a custom configuration:

• Preset scan modes are predefined collections of scan settings. They enable you to quickly adjust how the scan balances speed and coverage. To select a preset scan mode, make sure that Use a preset scan mode is selected and click one of the available options.

• Custom scan configurations enable you to fine-tune Burp Scanner's behavior to meet your needs. To manage custom scan configurations, select Use a custom configuration. You can perform the following actions:

  • Create new configurations from scratch.
  • Select existing configurations from your configuration library.
  • Import configurations from other installations of Burp Suite.

If you want Burp to remember the selected scan mode next time you open the scan launcher, select Remember my choice for future scans.

Once you have selected your configuration, either click OK to start the scan or select another tab to configure further details.

Step 3: Configure application logins (optional)

The Application login tab enables you to provide credentials for Burp Scanner to submit when it finds login forms. This enables it to discover and audit content that is only accessible to authenticated users.

There are two types of login credential you can add in Burp Suite Professional:

• Username and password pairs are intended for sites that use a basic, single-step login mechanism. To manage username and password pairs, select Use login credentials. From here, you can add new credentials or edit your existing ones.
• Recorded login sequences are intended for sites that use more complex login mechanisms, such as Single Sign-On. You can record login sequences using the Burp Suite Navigation Recorder Chrome extension, which is pre-installed in Burp's browser. To manage your recorded login sequences, select Use recorded login sequences. From here, you can add new sequences or edit your existing ones.

You can only use one of the login mechanisms per site.

Step 4: Select a resource pool (optional)

A resource pool is a group of tasks that share a quota of network resources. You can configure each resource pool with its own throttling settings. These control the number of requests that can be made concurrently, or the rate at which requests can be made, or both.

The Resource pools tab enables you to define the pool in which your scan will run. You can select an existing resource pool from the list, or create a new resource pool.