Running httpx
Learn about running httpx with examples including commands and output
For all of the flags and options available for httpx be sure to check out the Usage page. On this page we’ll share examples running httpx with specific flags and goals
and the output you can expect from each.
Basic Examples
ASN Fingerprint
Use httpx with the -asn flag for ASN (Autonomous System Number) fingerprinting, an effective technique for mapping the network affiliations of various domains.
ASN Input
Specify an autonomous system’s number (ASN) and httpx will fetch all ip addresses of that autonomous system and probe them
CIDR Input
Run httpx with CIDR input (for example 173.0.84.0/24)
Docker Run
Use Docker to run httpx in an isolated container. For example, by piping subdomain lists into the Docker container, you can seamlessly perform probing across multiple targets, harnessing the power of httpx without direct installation requirements.
Error Page Classifier and Filtering
The Error Page Classifier and Filtering feature aims to add intelligence to httpx by enabling httpx to classify and filter out common error pages returned by web applications.
It is an enhancement geared towards reducing noise and helping focus on actual results.
Using the -fep or -filter-error-page option creates a filtered error page in the file filtered_error_page.json in jsonline format.
Favicon Hash
Extract and display the mmh3 hash of the ‘/favicon.ico’ file from given targets.
File/Path Bruteforce
Use httpx with the -path option for efficient File/Path Bruteforcing. This feature allows probing specific paths across multiple URLs, uncovering response codes and revealing potentially vulnerable or unsecured endpoints in web applications.
File Input
Run httpx with the -probe flag against all the hosts in hosts.txt to return URLs with probed status.
JARM Fingerprint
Use httpx with the -jarm flag to leverage JARM fingerprinting, a specialized tool for active TLS server fingerprinting.
This approach enables the identification and categorization of servers based on their TLS configurations, making it an effective method for detecting and analyzing diverse internet servers,
including potential security threats.
Tool Chain
Combining httpx with other tools like subfinder can elevate your web reconnaissance.
For example, pipe results from subfinder directly into ‘httpx’ to efficiently identify active web servers and their technologies across various subdomains of a given target.
URL probe
Run httpx against all the hosts and subdomains in hosts.txt to return URLs running an HTTP webserver.
UI Dashboard (PDCP Integration)
Configure API Key
To upload your assets to PDCP you will need to create a free API Key
-
Obtain API Key:
- Visit https://cloud.projectdiscovery.io
- Open the setting menu from the top right and select “API Key” to create your API Key
- Use the
httpx -authcommand, and enter your API key when prompted.
Configure Team (Optional)
If you want to upload the asset results to a team workspace instead of your personal workspace, you can configure the Team ID. You can use either the CLI option or the environment variable, depending on your preference.
-
Obtain Team ID:
- To obtain your Team ID, navigate to https://cloud.projectdiscovery.io/settings/team and copy the Team ID from the top right section.
-
CLI Option:
- Use the
-tidor-team-idoption to specify the team ID. - Example:
nuclei -tid XXXXXX -dashboard
- Use the
-
ENV Variable:
- Set the
PDCP_TEAM_IDenvironment variable to your team ID. - Example:
export PDCP_TEAM_ID=XXXXX
- Set the
Either of these options is sufficient to configure the Team ID.
Run httpx with UI Dashboard
To run httpx and upload the results to the UI Dashboard:
Uploading to an Existing Asset Group
To upload new assets to an existing asset group:
Setting an Asset Group Name
To set a custom asset group name:
Additional upload options
-pd, -dashboard: Enable uploading ofhttpxresults to the ProjectDiscovery Cloud (PDCP) UI Dashboard.-aid, -asset-id string: Upload new assets to an existing asset ID (optional).-aname, -asset-name string: Set the asset group name (optional).-pdu, -dashboard-upload string: Uploadhttpxoutput file (jsonl) to the ProjectDiscovery Cloud (PDCP) UI Dashboard.
Environment Variables
export ENABLE_CLOUD_UPLOAD=true: Enable dashboard upload by default.export DISABLE_CLOUD_UPLOAD_WARN=true: Disable dashboard warning.export PDCP_TEAM_ID=XXXXX: Set the team ID for the ProjectDiscovery Cloud Platform.
Expanded Examples
Using httpx as a library
httpx can be used as a library by creating an instance of the Option struct and populating it with the same options that would be specified via CLI. Once validated, the struct should be passed to a runner instance (to be closed at the end of the program) and the RunEnumeration method should be called.
- A basic example of how to use httpx as a library is available in the GitHub examples folder.
Using httpx screenshot
Httpx includes support for taking a screenshot with -screenshot that gives users the ability to take screenshots of target URLs, pages, or endpoints along with the rendered DOM.
This functionality enables a comprehensive view of the target’s visual content.
Rendered DOM body is also included in json line output when -screenshot option is used with -json option.
To use this feature, add the -screenshot flag to the httpx command.
httpx -screenshot -u https://example.com
httpx will be slower when using the -screenshot option.Domain, Subdomain, and Path Support
The -screenshot option is versatile and can be used to capture screenshots for domains, subdomains, and even specific paths when used in conjunction with the -path option:
Using with Other Tools
In the example below we’re providing subfinder output to the httpx screenshot.
System Chrome Support
By default, httpx uses the go-rod library to install and manage Chrome for taking screenshots.
However, if you prefer to use your locally installed system Chrome, add the -system-chrome flag:
Output Directory
Screenshots are stored in the output/screenshot directory by default. To specify a custom output directory, use the -srd option:
Body Preview
Body preview shows first N characters of response. And strip html tags in response.
Running httpx
Learn about running httpx with examples including commands and output
For all of the flags and options available for httpx be sure to check out the Usage page. On this page we’ll share examples running httpx with specific flags and goals
and the output you can expect from each.
Basic Examples
ASN Fingerprint
Use httpx with the -asn flag for ASN (Autonomous System Number) fingerprinting, an effective technique for mapping the network affiliations of various domains.
ASN Input
Specify an autonomous system’s number (ASN) and httpx will fetch all ip addresses of that autonomous system and probe them
CIDR Input
Run httpx with CIDR input (for example 173.0.84.0/24)
Docker Run
Use Docker to run httpx in an isolated container. For example, by piping subdomain lists into the Docker container, you can seamlessly perform probing across multiple targets, harnessing the power of httpx without direct installation requirements.
Error Page Classifier and Filtering
The Error Page Classifier and Filtering feature aims to add intelligence to httpx by enabling httpx to classify and filter out common error pages returned by web applications.
It is an enhancement geared towards reducing noise and helping focus on actual results.
Using the -fep or -filter-error-page option creates a filtered error page in the file filtered_error_page.json in jsonline format.
Favicon Hash
Extract and display the mmh3 hash of the ‘/favicon.ico’ file from given targets.
File/Path Bruteforce
Use httpx with the -path option for efficient File/Path Bruteforcing. This feature allows probing specific paths across multiple URLs, uncovering response codes and revealing potentially vulnerable or unsecured endpoints in web applications.
File Input
Run httpx with the -probe flag against all the hosts in hosts.txt to return URLs with probed status.
JARM Fingerprint
Use httpx with the -jarm flag to leverage JARM fingerprinting, a specialized tool for active TLS server fingerprinting.
This approach enables the identification and categorization of servers based on their TLS configurations, making it an effective method for detecting and analyzing diverse internet servers,
including potential security threats.
Tool Chain
Combining httpx with other tools like subfinder can elevate your web reconnaissance.
For example, pipe results from subfinder directly into ‘httpx’ to efficiently identify active web servers and their technologies across various subdomains of a given target.
URL probe
Run httpx against all the hosts and subdomains in hosts.txt to return URLs running an HTTP webserver.
UI Dashboard (PDCP Integration)
Configure API Key
To upload your assets to PDCP you will need to create a free API Key
-
Obtain API Key:
- Visit https://cloud.projectdiscovery.io
- Open the setting menu from the top right and select “API Key” to create your API Key
- Use the
httpx -authcommand, and enter your API key when prompted.
Configure Team (Optional)
If you want to upload the asset results to a team workspace instead of your personal workspace, you can configure the Team ID. You can use either the CLI option or the environment variable, depending on your preference.
-
Obtain Team ID:
- To obtain your Team ID, navigate to https://cloud.projectdiscovery.io/settings/team and copy the Team ID from the top right section.
-
CLI Option:
- Use the
-tidor-team-idoption to specify the team ID. - Example:
nuclei -tid XXXXXX -dashboard
- Use the
-
ENV Variable:
- Set the
PDCP_TEAM_IDenvironment variable to your team ID. - Example:
export PDCP_TEAM_ID=XXXXX
- Set the
Either of these options is sufficient to configure the Team ID.
Run httpx with UI Dashboard
To run httpx and upload the results to the UI Dashboard:
Uploading to an Existing Asset Group
To upload new assets to an existing asset group:
Setting an Asset Group Name
To set a custom asset group name:
Additional upload options
-pd, -dashboard: Enable uploading ofhttpxresults to the ProjectDiscovery Cloud (PDCP) UI Dashboard.-aid, -asset-id string: Upload new assets to an existing asset ID (optional).-aname, -asset-name string: Set the asset group name (optional).-pdu, -dashboard-upload string: Uploadhttpxoutput file (jsonl) to the ProjectDiscovery Cloud (PDCP) UI Dashboard.
Environment Variables
export ENABLE_CLOUD_UPLOAD=true: Enable dashboard upload by default.export DISABLE_CLOUD_UPLOAD_WARN=true: Disable dashboard warning.export PDCP_TEAM_ID=XXXXX: Set the team ID for the ProjectDiscovery Cloud Platform.
Expanded Examples
Using httpx as a library
httpx can be used as a library by creating an instance of the Option struct and populating it with the same options that would be specified via CLI. Once validated, the struct should be passed to a runner instance (to be closed at the end of the program) and the RunEnumeration method should be called.
- A basic example of how to use httpx as a library is available in the GitHub examples folder.
Using httpx screenshot
Httpx includes support for taking a screenshot with -screenshot that gives users the ability to take screenshots of target URLs, pages, or endpoints along with the rendered DOM.
This functionality enables a comprehensive view of the target’s visual content.
Rendered DOM body is also included in json line output when -screenshot option is used with -json option.
To use this feature, add the -screenshot flag to the httpx command.
httpx -screenshot -u https://example.com
httpx will be slower when using the -screenshot option.Domain, Subdomain, and Path Support
The -screenshot option is versatile and can be used to capture screenshots for domains, subdomains, and even specific paths when used in conjunction with the -path option:
Using with Other Tools
In the example below we’re providing subfinder output to the httpx screenshot.
System Chrome Support
By default, httpx uses the go-rod library to install and manage Chrome for taking screenshots.
However, if you prefer to use your locally installed system Chrome, add the -system-chrome flag:
Output Directory
Screenshots are stored in the output/screenshot directory by default. To specify a custom output directory, use the -srd option:
Body Preview
Body preview shows first N characters of response. And strip html tags in response.