Skip to content

Recon Part 3 – Gobuster and EyeWitness

Gobuster

TheColonial wrote a really cool tool called Gobuster which is similar to fierce but programmed in Go. I wanted to include it here because I tend to have better performance using this tool than fierce, by a LOT. Gobuster can be found on github here. There are a few issues to work out when setting up your Go environment but the official documentation should walk anyone through it and, if you have issues with the workspace creation, this StackOverflow article was useful.  Essentially, make sure to build the workspace to the required specifications to compile the code. Once done you’re free to do something like ln -s /path/to/compiled/gobuster /path/to/recon/tools/gobuster to make it easier to access.

Scanning with Gobuster

Once completed you should grab a wordlist or use ones from the fierce directory we talked about earlier. When we try to run the script here’s our options:

abraxas@AttackVM:~/recon/gobuster$ ./gobuster -h
Usage of ./gobuster:
 -P string
 Password for Basic Auth (dir mode only)
 -U string
 Username for Basic Auth (dir mode only)
 -a string
 Set the User-Agent string (dir mode only)
 -c string
 Cookies to use for the requests (dir mode only)
 -e Expanded mode, print full URLs
 -f Append a forward-slash to each directory request (dir mode only)
 -i Show IP addresses (dns mode only)
 -l Include the length of the body in the output (dir mode only)
 -m string
 Directory/File mode (dir) or DNS mode (dns) (default "dir")

 -n Don't print status codes
 -p string
 Proxy to use for requests [http(s)://host:port] (dir mode only)
 -q Don't print the banner and other noise
 -r Follow redirects
 -s string
 Positive status codes (dir mode only) (default "200,204,301,302,307")
 -t int
 Number of concurrent threads (default 10)
 -u string
 The target URL or Domain
 -v Verbose output (errors)
 -w string
 Path to the wordlist
 -x string
 File extension(s) to search for (dir mode only)

So we’ve got a good set of options to investigate. One of the important things to note is that this doesn’t strictly do DNS brute-forcing but also allows directory scanning/traversal. We won’t go in to this right now but will cover it at a later time in an article on Burp, DirBuster, and GoBuster.

The important flags from above, ignoring the directory mode options, are:

  1. m – set this to ‘dns’ for now
  2. i – if we want to see IPs from scan
  3. w – Path to the wordlist
  4. u – Target URL/Domain

Test Gobuster Scan

Let’s try this again on XKCD:

abraxas@AttackVM:~/recon/gobuster$ ./gobuster -u xkcd.com -w ../fierce-domain-scanner/hosts.txt -m dns -i

Gobuster v1.1                OJ Reeves (@TheColonial)
=====================================================
[+] Mode         : dns
[+] Url/Domain   : xkcd.com
[+] Threads      : 10
[+] Wordlist     : ../fierce-domain-scanner/hosts.txt
=====================================================
Found: blog.xkcd.com [192.0.78.13, 192.0.78.12]
Found: c.xkcd.com [107.6.98.34]
Found: dynamic.xkcd.com [107.6.98.34]
Found: es.xkcd.com [78.46.202.206]
Found: forums.xkcd.com [104.196.146.194]
Found: m.xkcd.com [104.156.81.67, 104.156.85.67, 23.235.33.67, 23.235.37.67]
Found: irc.xkcd.com [107.6.89.242, 178.79.176.77, 208.87.120.111, 216.93.242.12, 66.228.37.186, 2a01:7e00::f03c:91ff:fedf:5bf1, 2001:4830:120:1::2, 2600:3c03::f03c:91ff:fedf:a1ab]
Found: mobile.xkcd.com [104.156.85.67, 23.235.33.67, 23.235.37.67, 104.156.81.67]
=====================================================

A few things to note:

  1. This scan, for me, is many times faster than fierce (it also defaults to 10 threads instead of fierce’s one thread)
  2. This scan appears to be more accurate. While it does catch more than fierce I still recommend running both tools more than once for optimal results

Tell me if you have the same experience! I’d love to benchmark these tools in a later post and knowing what experiences others have would be great as well.

EyeWitness

Now on to one of my favorite tools in the recon arsenal! EyeWitness is a tool by ChrisTruncer which takes a set of IP addresses or hostnames, attempts to screenshot them, and creates a nice report file!

First, we’ll clone the repository from here:

abraxas@AttackVM:~/recon git clone https://github.com/ChrisTruncer/EyeWitness

Once cloned and built we can run a -h on it and see our options. I’m excluding some of the text for brevity:

abraxas@AttackVM:~/recon/eyewitness$ ./EyeWitness.py -h

################################################################################
#                                  EyeWitness                                  #
################################################################################

usage: EyeWitness.py [--web] [--headless] [--rdp] [--vnc] [--all-protocols]
                     [-f Filename] [-x Filename.xml] [--single Single URL]
                     [--createtargets targetfilename.txt] [--no-dns]
                     [--timeout Timeout] [--jitter # of Seconds]
                     [--threads # of Threads] [-d Directory Name]
                     [--results Hosts Per Page] [--no-prompt]
                     [--user-agent User Agent] [--cycle User Agent Type]
                     [--difference Difference Threshold]
                     [--proxy-ip 127.0.0.1] [--proxy-port 8080]
                     [--show-selenium] [--resolve]
                     [--add-http-ports ADD_HTTP_PORTS]
                     [--add-https-ports ADD_HTTPS_PORTS] [--prepend-https]
                     [--vhost-name hostname] [--active-scan] [--resume ew.db]

The important things to note are the following:

  1. Use --web when looking for web page headers and screenshots while using an OS with a GUI
  2. Use –-headless when running from a GUI-less system, --web will not work on a terminal-only system
  3. Make sure your targets file (-f) is only IP addresses or URLs

Let’s take our XKCD scan we did earlier, modify it a bit for readability, and use this to prepare some data to run through EyeWitness.

Preparing EyeWitness Data

The command I will use to pull urls from GoBuster is this:

abraxas@AttackVM:~/recon/eyewitness$ ../gobuster/gobuster -w ../fierce-domain-scanner/hosts.txt -u xkcd.com -m dns > ../eyewitness/xkcd_urls.txt

If we cat that file we’ll see the following:

abraxas@AttackVM:~/recon/eyewitness$ cat xkcd_urls.txt

Gobuster v1.1                OJ Reeves (@TheColonial)
=====================================================
[+] Mode         : dns
[+] Url/Domain   : xkcd.com
[+] Threads      : 10
[+] Wordlist     : ../fierce-domain-scanner/hosts.txt
=====================================================
Found: blog.xkcd.com
Found: c.xkcd.com
Found: dynamic.xkcd.com
Found: es.xkcd.com
Found: forums.xkcd.com
Found: irc.xkcd.com
Found: m.xkcd.com
Found: mobile.xkcd.com
=====================================================

This isn’t quite usable yet, we need the file to be in the form of: hostname <newline> hostname <newline>

Let’s write a quick bash command to extract this information for us:

cat xkcd_urls.txt | grep "Found" | cut -d" " -f2 > targets.txt

This will generate a targets.txt file from the xkcd_urls.txt for us, it looks like this:

abraxas@AttackVM:~/recon/eyewitness$ cat xkcd_urls.txt | grep "Found" | cut -d" " -f2 > targets.txt
abraxas@AttackVM:~/recon/eyewitness$ cat targets.txt
blog.xkcd.com
c.xkcd.com
dynamic.xkcd.com
es.xkcd.com
forums.xkcd.com
irc.xkcd.com
m.xkcd.com
mobile.xkcd.com

This looks okay but I decided I wanted to write something a little more reusable so I don’t have to remember how to do that each time. Here is a bash script I wrote to handle that in a reusable way:

#!/bin/bash

read -p "Please input filename to extract hosts from: " filename
read -p "Input target name: " targetname
mkdir $targetname 2> /dev/null || true
result=$(cat $filename | grep "Found" | cut -d" " -f2 > $targetname/targets.txt)
echo "Success!"
echo "Created folder $targetname with extracted urls in 'targets.txt'"

This prompts for your input file, prompts for the name of the target, and then creates a directory for the target and saves your targets there! Pretty slick, right? Here’s what running it looks like:

abraxas@AttackVM:~/recon/eyewitness$ ./gob-extractor.sh
Please input filename to extract hosts from: xkcd_urls.txt
Input target name: xkcd
Success!
Created folder xkcd with extracted urls in 'targets.txt'

Now if we cat that file we should have the same desired output:

abraxas@AttackVM:~/recon/eyewitness$ cat xkcd/targets.txt
blog.xkcd.com
c.xkcd.com
dynamic.xkcd.com
es.xkcd.com
forums.xkcd.com
irc.xkcd.com
m.xkcd.com
mobile.xkcd.com

Success!

Using EyeWitness

Now that we have our targets file, lets see how this works and run the following:

 ./EyeWitness.py  --headless -f xkcd/targets.txt -d xkcd_report

This returns the following:

################################################################################         
#                                  EyeWitness                                  #         
################################################################################         
                                                                                         
Starting Web Requests (8 Hosts)                                                          
Attempting to screenshot http://blog.xkcd.com                                            
Attempting to screenshot http://c.xkcd.com                                               
Attempting to screenshot http://dynamic.xkcd.com                                         
Attempting to screenshot http://es.xkcd.com                                              
Attempting to screenshot http://forums.xkcd.com                                          
Attempting to screenshot http://irc.xkcd.com                                             
Attempting to screenshot http://m.xkcd.com                                               
Attempting to screenshot http://mobile.xkcd.com                                          
Finished in 18.7914259434 seconds                                                        
                                                                                         
[*] Done! Report written in the /home/abraxas/recon/eyewitness/xkcd_report folder!       
Would you like to open the report now? [Y/n]

So it looks like it worked! I’ll SCP the files to my desktop and we can take a look at the results.

EyeWitness Reports

Without going in to too much detail, here’s some of the information EyeWitness returns back to us:

  • Report Folder Contents
  • Summary section
  • Sample Report

As you can see, there’s some really good data – headers, source code, screenshots, and more.

It’s worth noting that, unlike our other scans/tools which are almost entirely passive and don’t touch target infrastructure, this will attempt to load each entry in our targets.txt file.

In our next article we will cover additional recon tools and strategies! As always, any criticism or feedback is welcome.

Leave a Reply

Your email address will not be published. Required fields are marked *