For a while I’ve wanted to write about the demise of WHOIS and the challenges faced by investigators trying to identify who owns or operates illegal websites that are hidden behind layers of obfuscation and obscurity. There’s no perfect solution, but there are a number of methods that can help shed light on the ownership of websites, especially when combined with other strands of information.
Anyone who has tried to investigate suspicious websites will have no doubt come up against Cloudflare at some point. (I should stress that Cloudflare are a legitimate and useful service provider whose massive network resources allow websites to be safely protected from DDOS and other malicious activity.) Look up any domain hosted behind Cloudflare with a service like DNSLytics or CentralOps and all you’ll get is the IP address and other details for the Cloudflare server. You can’t see who really hosts the domain, or what it’s true IP address is.
Enter Cloudflair
Cloudflair, made by Christophe Tafani-Dereeper, is a really useful tool that helps to expose the real IP addresses for malicious domains that hide behind Cloudflare. It works on the principle that every domain that uses https has a unique SSL security certificate issued, and that by looking across the internet for that exact certificate, it can also identify other IP addresses linked to that same certificate, regardless of whether it’s behind Cloudflare or not.
For example, let’s say I run a criminal website at www.humantrafficking.org and I protect it with Cloudflare. Anyone who does a DNS lookup or WHOIS enquiry will just see a load of data about Cloudflare and the true details of my domain will be hidden. The IP address presented to the investigator will be something similar to 104.28.13.49 (a real Cloudflare IP) even though my real IP address is 45.64.244.13* and my real webhost is Evildude Web Enterprises.
Despite this a webcrawler like Censys will record the details of every SSL certificate it comes across and what the associated IP address is. This means that although people browsing to my evil website or conducting Whois/DNS lookups will only find the Cloudflare IP 104.28.13.49, Censys will also visit the real IP 45.64.244.13 and document the fact that the same SSL certificate for the same domain is linked to both IPs. In other words as long as your website’s true IP address is visible to the internet, Censys will still be able to find and document it in spite of Cloudflare. Cloudflair simply takes the details of the domain you’re interested in, checks them with Censys, and then reveals all the IP addresses linked to a particular domain, whether they are behind Cloudflare or not. There’s a more detailed explanation of how it all works on the author’s blog here.
Setting Up And Running Cloudflair
Cloudflair runs best in a Linux environment where Python is pre-installed as standard. There are two options to run and install the program, either by command line installation or by running it from a Docker image. I found there were a few conflicts and issues with a couple of Python libraries when running it from the command line but using the Docker image was much easier so I stuck with that. I’m fairly new to using Docker but with Cloudflair I found it to be very smooth and straightforward. My standard OSINT build runs on Linux Mint 19.1 and running Cloudflair was pretty trouble-free. These instructions are to run the Docker image rather than directly from the terminal with Python.
Step 1
Create a free account at Censys.io. You’ll need to go to the Account section to get your API key and API secret to run the program.
Step 2
Clone the Cloudflair Github Repository
$ git clone https://github.com/christophetd/cloudflair.git
Step 3
Install Docker if you haven’t got it already
$ sudo apt install docker.io
Step 4
If this is the first time installing Docker, you might run into issues when you try to run commands as a non-root user. Make sure your user account is part of the docker group. You’ll need to logout and log back in after making these changes.
$ sudo useradd -aG docker myusername
(In Mint/Ubuntu you might need to switch to root to do this. This can be done with sudo -i and then exit to return to your regular user account)
Step 5
Run the following command, with the target domain added to the end. Make sure to add in your Censys API key and API secret to make it work:
$ docker run --rm -e CENSYS_API_ID=xxxxxxxxxxx -e CENSYS_API_SECRET=xxxxxxxxxxx christophetd/cloudflair evilwebsite.org
The first time you run this command the Docker image will need to be downloaded. The output will look something like this (copied from the author’s GitHub page:)
[*] The target appears to be behind CloudFlare.
[*] Looking for certificates matching "myvulnerable.site" using Censys
[*] 75 certificates matching "myvulnerable.site" found.
[*] Looking for IPv4 hosts presenting these certificates...
[*] 10 IPv4 hosts presenting a certificate issued to "myvulnerable.site" were found.
- 51.194.77.1
- 223.172.21.75
- 18.136.111.24
- 127.200.220.231
- 177.67.208.72
- 137.67.239.174
- 182.102.141.194
- 8.154.231.164
- 37.184.84.44
- 78.25.205.83
[*] Retrieving target homepage at https://myvulnerable.site
[*] Testing candidate origin servers
- 51.194.77.1
- 223.172.21.75
- 18.136.111.24
responded with an unexpected HTTP status code 404
- 127.200.220.231
timed out after 3 seconds
- 177.67.208.72
- 137.67.239.174
- 182.102.141.194
- 8.154.231.164
- 37.184.84.44
- 78.25.205.83
[*] Found 2 likely origin servers of myvulnerable.site!
- 177.67.208.72 (HTML content identical to myvulnerable.site)
- 182.102.141.194 (HTML content identical to myvulnerable.site)Now you’ve got behind Cloudflare and have some real IPs to begin working with for your enquiries!
Step 6
To avoid having to enter the long Docker command each time, create the following alias in your bashrc file:
$ nano ~/.bashrc
Add the following lines to the end of your .bashrc file:
alias cloudflair='docker run --rm -e CENSYS_API_ID=xxxxxxxxxxx -e CENSYS_API_SECRET=xxxxxxxxxxx christophetd/cloudflair'
Save and exit nano, then exit and restart the terminal to effect the changes.
Now you can conduct enquiries by simply entering:
$cloudflair mytargetwebsite.com
*All IP addresses in this post are made up unless stated.