IP Address Open Source Intelligence for the Win

During the last edition of the Troopers security conference in March, I attended a talk about “JustMetaData”. It’s a tool developed by Chris Truncer to perform open source intelligence against IP addresses. Since then, I used this tool on a regular basis. Often when you’re using a tool, you have ideas to improve it. JustMetaData being written in Python and based on “modules”, it is easy to write your own. My first contribution to the project was a module to collect information from DShield.

Passive DNS is a nice technique to track the use of IP addresses, mainly to track malicious domains but it can be combined with other techniques to have a better view of the “scope” of an IP address. My new contribution is a module which uses the Microsoft bing.com API to collected hostnames associated with an IP address. Indeed, bing.com has a nice search query ‘ip:x.x.x.x’. To have a good view of an IP address is a key when you collect them to build blacklists. Indeed, behind an IP address, hundreds of websites can be hosted (in a shared environment). A good example is big WordPress providers. Blacklisting an IP address might have multiple impacts:

• Your users could be blocked while trying to visit a legitimate site hosted on the same IP address.
• This could generate a lot of false-positive events in your log monitoring environment.

Here is an example of the bing.com module:

$ ./Just-Metadata.py
################################################################################
# Just-Metadata #
################################################################################

[>] Please enter a command: load ip.tmp
[*] Loaded 1 systems
[>] Please enter a command: gather Bing_IP
Found 549 hostnames for 184.168.47.225
[>] Please enter a command: ip_info 184.168.47.225
...stuff deleted...
hostnames: la-reserve.be
cityplus.be
www.mintjens.be
consortiunews.com
www.marten-glas.be
...stuff delete...
[>] Please enter a command: exit

I wrote a Dockerfile for Just-Metadata to run it in a container. To fully automate the gathering process and reporting, I wrote a second patch to allow the user to specific a filename where to save the state of a research or the export of results in the right volume. Once the Docker created with the following command:

# docker build -t justmetadata --build-arg SHODAN_APIKEY=<yourkey> --build-arg BING_APIKEY= <yourkey> .

Now, the analyse of IP addresses can be fully automated (with the target IP addresses stored in ip.tmp):

# docker run -it --rm -v /tmp:/data justmetadata \
             -l /data/ip.tmp -g All \
             -e /data/results.csv

As a bonus, CVS files can be index by a Splunk instance (or any other tool) for further processing:

Here is an example of complete OSINT details about an IP address:

Based on these details, you can generate more accurate blacklists. I sent a pull request to Chris with my changes. In the meantime, you can use my repository to use the “Bing_IP” module.