MISP/Cuckoo

Quick Integration of MISP and Cuckoo

With the number of attacks that we are facing today, defenders are looking for more and more IOC’s (“Indicator of Compromise) to feed their security solutions (firewalls, IDS, …). It becomes impossible to manage all those IOC’s manually and automation is the key. There are two main problems with this amount of data:

  1. How to share them in a proper way (remember: sharing is key).
  2. How to collect and prepare them to be shared.

Note that I’m considering in this post only the “technical” issues with IOC’s. There are much more issues like their accuracy (which can be different between different environments).

To search for IOC’s, I’m using the following environment: A bunch of honeypots capture samples that, if interesting, are analyzed by a Cuckoo sandbox. To share the results with peers, a MISP instance is used.

MISP-Cuckoo

In this case, a proper integration between Cuckoo and MISP is the key. It is implemented in both ways. The results of the Cucko analyzis are enriched with IOC’s found in MISP. IOC’s found in the sample are correlated with MISP and the event ID, description and level are displayed:

In the other way, Cuckoo submits the results of the ianalyzes to MISP:

Cuckoo 2.0 comes with ready-to-use modules to interact with the MISP REST API via the PyMISP Python module. There is one processing module (to search for existing IoC’s in MISP) and one reporting module (to create a new event in MISP). The configuration is very simple, just define your MISP URL and API key in the proper configuration files and you’re good to go:

# cd $CUCKOO_HOME/conf
# grep -A 2 -B 2 misp *.conf
processing.conf-enabled = yes
processing.conf-
processing.conf:[misp]
processing.conf-enabled = yes
processing.conf:url = https://misp.xxxxxxxxxx
processing.conf-apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
maxioc = 100
--
reporting.conf-logname = syslog.log
reporting.conf-
reporting.conf:[misp]
reporting.conf-enabled = yes
reporting.conf:url = https://misp.xxxxxxxxxx
reporting.conf-apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

But, in my environment, the default modules generated too many false positives in both ways. I patched them to support the more configuration keywords to better control what is exchanged between the two tools.

In the processing module:

only_ids = [yes|no]

If defined, only attributes with the “IDS” flag set to “1” will be displayed in Cuckoo.

ioc_blacklist = 8.8.8.8,8.8.4.4,www.google.com

This parameter allows you to define a list (comma delimited) of IOC’s that you do NOT want in Cuckoo. Typically, you’ll add here DNS servers, specific URLs.

In the reporting module:

tag = Cuckoo

Specify the tag to be added to created event in MISP. Note that the tag must be created manually.

# Default distribution level:
# your_organization = 0
# this_community = 1
# connected_communities = 2
# all_communities = 3
distribution=0

The default distribution level assigned to the created event (default: 0)

# Default threat level:
# high = 1
# medium = 2
# low = 3
# undefined = 4
threat_level_id=4

The default threat level assigned to the created event (default: 4)

# Default analysis level:
# initial = 0
# ongoing = 1
# completed = 2
analysis = 0

The default analysis status assigned to the created event (default: 0)

ioc_blacklist = 8.8.8.8,8.8.4.4,www.google.com,crl.verisign.com,sc.symcb.com

Again, a blacklist (comma separated) of IOC’s that you do NOT want to store in MISP.

Events are created in MISP with an “unpublished” status. This allows you to review them, add manually some IOC’s, to merge different events, add some tags or change default values.

The patched files for Cuckoo are available in my GitHub repository.

11 comments

  1. That’s… interesting! I’ve no idea about this issue… Is the cuckoo.conf correct? (like a non printable character in the config?)

  2. Very strange.
    My API has been renewed.
    HTTPS has been removed.
    From Cuckoo box, I ran an example PyMISP script (last.py) and it can retrieve some IOCs ; with the same inputs (UR & APIkey).

    But it seems that Cuckoo can’t reach MISP during analysis process and I get the same error.
    Here is the full log on Cuckoo side:
    ====
    2017-02-23 11:27:43,677 [lib.cuckoo.core.plugins] ERROR: Failed to run the processing module “MISP” for task #87:
    Traceback (most recent call last):
    File “/opt/cuckoo/lib/cuckoo/core/plugins.py”, line 242, in process
    data = current.run()
    File “/opt/cuckoo/modules/processing/misp.py”, line 81, in run
    self.misp = PyMISP(self.url, self.apikey, False, “json”)
    File “/usr/lib/python2.7/site-packages/pymisp/api.py”, line 113, in __init__
    raise PyMISPError(‘Unable to connect to MISP ({}). Please make sure the API key and the URL are correct (http/https is required): {}’.format(self.root_url, e))
    PyMISPError: Unable to connect to MISP (http:///). Please make sure the API key and the URL are correct (http/https is required): 504 Server Error: Gateway Timeout
    ===
    Here is the full log on MISP side. I get some 403 errors, but I didn’t found this file on server so I couldn’t check permissions.
    ===
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:23 +0100] “GET /servers/getPyMISPVersion.json HTTP/1.1” 403 301 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:23 +0100] “GET /attributes/describeTypes.json HTTP/1.1” 403 301 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:23 +0100] “POST /events/restSearch/download HTTP/1.1” 403 299 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:27 +0100] “GET /servers/getPyMISPVersion.json HTTP/1.1” 403 301 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:27 +0100] “GET /attributes/describeTypes.json HTTP/1.1” 403 301 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:08:28 +0100] “POST /events/restSearch/download HTTP/1.1” 403 299 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:13:17 +0100] “POST /events/restSearch/download HTTP/1.1” 200 910165 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:17:07 +0100] “GET /servers/getPyMISPVersion.json HTTP/1.1” 200 20 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:17:07 +0100] “GET /attributes/describeTypes.json HTTP/1.1” 200 15178 “-” “PyMISP 2.4.65 – Python 2.7.5”
    xxx.xxx.xxx.xxx – – [23/Feb/2017:14:17:07 +0100] “POST /events/restSearch/download HTTP/1.1” 200 910165 “-” “PyMISP 2.4.65 – Python 2.7.5”

    I think I will open a issue on MISP’s Github.
    Thanks,
    Regards,

  3. Many thanks Xavier for your post.
    It’s very useful if you begin with MISP 🙂

    Tried today, but after reviewing my settings, I still reach an error on Cuckoo’s side when I submit a sample:
    PyMISPError: Unable to connect to MISP (http://<my_misp_IP/). Please make sure the API key and the URL are correct (http/https is required): 504 Server Error: Gateway Timeout

    If you have any idea 😉
    Thanks again.

Leave a Reply

Your email address will not be published. Required fields are marked *