OSSEC Speaks “ArcSight”

Log management… A hot topic! There are plenty of solutions to manage your logs. Like in all IT domains, there are two major categories: free and commercial tools. Both have pro and cons. No big debate here, contrariwise I’ll show you a good example of a mix between both worlds. Let’s take two products: OSSEC & ArcSight. OSSEC is a free log management solution. If you follow my blog, you know that it’s one of my favorite toy at the moment. But I’m also involved in ArcSight projects. It’s a very robust commercial log management / SIEM solution (let’s stop the promotion here).

One of the main issue in log management is the way of handling events. All of them are generated by multiple devices or applications in several formats. Once collected, those events must be parsed and normalized. ArcSight developed a specific format called CEF (“Common Event Format“). It is described on their website as following:

“The Common Event Format (CEF) is an open log management standard that improves the interoperability of security-related information from different security and network devices and applications. CEF is based on expertise from building support for over 275 products across more than 35 solution categories and is the first log management standard to support a broad range of device types. CEF enables technology companies and customers to use a common event log format so that data can easily be collected and aggregated for analysis by an enterprise management system.“

More information is available here if you are interested. Honestly, there are not so many products which support CEF today. But it looks that it’s changing and big players started to implement the generation of native CEF events in their products. What makes CEF important? By generating CEF events, you remove an important step in the log management process: the normalization. CEF events already contains all the important information. Basically, a CEF event is based on the following structure:

  CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|\
  Name|Severity|Extension

The field names explain by themselves their meaning. The last one (“Extension“) is used to pass extra information. This field contains a collection of key=value pairs.

Example:

  CEF:0|ArcSight|Logger|5.0.0.5355.2|sensor:115|Logger Internal Event|1|\
  cat=/Monitor/Sensor/Fan5 cs2=Current Value cnt=1 dvc=10.0.0.1 cs3=Ok \
  cs1=null type=0 cs1Label=unit rt=1305034099211 cs3Label=Status cn1Label=value \
  cs2Label=timeframe

Let’s go back to OSSEC. This software can grab very easily events from multiple devices and applications, process them and generate useful alerts. OSSEC has also an interesting “remote syslog” feature to forward alerts to a remote Syslog server.  Why not let OSSEC generate CEF events by itself? ArcSight could be used to centralize logs coming from multiple sources and agents as well as OSSEC. OSSEC is very light and free, why not integrate it with ArcSight? A winning team!

I wrote a small patch to add CEF support to the csyslogd daemon. This one is responsible of sending alerts to our remote Syslog server. My patch adds a new format that can be specified in your ossec.conf file:

  <syslog_output>
    <server>10.0.0.1</server>
    <port>514</port>
    <format>cef</format>
  </syslog_output>

Once you applied the patch, recompile the os_csyslogd module, adapt your ossec.conf and restart OSSEC! Alerts will  now be sent in CEF format. The following information is mapped into the new event:

• Vendor (Trend Micro Inc.)
• Product (OSSEC HIDS)
• Version (v2.5.1)
• Rule ID
• Rule Name
• Severity
• Message (which will contains the “srcip” and “user” fields)

Example:

  CEF:0|Trend Micro Inc.|OSSEC HIDS|v2.5.1|5302|\
  User missed the password to change UID to root.|9|dvc=ubuntusvr \
  cs2=ubuntusvr->/var/log/auth.log cs2Label=Location src= suser=root \
  msg=May 11 21:16:05 ubuntusvr su[24120]: - /dev/pts/1 xavier:root

If you specify in the <syslog_output> XML block the IP address of your ArcSight device (properly configured to receive CEF events over UDP), they will be smoothly indexed and stored in the database:

But, will you ask why implement this? Here are some scenarios:

• OSSEC integrates nice features like a rootkit detection and FIM (“File Integrity Monitoring“) but not ArcSight. This could add more value to your SIEM.
• During a transition period between the two products (and avoid multiple agents on your servers)
• To use the powerful ArcSight correlation engine without deploying a huge platform (Log archiving remains at OSSEC level)
• OSSEC Decoders are much easier to write and deploy compared to the ArcSight FlexConnector s(which also require a developer license !)
• Etc…

Note that this patch does not implement the classification of the OSSEC alerts using the ArcSight mechanism (via the field “categorySignificance“). It’s just a dumb rewrite of the alerts using a CEF pattern. If you have ideas or comments, they are welcome!

My patch is available here and, as usual, is available for free and without any warranty.