Are You Playing “Cold Case” with Your Logs?
Two days ago, I attended an event about “big data” (yeah, another buzz word) and how to use it for security purposes. One of the presented talks was very interesting and almost changed my mind about our best friends (or nightmare)… logs!
When I’m talking about log management with customers, I always insist on the fact to work with “business cases“, for two major reasons. The first one is purely technical: to avoid a constant flood of events and to be able to process correctly the important ones. The second one is more focused on the business: to get a quick return on investment! Log management is often expensive and management will be happy if they get “something to eat” as soon as possible. Business cases can be related to internal requirements or to compliance requirements. PCI-DSS is a good example. Here are some reports that must be produced:
- IPS reports
- User accesses on devices
- Account changes
- Configuration changes
There is a big side effect with compliance requirements: organizations tend to do the strict minimum to be “compliant“. Still today, some companies see this as a stamp on a paper. Fail! Let’s put the “business” aside and let’s view the log management as a tool to perform incident handling or forensics researches. In this case, it will be more efficient if we would investigate across a broader amount of data. Sometimes investigations occur a long time after the incident. Incident handlers are playing like the Cold Case characters, trying to find who did what, when and where! And evidences could be located at so many places! Let’s take an IP address. We can find them in firewall, IDS or proxy logs but also:
- In SMTP headers
- In Excel sheets (low cost IPAM solution but commonly used
- In configuration files
To increase the chances to find relevant information, the log management solution must collect and index as much data as possible. It’s the opposite of the compliance requirements. Do I change sides? Not completely but I think that both points of view must be taken into account with their pros & cons.
Howto: Distributed Splunk Architecture
Implementing a good log management solution is not an easy task! If your organisation decides (should I add “finally“?) to deploy “tools” to manage your huge amount of logs, it’s a very good step forward but it must be properly addressed. Devices and applications have plenty of ways to generate logs. They could send SNMP traps, Syslog messages, write in a flat file, write in a SQL database or even send smoke signals (thanks to our best friends the developers). It’s definitively not an out-of-the-box solution that must be deployed. Please, do NOT trust $VENDORS who argue that their killing-top-notch-solution will be installed in a few days and collect everything for you! Before trying to extract the gold of your logs, you must correctly collect events. This mean first of all: do not loose some of them. It’s a good opportunity to remind the Murphy’s laws here: The lost event will always be the one which contained the most critical piece of information! In most cases, a log management solution will be installed on top of an existing architecture. This involves several constraints:
- From a security point of view, firewalls will for sure block flows used by the tools. Their policy must be adapted. The same applies to the applications or devices.
- From a performance point of view, the tools can’t have a negative impact on the “business” traffic.
- From a compliance point of view, the events must be properly handled in respect of the confidentiality, integrity and availability (you know the well-know CIA principle).
- From a human point of view (maybe the most important), you will have to fight with other teams and ask them to change the way they work. Be social!
To achieve those requirements, or at least trying to reach them, your tools must be deployed in a distributed architecture. By “distributed“, I mean using multiple software componants desployed in multiple places in your infrastructure. The primary reason for this is to collect the events as close as possible to their original source. If you do this, you will be able to respect the CIA principle and:
- To control the resources usage to process them and centralise them
- To get rid of proprietary or open multiple protocols
- To control the good processing of them from A to Z.
For those who are regular readers of my blog, you know that I’m a big fan of OSSEC. This solution implements a distributed architecture with agents installed on multiple collection points to grab and centralise the logs:
OSSEC is great but lack of a good web interface to search for events and generate reports. Lot of people interconnect their OSSEC server with a Splunk instance. There is a very good integration of both products using a dedicated Splunk app. Usually, Splunk is deployed on the OSSEC server itself. The classic way to let Splunk collect OSSEC events is to configure a new Syslog destination for alerts like this (in your ossec.conf file):
<syslog_output> <server>10.10.10.10</server> <port>10001</port> </syslog_output>
This configuration blog will send alerts (only!) to Splunk via Syslog messages sent to 10.10.10.10:10001 (where Splunk will listen for them). Note that the latest OSSEC version (2.7) can write native Splunk events over UDP. Personally, I don’t like this way of forwarding events because UDP remains unreliable and only OSSEC alerts are forwarded. I prefer to process the OSSEC files using the file monitor feature of Splunk:
[monitor:///data/ossec/logs] whitelist=\.log$
But what if you have multiple OSSEC server across multiple locations? Splunk has also a solution for this called the “Universal Forwarder“. Basically, this is a light Splunk instance which is installed without any console. This goal is just to collect events in the native format and forward them to a central Splunk instance (the “Indexer“):
If you have experience with ArcSight products, you can compare the Splunk Indexer with the ArcSight Logger and the Universal Forwarder with the SmartConnector. The configuration is pretty straight forward. Let’s assume that you already have a Splunk server running. In your $SPLUNK_HOME/etc/system/local/inputs.conf, create a new input:
[splunktcp-ssl:10002] disabled = false sourcetype = tcp-10002 queue = indexQueue [SSL] password = xxxxxxxx rootCA = $SPLUNK_HOME/etc/auth/cacert.pem serverCert = $SPLUNK_HOME/etc/auth/server.pem
Restart Splunk and it will now bind to port 10002 and wait for incoming traffic. Note that you can use the provided certificate or use your own. It’s of course recommended to encrypt the traffic over SSL! Now install an Universal Forwarder. Like the regular Splunk, packages are available for most modern OS. Let’s play with Ubuntu:
# dpkg -i splunkforwarder-5.0.1-143156-linux-2.6-intel.deb
Configuration can be achieved via the command line but it’s very easy to do it directly by editing the *.conf files. Configure your Indexer in the $SPLUNK_HOME/etc/system/local/outputs.conf:
[tcpout] defaultGroup = splunkssl [tcpout:splunkssl] server = splunk.index.tld:10003 sslVerifyServerCert = false sslCertPath = $SPLUNK_HOME/etc/auth/server.pem sslPassword = xxxxxxxx sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
The Universal Forwarder inputs.conf file is a normal one. Just define all your sources there and start the process. It will start forwarding all the collected events to the forwarder. This is a quick example which demonstrate how to improve your log collection process. The Universal Forwarder will take care of the collected events and send them safely to your central Splunk instance (compressed, encrypted) and will queue them in case of outage.
A final note, don’t ask me to compare Splunk, OSSEC or ArcSight. I’m not promoting a tool. I just gave you an example of how to deploy a tool, whatever your choice is
Attackers Geolocation in OSSEC
If you follow my blog on a regularly basis, you probably already know that I’m a big fan of OSSEC. I’m using it to monitor all my personal systems (servers, labs, websites, etc). Being a day-to-day user, I have always new ideas to extend the product , by using 3rd party tools or by adding features. One of the missing feature (at least for me), is the lack of information when an alert is generated. Tracking the attackers source IP addresses is very nice. Example: OSSEC can trigger active-response scripts to blacklist them during a short period but how can we benefit of more “visibility” with those addresses. When you think how to give more visibility to IP addresses, you immediately think: Geolocation! I already posted an article about the power of geolocation (link) to map alerts onto a Google map (example: a brute-force attack). This is very interesting but required manual actions. As IP addresses are already known by OSSEC (saved in the variable “srcip“), why not let OSSEC do the job for us in real time?
The problem solved by the geolocation is the following: How to convert IP addresses in coordinates (longitude/latitude) first and then map them to a country and/or city? MaxMind has the solution. This company maintains a database of all the assigned IP addresses (Over 99.5% on a country level and 78% on a city level for the US within a 40 kilometer radius – source: Maxmind) with their mapping to geographic locations. Note that they provide two databases: IPv4 and IPv6. They propose API‘s for several languages like C. Good news OSSEC is written in C! So, I wrote a patch which perform a geolocation of the ‘srcip’ attackers to display the location of the IP address. Let’s go…
Step 1: Install the MaxMind GeoIP API
This is easy as any other open source tool/library. Download the source code and install it. It should not a problem on classic Linux flavors.
# wget http://www.maxmind.com/download/geoip/api/c/GeoIP-1.4.8.tar.gz # tar xzvf GeoIP-1.4.8.tar.gz # cd GeoIP-1.4.8 # ./configure # make # make install
By default, everything will be installed under /usr/local but you are free to change the multiple paths via the configure script. The API provides C include files and (dynamic/static) libraries, very common.
Step 2: Install or recompile OSSEC with GeoIP localization
If you have already an OSSEC running, you could simply apply my patch to the original source tree. Take care if you already performed personal changes! I created the patch from a standard 2.6 tarball. To enable the GeoIP feature, you have to enable it (like the MySQL support).
# wget http://www.ossec.net/files/ossec-hids-2.6.tar.gz # tar xzvf ossec-hids-2.6.tar.gz # wget http://blog.rootshell.be/wp-content/uploads/2012/06/ossec-geoip.patch # cd ossec-hids-2.6 # patch -p1 < ../ossec-geoip.patch # cd src # make setgeoip # cd .. # ./install.sh
Step 3: Install the MaxMing GeoIP DBs
MaxMind provides different versions of the databases. I’m using the GeoCityLite. This is the free version which provides precision up to the city. This is far enough for me. Databases are not provided with the API, they must be installed manually:
# cd /var/ossec/etc # wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz # gzip -d GeoLiteCity.dat.gz # wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz # gzip -d -c - >GeoLiteCityv6.dat.gz
I suggest you to download both databases (v4 & v6) and don’t forget that databases are regularly updated! It’s interesting to setup a small cron to install new versions to keep a good accuracy in the results.
Step 4: Fix the OSSEC configuration files
Once the patch installed, new parameters are available to set the GeoIP environment. First, in the ossec.conf “global” section, define the path of your databases.
<global> <geoip_db_path>/etc/GeoLiteCity.dat</geoip_db_path> <geoip6_db_path>/etc/GeoLiteCityv6.dat</geoip6_db_path> </global>
In the “alerts” section, activate the GeoIP feature:
<alerts> <use_geoip>yes</use_geoip> </alerts>
Finally, in the internal_options.conf, enable (or disable) the display of GeoIP information in the notification emails:
# Maild GeoIP support (0=disabled, 1=enabled) maild.geoip=1
How does it work? The OSSEC process which performs the GeoIP lookups is “ossec-analysisd“. When an alert must be logged and if a “srcip” has been decoded, the IP address is passed to a new function GeoIPLookup() which calls the MaxMind API and returns a string with the geolocation data. The data is added to the alert text. The second component which has been patched is “ossec-maild” which parses the alerts and send emails also with the GeoIP data (if enabled).
During the configuration, don’t forget that “ossec-analysisd” runs chrooted (in the main OSSEC directory). Don’t forget to adapt the path to the GeoIP databases! (they must be defined relative to the chroot environment).
Here follow some examples of alerts with GeoIP data enabled:
** Alert 1338899194.500996: - apache,invalid_request, 2012 Jun 05 14:26:34 (xxx) x.x.x.x->/var/log/apache/error_log Rule: 30115 (level 5) -> 'Invalid URI (bad client request).' Src IP: 94.45.169.133 Src Location: RU,Moscow City,Moscow [Tue Jun 05 14:26:34 2012] [error] [client 94.45.169.133] [deleted] ** Alert 1338901319.507426: - syslog,postfix,spam, 2012 Jun 05 15:01:59 (xxx) x.x.x.x->/var/log/syslog Rule: 3303 (level 5) -> 'Sender domain is not found (450: Requested mail action not taken).' Src IP: 188.142.66.168 Src Location: NL,Zuid-Holland,Alphen Jun 5 15:01:43 xxx postfix/smtpd[7397]: NOQUEUE: reject: [deleted] Received From: (xxx) x.x.x.x->/var/log/apache/access_log Rule: 100106 fired (level 15) -> "PHP CGI-bin vulnerability attempt." Src Location: IL,Tel Aviv,Tel Aviv-yafo Portion of the log(s): 77.126.209.178 - - [05/Jun/2012:16:42:07 +0200] [deleted]
Note that GeoIP lookups will be successful only for alerts which have a valid “srcip” field! In all case, the returned location will be “(null)“! What about the impact on performances? I’m using this patch in production for a few days and I did not notice any performance degradation. The GeoIP lookup is performed only once and parsed later by ossec_maild. (I’ve an average of 2000 alerts logged per day).
My OSSEC server is compiled with the support for MySQL. I did not detect any incompatibility between MySQL and my patch. At this time, the geolocation data are not sent to the MySQL alert table. But this could be done easily: Storing the latitude/longitude could be helpful to map attacks in real time using a 3rd party tool.
My patch is available here. Feel free to download it, use it and maybe improve it. Comments and suggestions will be appreciated. Final disclaimer, I won’t be responsible of you break your current OSSEC setup…
Monitor your Monitoring Tools
We (and I’m fully part of it) deploy and use plenty of security monitoring tools daily. As our beloved data is often spread across complex infrastructures or simply across multiple physical locations, we have to collect interesting information and bring them in a central place for further analysis. That’s called “log management“. Based on your collected events, you can generate alerts, build reports. Nice! But… if systems and applications generate [hundreds|thousands|millions] of events, those ones are processed by the same kind of hardware running some piece of software. Hardware may fail (network outage, power outage, disk crash) and softwares have bugs (plenty of).
Ranking People Like Domains or IP Addresses?
Real time events or network traffic analysis is interesting to track suspicious behaviors. And, if you add some external sources of information, you could increase even more the capability of detecting real events. Such ranking sources applies usually to IP addresses and domain names. They are plenty of online resources with huge lists of suspicious IP’s/domains (a good starting list is available here). You can of course create and maintain your own private lists. But can we implement the same ranking with “people” (humans)?
Tracking Tweets in your SIEM
My previous post about monitoring pastebin.com within your SIEM gave ideas to some of you. On aftershell.com, you will find a fork of my script to monitor WordPress RSS feeds. If pastebin.com may contain relevant piece of information as well as blogs, do not underestimate the value of social networks! They are plenty of them: Google+, LinkedIn, Twitter, Facebook, etc. Let’s focus on Twitter.
Monitoring pastebin.com within your SIEM
(Source: pastebin.com)
For those who (still) don’t know pastebin.com, it’s a website mainly for developers. Its purpose is very simple: You can “paste” text on the website to share it with other developers, friends, etc. You paste it, optionally define an expiration date, if it’s public or private data and your are good. But for a while, this on-line service is more and more used to post “sensitive” information like passwords or emails lists. By “sensitive“, I mean “stolen” or “leaked” data. Indeed, pastebin.com allows anybody to use their services without any authentication, it’s easy to remain completely anonymous (if you submit data via proxy chains, Tor or any other tool which takes care of your privacy)
In big organizations, marketing departments or agencies learned how to use social networks for a long time. They can follow what has been said about their products and marketing campaigns. In my opinion, it is equally important to follow what’s posted about your organization on pastebin.com! Many people are looking for interesting data on pastebin.com from an offensive point of view. Let’s see how this can also benefit to the defensive side.
For me, pastebin.com became an important source of information and I keep an eye on it every day. But, due to the huge amount of information posted every minute, it is impossible to process it manually. Of course, you can search for some keywords but it’s totally inefficient. In a first time, I grabbed and processed some HTML content using the classic UNIX tools. Later, I found a nice Python script developed by Xavier Garcia: python.py. It checks continuously for data leaks on pastebin.com using regular expressions. I kept it running for a while on a Linux box and it did a quite good job but I needed more! Xavier’s script send the found “pasties” on the console. It is possible to dump the detected pasties by sending a signal to the process. Not always easy. That’s why I decided to go a step further and write my own script! The principle remains the same as the script in Python (why re-invent the wheel?) but I added two features that I found interesting:
- It must run as a daemon (fully detached from the console) and started at boot time.
- It must write its finding in a log file.
The next step sounds logical: If you have a log file, why not process it automatically: Let’s monitor pastebin.com within your SIEM! If you find information posted on pastebin.com, it could be very interesting to be notified (a great added-value for your DLP processes). My script generates Syslog messages and (optionally) CEF (“Common Event Format“) events which can be processed directly by an ArcSight infrastructure. Syslog messages can be processed by any SIEM or log management solution like OSSEC (see below). It is now possible to completely automate the process of detecting potentially sensitive leaked data and to generate alerts on specific conditions.
First install the script on a Linux machine. Requirements are light: a Perl interpreter with a few modules are required (normally all of them are already installed on recent distribution) and a web connectivity to http://pastebin.com:80. If you are behind a proxy, you can define the following environment variable, it will be used by the script:
# export HTTP_PROXY=http://proxy.company.com:8080
The script can be started with some useful options:
Usage: ./pastemon.pl --regex=filepath [--facility=daemon ] [--ignore-case][--debug] [--help]
[--cef-destination=fqdn|ip] [--cef-port=<1-65535>] [--cef-severity=<1-10>]
Where:
--cef-destination : Send CEF events to the specified destination (ArcSight)
--cef-port : UDP port used by the CEF receiver (default: 514)
--cef-severity : Generate CEF events with the very easy to process and can be specified priority
(default: 3)
--debug : Enable debug mode (verbose - do not detach)
--facility : Syslog facility to send events to (default: daemon)
--help : What you're reading now.
--ignore-case : Perform case insensitive search
--regex : Configuration file with regular expressions (send SIGUSR1 to reload)
Once running, the script scans for newly uploaded pasties and search for interesting content using regular expressions. There is no limitation on the number of regular expressions (defined in a text file). To not disturb pastebin.com webmasters, the script waits a random number of seconds between each GET requests (between 1 and 5 seconds). There is only one mandatory parameter ‘–regex‘ which gives the text files with all the regular expressions to use (one per line). If one of the regular expressions matches, the following information will be sent to the local Syslog daemon:
Jan 16 14:43:24 lab1 pastemon.pl[29947]: Sending CEF events to 127.0.0.1:514 (severity 10) Jan 16 14:43:24 lab1 pastemon.pl[29947]: Loaded 17 regular expressions from /data/src/pastemon/pastemon.conf Jan 16 14:43:24 lab1 pastemon.pl[29947]: Running with PID 29948 <time flies> Jan 16 15:57:48 lab1 pastemon.pl[29948]: Found in http://pastebin.com/raw.php?i=hXYg93Qy : CREATE TABLE (9 times) -- phpMyAdmin SQL Dump (1 times)
All matching regular expressions are listed with their number of occurrences. This can be easily processed by OSSEC using the following decoder:
<decoder name="pastemon"> <program_name>^pastemon.pl</program_name> </decoder> <decoder name="pastemon-alert"> <parent>pastemon</parent> <regex>Found in http://pastebin.com/raw.php?i=\.+ : (\.+) \(</regex> <order>data</order> </decoder>
The first regular expression is stored in the OSSEC “data” variable to be used as conditions in rules. Here is an example: The rule #100203 will trigger an alert if some yahoo.com email addresses are leaked in pastebin.com. (Note: This regular expression must be defined in the script configuration file!)
<rule id="100203" level="0"> <decoded_as>pastemon</decoded_as> <description>Data found on pastebin.com.</description> </rule> <rule id="100204" level="7"> <if_sid>100203</if_sid> <description>Detected yahoo.com email addresses on pastebin.com!</description> <extra_data>@yahoo\.com$</extra_data> </rule>
If you have an ArcSight infrastructure, you can enable the CEF events support. The same event as above will be sent to the configured CEF destination and port:
<29>Jan 16 15:57:48 CEF:0|blog.rootshell.be|pastemon.pl|v1.0|regex-match|One or more regex matched|10|request=http://pastebin.com/raw.php?i=hXYg93Qy destinationDnsDomain=pastebin.com msg=Interesting data has been found on pastebin.com. cs0=CREATE TABLE cs0Label=Regex0Name cn0=9 cn0Label=Regex0Count cs1=-- phpMyAdmin SQL Dump cs1Label=Regex1Name cn1=1 cn1Label=Regex1Count
To process the CEF events on ArcSight’s side, configure a new SmartConnector, a new UDP CEF receiver and the events should be correctly parsed:
(Click to enlarge)
That looks great! But the next question is: “What to look for on pastebin.com?“. Well, it depends on you… Based on your organization or business, there are things that you can’t miss. Here is a list of useful regular expressions that I often use:
RegEx Purpose
--------------------------------------------------------------------- -----------------------------------
company\.com Your company domain name
@company\.com Corporate e-mail addresses
CompanyName Company name
MyFirstName MyLastName Your full name
@xme Twitter account
192.168.[1-3].[0-255] IP addresses ranges
anonbelgium Hackers groups
#lulz Trending Twitter hashtags
#anonymous
#antisec
-----BEGIN RSA PRIVATE KEY----- Interesting data!
-----BEGIN DSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
-- MySQL dump Interesting dumps!
belgium My country
city My city
((4\d{3})|(5[1-5]\d{2})|(6011))-?\d{4}-?\d{4}-?\d{4}|3[4,7]\d{13} Credit cards
If you have interesting regular expressions or ideas, feel free to share!
Source is available here. As usual, this is provided “as is” without any warranty. Happy monitoring!
From Logs to Hell!
I have hesitated a while before choosing the right image to illustrate this article. I read again a press-release about a new log-management product which pretends to provide “out-of-the-box security and compliance for business of all sizes“. Dear v€ndor, are you living in a care bears world or are you possessed?
Yes, your solution has maybe nice reports ready to be produced just by clicking a “Generate now” button. You allow searching across millions of events in seconds? Why not… But the power and reliability of a log management solution directly depend on how you feed it with events! Lack of events will produce poorly reports… then poor ROI!
Before deploying your top-notch log management solution, there are so many issues that could occur:
- Unreachable devices – They could be located on remote sites with limited bandwidth. Some firewalls might prevent monitoring protocols (which are often and hopefully blocked). Are they using private IP addresses with NAT rules?
- Supported format – Your devices might generate events in a unsupported format.
- Performance impacts on the network flows
- (De)commissioning of (old)devices
- Overlapping in IP subnets
- Procedures / follow-up
- …
Still today, most log management solutions are deployed urgently to face a need of compliance. For me, v€ndor$ pretending to provide “out-of-the-box” log management services could be blamed of “false advertising“! Implementing a log management solution is not a road without pitfalls. It could quickly drive you to hell…
OSSEC Speaks “ArcSight”
Log management… A hot topic! There are plenty of solutions to manage your logs. Like in all IT domains, there are two major categories: free and commercial tools. Both have pro and cons. No big debate here, contrariwise I’ll show you a good example of a mix between both worlds. Let’s take two products: OSSEC & ArcSight. OSSEC is a free log management solution. If you follow my blog, you know that it’s one of my favorite toy at the moment. But I’m also involved in ArcSight projects. It’s a very robust commercial log management / SIEM solution (let’s stop the promotion here).
One of the main issue in log management is the way of handling events. All of them are generated by multiple devices or applications in several formats. Once collected, those events must be parsed and normalized. ArcSight developed a specific format called CEF (“Common Event Format“). It is described on their website as following:
“The Common Event Format (CEF) is an open log management standard that improves the interoperability of security-related information from different security and network devices and applications. CEF is based on expertise from building support for over 275 products across more than 35 solution categories and is the first log management standard to support a broad range of device types. CEF enables technology companies and customers to use a common event log format so that data can easily be collected and aggregated for analysis by an enterprise management system.“
More information is available here if you are interested. Honestly, there are not so many products which support CEF today. But it looks that it’s changing and big players started to implement the generation of native CEF events in their products. What makes CEF important? By generating CEF events, you remove an important step in the log management process: the normalization. CEF events already contains all the important information. Basically, a CEF event is based on the following structure:
CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|\ Name|Severity|Extension
The field names explain by themselves their meaning. The last one (“Extension“) is used to pass extra information. This field contains a collection of key=value pairs.
Example:
CEF:0|ArcSight|Logger|5.0.0.5355.2|sensor:115|Logger Internal Event|1|\ cat=/Monitor/Sensor/Fan5 cs2=Current Value cnt=1 dvc=10.0.0.1 cs3=Ok \ cs1=null type=0 cs1Label=unit rt=1305034099211 cs3Label=Status cn1Label=value \ cs2Label=timeframe
Let’s go back to OSSEC. This software can grab very easily events from multiple devices and applications, process them and generate useful alerts. OSSEC has also an interesting “remote syslog” feature to forward alerts to a remote Syslog server. Why not let OSSEC generate CEF events by itself? ArcSight could be used to centralize logs coming from multiple sources and agents as well as OSSEC. OSSEC is very light and free, why not integrate it with ArcSight? A winning team!
I wrote a small patch to add CEF support to the csyslogd daemon. This one is responsible of sending alerts to our remote Syslog server. My patch adds a new format that can be specified in your ossec.conf file:
<syslog_output>
<server>10.0.0.1</server>
<port>514</port>
<format>cef</format>
</syslog_output>
Once you applied the patch, recompile the os_csyslogd module, adapt your ossec.conf and restart OSSEC! Alerts will now be sent in CEF format. The following information is mapped into the new event:
- Vendor (Trend Micro Inc.)
- Product (OSSEC HIDS)
- Version (v2.5.1)
- Rule ID
- Rule Name
- Severity
- Message (which will contains the “srcip” and “user” fields)
Example:
CEF:0|Trend Micro Inc.|OSSEC HIDS|v2.5.1|5302|\ User missed the password to change UID to root.|9|dvc=ubuntusvr \ cs2=ubuntusvr->/var/log/auth.log cs2Label=Location src= suser=root \ msg=May 11 21:16:05 ubuntusvr su[24120]: - /dev/pts/1 xavier:root
If you specify in the <syslog_output> XML block the IP address of your ArcSight device (properly configured to receive CEF events over UDP), they will be smoothly indexed and stored in the database:
(Click to enlarge)
But, will you ask why implement this? Here are some scenarios:
- OSSEC integrates nice features like a rootkit detection and FIM (“File Integrity Monitoring“) but not ArcSight. This could add more value to your SIEM.
- During a transition period between the two products (and avoid multiple agents on your servers)
- To use the powerful ArcSight correlation engine without deploying a huge platform (Log archiving remains at OSSEC level)
- OSSEC Decoders are much easier to write and deploy compared to the ArcSight FlexConnector s(which also require a developer license !)
- Etc…
Note that this patch does not implement the classification of the OSSEC alerts using the ArcSight mechanism (via the field “categorySignificance“). It’s just a dumb rewrite of the alerts using a CEF pattern. If you have ideas or comments, they are welcome!
My patch is available here and, as usual, is available for free and without any warranty.
