Importing Secunia Advisories into a SIEM/OSSEC

Secunia is a security company which, amongst other activities, maintains a huge database of vulnerabilities. On their website, they describes their business like this:

“Secunia collects, evaluates, verifies, and analyses security information. This security information is available through our databases and is distributed to our customers, segmented according to their specific business needs.“.

Their vulnerability database is available for all visitors for free. They also offer paying business services like VIF (“Vulnerability Intelligence Feed“) and some information are available only for the subscribed customers. The Secunia website is one of my reference site to find vulnerabilities and the free version offers enough useful information to be kept up-to-date.

But we are all annoyed by recurrent tasks: A daily visit to Secunia is so boring! (They don’t provide RSS feed for free anymore since the beginning of 2009) Why not get an overview of the new vulnerabilities in our internal monitoring tools? A SIEM (or a Log Management tool) is able to collect and normalize events from non-heterogeneous sources. Why not collect the Secunia advisories and store them in the SIEM events database? The “S” means “Security”, an updated feed of advisories could certainly by a plus:

• Reports could be generated to provide a list of advisories for a given period of time.
• Matching the devices names & types with the vulnerabilities could increase the classification of assets based on the vulnerabilities (severity).
• Real-time alerts based on specific vulnerabilities.

Thanks to the original script of a colleague (Thanks Bram!), I developed a new Perl script which stores the Secunia vulnerabilities into a MySQL database and, optionally, to a flat file. [Note: between the first version of this article and the publication, Secunia changed the HTML presentation of the vulnerabilities. I updated the script but more changes may occur in the future!] Many SIEM products could easily read files from a file system and “learn” the format to create normalized events. The events flow will look like:

The Perl script requires the following modules:

     LWP::Simple
     DateTime::Format::Strptime
     DBI
     HTML::TreeBuilder
     HTML::Element
     HTML::TableContentParser

At the beginning of the script, the MySQL information can be specified to match your environment (user, pass, dbname, etc). A small database must be created with only one table. The schema is:

mysql> describe advisory;
+------------+--------------+------+-----+---------+-------+
| Field      | Type         | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| id         | int(11)      | NO   | PRI | NULL    |       |
| date       | date         | NO   |     | NULL    |       |
| severity   | varchar(20)  | NO   |     | NULL    |       |
| product    | varchar(100) | YES  |     | NULL    |       |
| impact     | varchar(100) | YES  |     | NULL    |       |
| source     | varchar(100) | YES  |     | NULL    |       |
| descr      | varchar(256) | YES  |     | NULL    |       |
| url        | varchar(256) | YES  |     | NULL    |       |
| popularity | int(5)       | YES  |     | NULL    |       |
+------------+--------------+------+-----+---------+-------+
9 rows in set (0.01 sec)

mysql>

The Perl script accepts the following arguments:

./secunia_parse.pl -h
Usage: secunia_parse.pl [-h] [-d] [-o outputfile] [-a] [-s separator ]
                       [-m yyyy-mm-dd]
Where:
        -h              : Display command line usage.
        -d              : Enable debug mode (verbose output).
        -o outputfile   : Write advisories to a file (1/line).
        -a              : Append to the outfile file if already exists.
        -s separator    : Field separator for the logfile (default: "|")
        -m yyyy-mm-dd   : Parse advisories from today to the given date.

By default, the script fetches all new advisories if not yet present in the DB. If an advisory is already present, it will be updated (example: if the critical level has been raised or lowered). To prevent a flood of requests, only the first 100 pages of the Secunia history will be parsed. The “-m” flag is useful to force a fetch of advisories from the given date up to today’s date.

The script saves the vulnerabilities in the MySQL DB. By using the flag “-o” we can create (or append) the advisories to a flat file. The format will be:

38280!20100121!Less critical!Drupal Control Panel Module 6.x!\
  Cross Site Scripting!From remote!Drupal Control Panel Module Script\
  Insertion Vulnerability!http://secunia.com/advisories/38280!164

Important disclaimer: My Perl script has been developed for internal usage only. I couldn’t be held responsible for any mis-usage. As mentioned on the Secunia website, all information available on the website is copyrighted:

“Terms & Conditions for usage of the Web site and content

These terms and conditions apply to anyone using Secunia, visitors as well as customers.

Distribution, publishing, or reproduction of any information from this site is strictly forbidden without the written permission from Secunia. All content is copyrighted by Secunia. You may however use Secunia as a reference, as long as you clearly state Secunia as the source of information and link to the specific information at Secunia.”

Once the vulnerabilities log file is populated on a regular basis (an hourly crontab entry should be enough), your log management solution is able read the new events. Here follow two examples using different tools available (one free and one commercial).

ArcSight has a catalogue of powerful solutions to build a SIEM environment. These tools are able to learn unknown file formats using “FlexConnectors”. I wrote my own FlexConnector which parses the Secunia events and converts them into CEF (“Common Event Format”), the standard event format developed by ArcSight. Once collected and indexed, it’s possible to use the Secunia events like any other event type like showed on the picture below.

If you’re interested, the FlexConnector is available here. It is provided “as is” without any warranty.

The second example is based on OSSEC. This open source tool has many interesting features. It’s not only an HIDS (“Host based Intrusion Detection System“), it can also monitor the activity of a system by reading log files. Based on patterns, OSSEC is able to take actions (basically an alert via e-mail or Syslog, or an triggered script). If you want to learn more about the OSSEC basics, I recommend you to have a look at the presentation of Wim Remes performed during the last FOSDEM.

To parse the Secunia event, the configuration is straight forward (based on a standard OSSEC installation). Configure OSSEC to process a new log file in your ossec.conf file:

<localfile>
    <log_format>syslog</log_format>
    <location>/var/log/secunia_parse.log</location>
</localfile>

New alerts are defined in your local_rules.xml file (tip: add your customizations in this file only. It will remain untouched in case of an OSSEC upgrade). The firs rules will match a “highly critical” vulnerability and the second one will match a “highly critical” vulnerability affecting the Red Hat Linux distribution:

<rule id="100001" level="8">
    <description>Highly critical Secunia event</description>
    <decoded_as>secunia-parse</decoded_as>
    <status>Highly Critical</status>
</rule>

<rule id="100002" level="9">
    <description>Highly critical Red Hat vulnerability</description>
    <if_sid>100001</if_sid>
    <match>Red Hat</match>
</rule>

Example of alert generated by OSSEC when the rules above are matched:

** Alert 1266240348.11157: - local,syslog,
2010 Feb 15 14:25:48 zeroday->/var/tmp/secunia_parse.log
Rule: 100002 (level 6) -> 'Highly critical vulnerability affecting Debian'
Src IP: (none)
User: (none)
38607|20100215|Highly critical|Red Hat Enterprise Linux AS 3,Red Hat Enterprise Linux AS 4,Red Hat Enterprise Linux Desktop 5,Red Hat Enterprise Linux ES 3,Red Hat Enterprise Linux ES 4,Red Hat Enterprise Linux WS 3,Red Hat Enterprise Linux WS 4,RHEL Desktop Workstation 5RHEL Optional Productivity Applications (v. 5 server)|System access|From remote|Red Hat update for openoffice.org|http://secunia.com/advisories/38607|88

It’s easy to get notified when vulnerabilities are detected for your favorite applications and operating systems!

Deploying and maintaining a SIEM is a complex and recurrent process. Feeding your SIEM with extra sources like Secunia could greatly increase the visibility required to maintain your infrastructure at the highest security level.