[This blogpost has been published as a guest diary on isc.sans.org]
Handling log files is not a new topic. For a long time, people should know that taking care of your logs is a must have. They are very valuable when you need to investigate an incident. But, if collecting events and storing them for later processing is one point, events must be properly generated to be able to investigate suspicious activities! Let’s take by example a firewall… Logging all the accepted traffic is one step but what’s really important is to log all the rejected traffic. Most of the modern security devices (IDS, firewalls, web application firewalls, …) can integrate dynamic blacklists maintained by external organizations. They are plenty of usefull blacklists on the internet with IP addresses, domain names, etc… It’s quite easy to add a rule on top of your security policy which says:
if (source_ip in blacklist): Â Â drop_traffic()
With the “blacklist” table being populated by an external process. Usually, this rule is defined at the beginning of the security policy for performance reason. Very efficient, but is it the right place?
Let’s assume a web application firewall which has this kind of feature. It will drop all connections from a (reported as) suspicious IP address from the beginning without more details. Let’s put the blacklist rule at the end of the policy of our WAF. We have now something like this:
if (detected_attack(pattern1)):   drop_traffic() elif (detected_attack(pattern2)):   drop_traffic() elif (detected_attack(pattern3)):  drop_traffic() elif  (source_ip in blacklist):  drop_traffic()
If we block the malicious IP addresses at the beginning of the policy, we’ll never know which kind of attack has been tried. By blocking our malicious IP addresses at the end, we know that if one IP is blocked, our policy was not effective enough to block the attack! Maybe a new type of attack was tried and we need to add a new pattern. Blocking attackers is good but it’s more valuable to know why they were blocked…
Tx for the comment! That’s indeed one of the issues with blacklists… The risk of DoS’ing yourself our your valid users/customers.
completely agree with your thoughts around block blacklist at the end. One caveat i think would benefit others, especially those users new to or recently adopting a threat intel feed or blacklist ip feed is that all are not created equal. the decision to block based solely on that is only a good decision after you have vetted your feed (or feeds) and have a good feel that the blocks would not impact valid users coming from aged ips associated with old/stale IOCs or blacklists. Love your blog, thanks for always having good nuggets of info here and there.
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
Depending on the traffic volume and (network)architecture you can do
if (source_ip in blacklist):
redirect_to_honeypot(source_ip)
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe
RT @xme: [/dev/random] The Art of Logging http://t.co/mq5gMgqsPe