Logs: For Better or For Worse?

Last week, a vulnerability regarding Apache was disclosed. More precisely, the issue was located on the mod_rewrite module. This module rewrites (now, you understand its name) URLs on the fly. This is very useful during web page migrations, attacks migitations etc. The security flaw does not affect the core feature of this module but the way it writes events in the Apache log file. The output is not properly sanitized and could allow the execution of arbitrary commands if the logs are displayed in a terminal emulator. This is not the first time that log files are vulnerable to command injections. Another product affected by this issue was Fail2ban in 2012 (CVE-2012-5642).

This is not a brand new type of attack. For a while OWASP is considering log injection as a real thread and has a page about this. Developers are asked to generate events and write log files. Why? Because it’s often the first place where incidents can be detected and investigated. Logs can also be interesting to generate statistics to help debugging code, etc. And also for compliance or governance reasons! (don’t forget the business). The issue with our beloved logs is that they are trusted: They are generated by devices or applications that we trust. We collect them from decentralized sources (that we can’t trust!) and allow them to pass through firewalls without any inspection. They are even encrypted!

What kind of thread can affect log files? Here follow some examples:

• New line injection
• Separator injection
• Timestamp injection
• HTML injection
• Terminal charactoers injection
• Buffer overflow
• SQL injection
• Command injection
• Format strings

I’m just wondering: If it is proven that events generated by an application can be tampered by the attacker, does the log file remain a valid evidence? A valid log management solution takes care of the events consistency when it has been created. It helps to detect any alteration of the event during the process (data in transit, data at rest) but what if the event is forged at the source? (“data at creation“) Just think about it!

The next question which will arise is: “Where to apply the sanitization of events? At the application level or at log management level?“. My opinion is to address the problem as close as possible of the source (read: at application level). Indeed, events could be processed by a multitude of scripts, command lines and languages starting from a toolbox based on tail/grep/sed/awk to top-notch SIEM solutions.