Pwned or not Pwned?

Just before the announce of the Full-Disclosure shutdown a few days ago, a thread generated a lot of traffic and finally turned into a small flame war. In the beginning of the month, a security researcher reported a vulnerability found on Youtube. According to him, the Google service was suffering of a file upload vulnerability. Reading such kind of post is juicy! Accepting files sent by visitors is always a touchy feature on a website. By example, if you allow your users to upload images to create an avatar, you must implement proper controls to be sure that the uploaded file is in the correct format and does not contain any malicious code. I won’t describe how to protect against this vulnerability and even less discuss about the Full-Disclosure thread but it reveal an important fact: the severity of an issue is linked to its “context“…

When you are mandated by a customer to perform a pentest against his infrastructure, you always have two parties which have different expectations. The customer hopes that nothing critical will be found and the pentester expects interesting stuff. In most cases, a pentest is based on highly technical tests and requires very specific tools and procedures. Once the fun is over, it’s time for the boring homework: to write the report. Until today, I never met a pentester who likes writing reports! Unfortunately, there is not “translator” to explain technical findings in common (English|French|Duch|German) words (choose your best language). We are all dreaming of an interface like this one:

From a pentester point of view, It’s tempting to be cynic and to point to the customer: “Ah ah! You have a XSS on your homepage! Fail!“. First of all this is not professional. Don’t forget that you’re in a customer-contractor relationship. The customer don’t expect a blame but some inputs to improve the overall security. It’s not the pentester job to provide a turnkey solution to fix an issue but it must be properly reported with a valid severity. Again, I won’t discuss the severity of a XSS vulnerability because the context is a key element here. When I report an issue in my report, there is of course a complete technical description of the issue, how to reproduce it and a PoC (proof-of-concept) which can be a screenshot, a password, a DB dump, … Then the vulnerability must be placed in the context of the customer’s business and properly rated. The final goal isn’t to stress the customer.

The rating scheme I’m using adds the following flags to findings:

• A likelihood which is based on the exploitability, the reproductibility, the discoverability and the frequency.
• A consequence (like minor, moderate, major or catastrophic)
• A risk based on the two first indicators (Informative, low, medium or high)

Some examples? A likelihood of “rare” is assigned to an issue which requires highly skilled and determined attacker with substantial resources. Consequences are translated in business impacts: A catastrophic consequence on system confidentiality is a major disclosure or highly-confidential information.

To apply correctly those flags, a good understanding of the business (the context) is required. It’s easy to understand that a website operated by a major bank will be more critical than the website of your local flowershop. But, in the same idea, a XSS vulnerability on the corporate bank portal will have less impact than the same issue on the e-banking front-end! I always suggest to the customer sit down together around the table and to review the issues found. An open discussion might change the initial flags that were applied. To conclude, each issue found during a pentest must be properly addressed and reviewed with the customer.