Web App Security: Comparing and contrasting Black Box, White Box, Fault Injection, and SCA
This article is based on a talk I gave at the Phoenix OWASP chapter on May 10th.My intention is to summarize the methods used to assess the security of web applications, identify what they are good and not so good at finding, and outline their varying strengths and weaknesses. If you’ll indulge me, I’d like to spend some time building up to that with some background material.
What’s the big deal, anyway?
I am very pleased about the about the growing awareness surrounding web application security threats. Several organizations have been formed to promote the issue, such as OWASP and the Web Application Security Consortium, and for good reason: it is currently the most prolific attack vector. In fact, Gartner estimates that 75% of all attacks now come at the application layer.
The reason why is no mystery. Whereas in the 90’s, system configuration, buffer overflow, and other platform level type flaws were all the rage, these have become increasingly easy to manage. Economies of scale have given ubiquity and commodity status to packet-filtering firewalls, multi-platform patch management systems, vulnerability scanners, and intrusion prevention systems. The kinds of attacks most often prevented by these technologies are now considered ‘low hanging fruit.
At the same time, the population of attackers has vastly increased. The maxim goes that a security system is only as strong as its weakest link, so that’s what attackers look for. Attacks have moved both up and down the stack. By this I mean, up to the application and even client level, and down to the system internals and driver level. Blue Pill, a virtual machine malware platform, is one such example that takes advantage of hardware features at the bottom level, while at the top level you have the world of web application attacks, where web applications are used as proxies to attack the integrity of the application as well as its architectural dependencies, and Javascript attacks, which are used to attack the softest target of all – the end user. At the Javascript / client attack level, state of the art is represented by PDP’s AttackAPI.
Custom web application security is different than platform security, to say the least. There are no vendor advisories or patches. Attackers like web applications because they have built in, exposed mechanisms that must have connectivity to the data the attacker is after. The attacker thinks, why compromise an entire system when you can manipulate the application into coughing up what you’re looking for? Most protection is at the network, not application layer, so the chances of getting caught are much lower. Application attacks are much harder to catch and prevent at the network layer, because the network components don’t understand the application, it’s logic, or which resources should be accessed and by which user roles. Web Application Firewalls (WAF) are an incomplete solution, often being network layer devices. The WEBAPPSEC mailing list has a great thread on this topic going on, right now. (See the May/June 2007 thread called “PCI 6.6 Questions) I’m not even certain WAF should be called a “firewall,” since they’re more of an Application layer Intrusion Prevention System, only they typically operate at the network layer, having no visibility into application internals. Fortify Defender is a notable exception. I’m starting to stray – this should probably be the subject of a future blog article…
As a result, it is incumbent upon organizations to understand the attackable surface area represented by web applications, particularly those that store and process confidential personal or payment card data.
Integrating Web Application Security Testing Approaches
One of the many hats I wear at QuietMove is to create our testing methodologies such that they maximize the efficiency and effectiveness of the time scoped for a particular assessment activity. I tackle this in two ways. One is by identifying the most comprehensive automated tools. I am a big proponent of automation – computers are good at automating things in a repeatable, measurable way. The second is in accounting for the fact that there are many classes of vulnerabilities which automated tools have serious trouble finding. This is partly a function of the perspective from which the tool operates, such as Source Code Analysis vs Fault Injection (more on this later), as well as the state of the art of each of these evolving technologies. Therefore, the second way recognizes that it’s critical we understand what automated tools can and can’t find, and develop other methods for identifying the “false negatives” – vulnerabilities that exist, but were missed.
The two main approaches that exist at present for web application testing are “Fault Injection” and “Source Code Analysis.” There are also two more philosophical approaches, “white box,” and “black box.” The results gained from a test are in no small way closely related to the assessment approach taken.
I’m going to define four terms that are key to this discussion:
White Box - a “full knowledge” approach to an assessment. This includes access to things like functional specifications and other design documents, network architecture, and source code.
Black Box – a “zero knowledge” approach to an assessment. The assessor starts off with no advance knowledge of the application. This is typically performed using automated fault injection tools, a web browser, and an HTTP proxy like Paros, Burp Proxy, or one of their many equivalents.
Fault Injection – interactive testing of a website that includes spidering, querying for known vulnerable scripts or components, testing for conditions like forceful browsing, directory traversal, and using the results of spidering to identify all points of user input to test for flaws like SQL injection, XSS, CSRF, command execution, etc. Typically a combination of fuzzing and injection of strings known to cause error conditions are used.
Static Code Analysis – is also often known as “Source Code Analysis.” This is often employs a mix of techniques such as searching for strings, identifying user input vectors, tracing the flow of data through the application, and mapping execution paths. Depending on the tool, it’s employed against source code or binaries.
Awareness is growing for a different technique often referred to as “Grey Box assessment,” which integrates the approaches described above. This approach combines static and fault injection testing techniques, in order to compensate for their different detection capabilities, and also integrates elements of white and black box methodologies. In practice, my observation is that the most comprehensive results are achieved through an iterative process involving an initial “white box” fault injection assessment, followed by static code analysis. The results of the static/source code analysis assessment are then fed back into further “fault injection” testing to validate the code analysis results and better inform the tester about the application architecture and areas to examine for further vulnerabilities using hands-on techniques. In my experience, this is the most comprehensive methodology, though for obvious reasons it’s also the most time consuming.
The following matrix was developed to present a high level view of the strengths and weaknesses of each approach. Suggestions for additional strengths and weaknesses would be welcomed.
(click to view)
The matrix view demonstrates some of the trade-offs at play: Detection capability, expense, and time. It visualizes a common theme of public talks I give about web application security, specifically how the testing methodology chosen impacts the results that will be achieved. It brings to mind the old engineering maxim. “Good, cheap, fast: Pick any two.”
This is the first of several blog posts I’ll be posting at Security Catalyst about web application security testing. The next will be about several different approaches to planning test scenarios, and the relative strengths and weaknesses of each.
Hackers from India Indicted for Online Brokerage Intrusion Scheme
From http://www.infozine.com/news/stories/op/storiesView/sid/21633/
A few snippets from the article:
“A federal grand jury in Omaha, Neb., has indicted three individuals on charges of conspiracy, fraud and aggravated identity theft stemming from a high-tech, international fraud scheme designed to hijack online brokerage accounts for profit…”
“As part of this ongoing investigation, at least 60 customers and nine brokerage firms in the United States and elsewhere have been identified as victims, with one of the brokerage firms reporting more than $2 million in losses. Today’s case marks the first time that individuals have been arrested overseas in connection with an online brokerage intrusion scheme perpetrated in the United States. “
Bravo for catching these guys, but I’m frankly surprised that it’s the first time an overseas arrest has happened for this kind of activity! Does that mean everyone else who has done it, has gotten away with it?
Here’s what they actually did. Smells like XSS flaws were involved but the article doesn’t say.
“Hack, Pump and Dump” Scheme
“In one of many examples alleged in the indictment, Marimuthu placed orders on Aug. 28, 2006, through his personal online brokerage account, to purchase 32,000 shares of stock in a company at prices from $2 to $3.20 per share. Chockalingam also placed an order through his personal online brokerage account to purchase 450 shares of the same stock for $3.20 per share.”
“The same day, the defendants gained unauthorized access to the online brokerage account of an unsuspecting investor. According to the indictment, the defendants used this account to illegally acquire 26,000 shares of the same stock at prices from $2.84 to $3.40 per share, causing the stock’s trading volume to rise to more than nine times its 15-day average.”
“Marimuthu then placed an order to sell 1,500 shares of the same stock from his personal online brokerage account at five dollars per share. This was one of at least 22 sell orders for this stock placed in Marimuthu’s personal online brokerage accounts between Aug. 28, 2006, and the morning of Aug. 29, 2006. These transactions allegedly resulted in the sale of 30,700 shares of this stock, yielding a substantial profit for the defendants over the course of just a few hours. The defendants used this type of scheme with various stocks between July and November. 2006.”
This was a creative way to apply hacking to the “pump and dump” stock scam. Fortunately the miscreants have been caught (except for one) and are being prosecuted.
I suspect it was a combination of phishing and XSS which was used to compromise the accounts. As the court cases unfold, I’m going to follow this because I am very curious to find out about how they were caught.
-Adam
Photocopiers - PCI and Data Security Threat?
“Photocopiers are the newest threat to identity theft, a copier maker said today, because newer models equipped with hard drives record what’s been duplicated. At tax time, when Americans photocopy tax returns, confidential information may be easily available to criminals.”
Not only that, but documents containing credit card numbers, or actual live cards, can be and often are photocopied as well.
What other sensitive documents to you photocopy at your business?




Save to del.icio.us