Highlights from the 2015 Gartner Magic Quadrant for application security testing
Gartner recently published its 2015 Gartner Magic Quadrant for Application Security Testing (AST) report, which analyzes security vendors’ static, dynamic, mobile, and interactive application testing capabilities.
The report is based on analysis from 350 application-security-related inquiries in the past year, vendors’ responses to a detailed survey, and a survey of approximately 100 enterprises that used AST products, tools, technologies, and services. Based on that input, Gartner determines which vendors and products will be positioned in the Leaders, Challengers, Visionaries, and Niche Players quadrants in its reports.
If you’re not familiar with the Magic Quadrant, here’s a quick breakdown of how Gartner ranks vendors:
- Ability to execute: Product/service, overall viability, sales execution/pricing, market responsiveness/record, customer experience, and operations.
- Completeness of vision: Market understanding, marketing strategy, sales strategy, offering (product) strategy, business model, vertical/industry strategy, innovation, and geographic strategy.
For those of you evaluating application security testing tools and products, this article summarizes the analysis of each of the 19 vendors in the 2015 Magic Quadrant for AST. For a full analysis, download the full Gartner report:
2015 Gartner Magic Quadrant reveals vibrant application security testing market
Application security testing has become a ripe area for developers as more and more companies become aware of the key role the practice plays in securing their data.
Applications weren’t always as inviting to hackers as they are today. “Attackers used to go after networks when people didn’t manage their networks very well,” explains Chris Wysopal, CTO of Veracode, which makes application testing products.
“…Companies got better at securing their networks,” he continues, “so attackers now are going after two separate vectors: people and applications—especially web applications where an attacker can get the data in the application or use the application as a vector to get into the company.”
“Attackers go where it’s easiest for them to get in and now applications is one of those vectors,” he adds.
At the same time security on the inside of networks was improving, their perimeters were becoming more porous. “Application security has become a lot more important to companies because there are fewer perimeter technologies that can protect against the expanding points of attack,” says Maria Bledsoe, product marketing director for HP Fortify. “With mobile, with BYOD, with the Internet of Things, there is no way to protect data with traditional perimeter defenses.”
Bledsoe adds that at the heart of most data breaches is an application flaw. “Eight out of 10 breaches happen at the application layer,” she says. “The reason that happens is because there are so many applications out there, and it’s a new attack vector for hackers.”
The application layer is not only a new attack vector, it’s a good attack vector because chances are good that an application will be flawed. “Ninety-eight percent of the applications we test have at least one vulnerability,” says Charles Henderson, vice president of managed security testing at Trustwave.
Moreover, these days, applications hold something that information highwaymen lust after. “Why do you rob banks?” Henderson asks. “It’s where the money is. Why do you attack applications? Because that’s where the data is. Criminals are after data they can monetize and applications are where it’s at.”
Analysis of all 19 vendors in the latest Magic Quadrant
In the latest 2015 Magic Quadrant report, Gartner identified the ways security is tested in applications and the leading companies offering testing solutions. Those testing methods include:
- Static AST (SAST): This technology analyzes an application’s source and binary code for security vulnerabilities, typically at the programming or testing phases of the software lifecycle.
- Dynamic AST (DAST): This testing method analyzes applications while they’re running. It simulates attacks against an application, analyzes the application’s reactions to the attack, and then determines whether it’s vulnerable or not.
- Interactive AST (IAST): This technology combines elements of SAST and DAST simultaneously. It’s typically implemented as an agent within the test runtime environment.
- Mobile AST: This method uses a combination of traditional SAST and DAST, and behavioral analysis using static and dynamic techniques to discover malicious or potentially risky actions the app may be taking unbeknownst to the user.
All the testing methods can be delivered as either a tool or a service, the Gartner report explains, and adoption is variable. The most popular method is DAST, followed by SAST, while IAST and mobile AST have only recently emerged. “Mobile has some unique properties to it,” Veracode’s Wysopal says. “We ended up building a solution specifically for mobile applications because they’re different enough from other apps that they needed their own approach.”
Each of the testing methods has advantages and disadvantages, which is why multiple methods are often applied to applications. “Static testing, for example, is comprehensive,” Wysopal says. “Dynamic testing, on the other hand, isn’t as comprehensive. It can’t get to the entire application, but it has other benefits, such as you don’t have to have to access the code and you can test a website that’s running in production. So companies use different techniques to get more comprehensive results.”
In placing companies in its Magic Quadrant (MQ) report, Gartner focused on a vendor’s maturity in offering SAST and DAST features as tools or as security as a service. It weighed more heavily vendors’ innovation in AST for mobile applications, IAST, and emerging runtime application security protection (RASP) capabilities.
A company’s placement in the four quadrants of the MQ—as Leaders, Challengers, Niche Players, and Visionaries—is based on a number of criteria. They include core products and services offered, viability, sales execution and pricing, responsiveness to market conditions, marketing execution, customer service, operational effectiveness, market understanding, market and sales strategies, product differentiation, business model, approach to vertical markets, innovation, and geographic strategy.
Who are the “Leaders” in the AST Magic Quadrant?
Companies in the Leaders quadrant of the MQ have the highest ability to execute and completeness of vision. Gartner identified four application testing leaders in the MQ.
HP is the only AST vendor that provides capabilities in all four areas: SAST, DAST, IAST, and RASP. HP’s SAST has the broadest language support of any of the SAST providers, and its WebInspect IAST agent for Java and .NET is included at no cost for WebInspect DAST tool customers.
HP has a comprehensive set of enterprise capabilities for software composition analysis, Gartner adds, and it’s the only large AST vendor that provides SLAs with financial penalties on the turnaround time for its AST-as-a-service offerings.
Gartner cautions, however, that some of HP’s AST capabilities, such as malware detection, are only available with the Fortify on Demand offering. Although it offers security testing for mobile device languages on Android, iOS, and Windows Phone, it has limited behavioral mobile application security testing and its database of tested apps is relatively small (100,000 applications). The cost of equipping every developer with Fortify’s SAST capabilities can be high, if an organization chooses to equip individual developers, Gartner adds.
IBM has market strength, according to Gartner, with its Security AppScan, which is well-known for its enterprise application-security testing capabilities. IBM offers SAST, DAST, and IAST technologies. Its IAST for Java and .NET applications is integrated with its DAST offering, making IAST available at no cost to DAST users. It has begun offering mobile AST as a cloud service. Its IAST for Android offering includes a version of its Java Glass box testing technology to improve DAST scanning results when testing mobile applications. IBM is the first vendor to apply IAST to mobile applications.
Gartner cautions that IBM has not offered SAST as a service. It relies on Cigital for delivery of managed, human-augmented DAST services. It doesn’t provide RASP, and its mobile AST capabilities don’t include commercial application reputation ratings, proactive testing, or integration with EMM technologies.
Veracode offers SAST, DAST, and mobile AST, all as cloud services. Results of the different types of testing can be integrated into a single dashboard to simplify vulnerability management and remediation. Its AST as a service is scalable, and the company tests tens of thousands of applications per year. It includes an innovative “web application perimeter monitoring” service that discovers and tests web applications on the public Internet.
For Veracode, Gartner cautions that the company does not offer AST tools, only AST as a service. For compiled languages such as Java, C/C++, and Objective-C, applications must be in a compiled state to be analyzed by Veracode. The company does not yet offer IAST or RASP, and its web application firewall (WAF) integration is limited.
WhiteHat Security’s strength is its emphasis on the human element in its offering, Gartner notes. The results of all WhiteHat’s DAST and SAST scans are reviewed by a human expert before delivery to the customer. It also has some innovative architecture for SAST scanning that doesn’t require the code to compile correlation between its SAST and DAST when both are used to test an application. That allows SAST discoveries to be submitted for DAST execution to confirm or disprove suspected vulnerabilities. WhiteHat also puts its money where its protection is. If it fails to identify a vulnerability it should have found, which is later exploited by hackers, it will pay damage fees up to $500,000, in addition to the customer’s subscription fee.
WhiteHat doesn’t sell DAST and SAST tools, Gartner cautions. However, its on-premise virtual appliance can keep scanning and scanning results local. Its SAST is only for a limited number of programming languages. For mobile AST, WhiteHat provides source code analysis for Objective-C and Java but doesn’t offer automated mobile behavioral testing, reputation service, proactive testing, or integration with EMM. IAST and RASP aren’t currently supported, but RASP is on its road map.
Who are the “Challengers” in the AST Magic Quadrant?
Companies in the Challengers quadrant have a high ability to execute but a less complete vision compared to the leaders. Gartner includes seven companies in the Challenger quadrant.
Gartner cautions, however, that human-augmented testing must be purchased separately with Acunetix. In addition, the company doesn’t have SAST capabilities, and its IAST capabilities aren’t available for Java. No specific mobile application testing capabilities other than testing the HTTP-based interfaces to and from the mobile application are offered by the company.
Cigital has an innovative SecureAssist tool that can check for a limited number of SAST coding issues by integrating directly within the developer’s integrated development environment (IDE).
Gartner cautions that Cigital’s SAST as a service is a lesser-known offering and has not yet been widely adopted. In addition, the company doesn’t have IAST or RASP testing capabilities, and it doesn’t offer WAF integration.
Checkmarx has one of the strongest SAST technologies. Its universal application model can be queried to discover vulnerabilities and to check for code adherence to secure programming best practices. It enables incremental scans and analysis across components of composite applications written in different programming languages and with the use of different frameworks. It also offers its SAST tech as a tool and as a cloud service, and it will test mobile apps, with support for Android, iOS, and Windows Phone platforms.
On the caution side, Gartner notes that Checkmarx doesn’t offer its own DAST for web applications, and its DAST for .NET and Java is in the planning stage, as well as its RASP technology. In addition, its SAST integration with WAF supports only ModSecurity and not leading commercial WAFs.
However, PortSwigger doesn’t offer SAST, IAST or RASP, and it doesn’t offer mobile application code analysis, behavioral analysis, integration with EMM, or commercial app reputation ratings. DAST as a service is also missing from its offerings, as well as integration with WAFs, IDEs, or QA systems.
Qualys has priced its service offering very competitively. Gartner says that Qualys has one of the lowest costs-per-application-scanned of any of the DAST-as-a-service providers. Its DAST scans include detection of malware on websites. All Qualys subscriptions come with 24/7 technical support and extensive WAF integration, including its own WAF-as-a-service offering.
Human augmentation isn’t part of the Qualys solution, Gartner cautions, and although the provider offers basic Web Services Description Language (WSDL) and SOAP web services fuzzing, it doesn’t support the rest of the WS-* standards, nor does it test RESTful application interfaces or test the content within JSON messages. In addition, it doesn’t have IAST, RASP, or SAST-as-a-service capabilities. Finally, there are no mobile AST capabilities other than testing the web-services-based interfaces used by the mobile application and no support of out-of-the-box trouble ticketing system integration for WAS vulnerabilities.
Rapid7 (NTO) has an innovative “universal translator” technology that normalizes how requests are handled in an application for specific attacks across HTML forms, SOAP, JSON, REST, and Action Message Format (AMF) and other formats as they emerge. It also has a broad array of enterprise AST capabilities and an innovative cloud platform that acts behind the scenes to automatically create new scan engines from Rapid7’s cloud to handle increased workloads with cloud servers in the US and Europe. Also in its repertoire is workflow-based sequence attacking for testing complex workflows while maintaining the state of a session.
On the other hand, Rapid7 doesn’t have any SAST, IAST, or RASP capabilities, and the company charges separately for vulnerability verification by a human being.
Trustwave has a WAF service, as well as several web monitoring services, including web malware monitoring and web content monitoring services. It supports a broad array of enterprise capabilities, including IDE integration, bug-tracking integration, quality testing tool integration, vulnerability replay, RBAC, proprietary Hailstorm Application Risk Metric (HARM) risk scoring, and a large selection of WAF integrations. It also has three tiers of mobile application security testing services for iOS, Android, Windows Mobile, and BlackBerry.
However, Gartner points out that Trustwave doesn’t offer SAST, IAST, or RASP, and although it offers mobile AST services, it doesn’t have a mobile AST product.
Who are the “Niche Players” in the AST Magic Quadrant?
Niche players have a lower ability to execute and completeness of vision than other the companies in the MQ. Six companies were placed in the niche quadrant by Gartner.
Absent from Appthority’s toolbox is the ability to analyze mobile applications for source code security vulnerabilities, an on-premise mobile AST tool, out-of-the-box integration with application development environments, and support of Windows Mobile, Windows Phone, or BlackBerry mobile application platforms.
NSFOCUS is a well-known security provider in China. It sells its own WAF product and managed WAF service offering.
Since the company is not well known outside of China, Gartner says it will have difficulty selling into security-sensitive industries, such as defense, aerospace, critical infrastructure, and government outside of that country. In addition, it has no integration WAF offerings other than its own; no capability to test web services, REST, JSON, or XML-based application interfaces; and no SAST or mobile application security testing capabilities.
N-Stalker, based in Brazil, supports the identification and scanning of more than 1,900 commercial off-the-shelf (COTS) and open-source software (OSS) packages for more than 5,000 Common Vulnerabilities and Exposures (CVEs) related to these packages, as well as the ability to discover unknown vulnerabilities. It has a broad array of enterprise features not typically found from smaller providers. Its Cloud Web Scan service can perform SAST on web applications.
Gartner notes that the company has limited brand awareness outside of South America. It adds that the company’s integrated SAST capabilities are only available via its Cloud Web Scan platform. In addition, it has no IAST or RASP capabilities and very limited mobile AST capabilities available only as a service.
Pradeo, a privately held startup based in France, offers testing services that combine the use of static, dynamic, and behavioral code analysis of mobile applications. It offers its technology as a service, either directly from the cloud or using an optional on-premise virtual appliance. The company has its own EMM agent, which can act on the results from its mobile AST. The agent can also integrate with EMM tools from AirWatch, MobileIron, Good Technology, and SAP.
Some missing pieces in Pradeo’s repertoire include the inability to analyze mobile applications for security vulnerabilities, retrieval of binaries only from iOS and Android, and no integration with application development environments and bug-tracking systems.
SiteLock is one of a few vendors that provide both DAST and SAST capabilities. It combines both approaches in a single-priced, comprehensive web application security scan. All of SiteLock’s web security testing services include automated malware detection and removal capabilities.
However, the company doesn’t yet have a strong brand in AST, and its AST-as-a-service capabilities are basic. It doesn’t offer IAST, RASP, or mobile AST capabilities, and it has no integration with WAFs. In addition, the company’s solution tests only web applications (not stand-alone native applications), and its human-augmented web scanning services are only available at extra cost.
Virtual Forge is a German company and the only vendor other than HP that is capable of static analysis of ABAP code. The company has deep expertise in SAP and has a SystemProfiler that scans the SAP environment for secure configuration and up-to-date patching. The company’s CodeProfiler is integrated into the SAP development environment, so it will be familiar to SAP developers without them having to learn a separate console.
Deep SAP integration can have its drawbacks, though. As SAP’s new SAPUI5 technology is phased in, Virtual Forge will need to improve its HTML5-testing capabilities; and while CodeProfiler is great for SAP developers, it will be unfamiliar to most security testing professionals. Other missing elements in the company’s solution include the absence of broad SAST language support, no RASP or true IAST support, and no mobile AST or out-of-the-box WAF integration.
Who are the “Visionaries” in the AST Magic Quadrant?
Visionary companies have the completeness of vision of the leaders, but the ability of the niche players to execute on that vision. Gartner identified two companies as visionaries.
Contrast Security has a self-testing model, where security testing is driven by any application test that is executed automatically or manually. This process is transparent to the interested parties (developers and security specialists) and doesn’t require training. The solution is highly scalable and enables analytics of production applications at runtime.
Synopsys has increased its testing capabilities through the acquisition of Quotium and Coverity in 2014 and Codenomicon in 2015. Quotium’s Seeker product is one of the most broadly adopted IAST technologies in the IAST market. It has a number of features that fit well into DevOps and DevOpsSec approaches. It includes IAST for Java, .NET, and PHP application server platforms, as well as support for PL/SQL and T-SQL.
On the downside, Seeker doesn’t have mobile AST or RASP capabilities, and WAF support is limited to F5.
A maturing application security industry
Overall, the application security testing industry appears to be maturing, as Gartner’s Magic Quadrant proves. “If [you] look at all the companies in the Magic Quadrant, most of the companies offer multiple testing techniques,” Veracode’s Wysopal says. “They either develop them themselves or they acquire companies that have the technologies they need.”
“That signals that the market is looking for companies that can combine multiple testing techniques together to give you a comprehensive approach,” he continues. “What you’re going to see is the smaller-point companies that do just one thing either join up with other small companies or [will] be acquired by bigger companies [in order] to survive.”
– See more at: http://techbeacon.com/highlights-2015-gartner-magic-quadrant-application-security-testing#sthash.66uY03rc.dpuf