Google has released a framework that harnesses the power of Large Language Models (LLMs) to streamline vulnerability discovery. This fuzzing framework, previously employed internally, has now been made accessible to developers and security researchers worldwide.
The tool generates specialized fuzz targets tailored for real-world C and C++ projects. These fuzz targets are then rigorously benchmarked using Google’s acclaimed OSS-Fuzz service, which is renowned for its automated vulnerability detection capabilities in open-source software.
The open-sourced framework includes support for Vertex AI code-bison, Vertex AI code-bison-32k, Gemini Pro, OpenAI GPT-3.5-turbo, and OpenAI GPT-4. Additionally, it employs a comprehensive evaluation process, assessing generated fuzz targets against real-world data from production environments, considering four metrics: compilability, runtime crashes, runtime coverage, and runtime line coverage.
“Overall, this framework manages to successfully leverage LLMs to generate valid fuzz targets (which generate non-zero coverage increase) for 160 C/C++ projects. The maximum line coverage increase is 29% from the existing human-written targets,” Google mentioned.
This AI-aided approach has notably led to the discovery of two previously undetected vulnerabilities in widely used projects like cJSON and libplist, which had undergone extensive fuzzing for years.
Google’s decision to integrate LLMs into its fuzzing processes dates back to August 2023, when they recognized the potential for AI to automate aspects of manual fuzz testing. This strategic move yielded remarkable results, with a 30% increase in code coverage across over 300 OSS-Fuzz C/C++ projects, enhancing the chances of uncovering vulnerabilities.
Beyond vulnerability discovery, Google is exploring the application of LLMs in vulnerability patching, aiming to develop an automated method for generating, testing, and selecting optimal code fixes. Initial trials have shown promising results, with the AI-powered patching approach resolving 15% of targeted bugs, potentially leading to significant time savings for engineers.
“This AI-powered patching approach resolved 15% of the targeted bugs, leading to significant time savings for engineers.” Google said.
As the software industry is faced with the ever-increasing complexity of code and the relentless pursuit of security, Google’s open-source AI-aided fuzzing framework represents a significant stride towards strengthening software security.
About the author:
