Gavin Black

MCP safety audit: LLMs with the Model Context Protocol allow major security exploits

Co-author, with John Halloran and Brandon Radosevich

When an LLM is connected to tools through the Model Context Protocol, it can be talked into attacking the developer's own machine: running malicious code, opening remote access, and stealing credentials. The paper shows this against leading models and introduces a scanner that audits MCP servers for these holes.

From LLMs to Randomness: Analyzing Program Input Efficacy With Resource and Language Metrics

Test inputs written by LLMs make programs do a wide variety of things, but they're worse than ordinary fuzzing at pushing a program's CPU and memory use around. Mixing the two covers more behavior than either one alone. Measured on over 5 million labeled samples from 50 Python programs.

Descriptor: Firewall Attack Detections and Extractions (FADE)

A dataset for testing web firewalls and language-model detectors: 50 million HTTP requests, about half of them attacks drawn from open-source firewall rule sets and public penetration-testing collections, mixed into realistic traffic. Every attack is labeled with its category and the exact bytes where the payload sits.

Balancing Security and Correctness in Code Generation: An Empirical Study on Commercial Large Language Models

I gave commercial LLMs (GPT-3.5 and 4, Claude Instant and Opus, Gemini 1.0) programming tasks that tend to produce well-known security bugs, then tested what they wrote. Security-focused prompts cut the vulnerabilities, but at a cost in correctness. No prompt reliably did both. Which model you use mattered more than any prompt.

Evaluating Large Language Models for Enhanced Fuzzing: An Analysis Framework for LLM-Driven Seed Generation

A fuzzer needs starting inputs, called seeds. I had LLMs write over 38,000 of them for 50 functions in widely used Python libraries and measured how much code each set reached. Which model you use matters most, and in some cases fewer than 100 LLM-written seeds beat more than 100,000 conventional ones.

RUFF: Resource Usage Fuzzing Framework

Most fuzzers only reward crashes and new code paths. RUFF also measures how much CPU and memory each input uses, and shows that these measurements are independent of each other and can tell where a set of inputs came from, including whether an LLM wrote them.

Process for Generating Offensive and Defense Security Dataset Augmentation with Invariance and Distribution Independence

With Paul F. Roysdon, assigned to Leidos. Application, pending.

A way to grow network and software security datasets so machine-learning models can be trained on them. The models feed a tool that automates training and deploying network defenses, and they learn what attacks look like as a whole instead of matching fixed patterns.

Security Dataset Augmentation Invariance and Distribution Independence

A security label depends on behavior, not statistics: a SQL injection is still an injection after you rewrite it, as long as the attack still works. That allows ways of growing a training set that go beyond the usual machine-learning rules. Every augmentation tested helped a query classifier, most of all the ones that extended the attack logic.