Teaching AI to Find Real Vulnerabilities

AI Engineer27m 17s
David Brumley in a blue top on black beside the headline ‘REAL VULNERABILITIES’ and a blue wireframe shield.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    David Brumley, a Carnegie Mellon University professor and Bugcrowd's Chief AI and Science Officer, explains how his team designs reinforcement-learning environmentsReinforcement learning trains an AI system to choose actions using feedback about the results of its earlier choices. for cybersecurity. He frames training along two axes: harder target programs and harder levels of exploitation. A student-learning story illustrates the progression from easier exercises to demanding real-world targets, but the technical focus is on how to measure what a model actually learned.

    The proposed environments place vulnerable software in reproducible containers and use deterministic gradersDeterministic software verification checks an output with a repeatable procedure that produces the same result for the same inputs and state. rather than letting an AI judge its own success. David Brumley argues that a crash is a useful early signal, but not proof that a model could control the program. He also explains why benchmarksA benchmark is a defined test or collection of tasks used to compare a system's performance under specified conditions. built around a single known bug can reward a model for finding the easiest flaw repeatedlyReward hacking occurs when a system improves a measured reward or score without achieving the intended underlying goal. or can leak the answer by pointing to the vulnerable function.

    His team's audit-task design instead asks for distinct proofs across all vulnerabilitiesA software vulnerability is a weakness in software that can permit an unintended or unauthorized effect on a system or its information. Its practical impact depends on the conditions under which it can be triggered. a model can find, including ones the benchmark authors did not know about. A grader separates duplicate crashes and uses precision and recall to discourage both repetition and unsupported claims. David Brumley presents this as a way to use real open-source software while keeping the reward signal tied to demonstrated behavior.

    For a harder evaluation, David Brumley reports results from 41 previously verified V8 vulnerabilities. The talk distinguishes triggering a crash from reaching stronger exploitation milestones and says the latter separates model performance much more sharply. These are speaker-reported findings; the official talk notes discrepancies in the displayed chart, so public copy does not assert exact model percentages without parent verification.

    The conclusion raises an open-science and safety tension: publishing a benchmark is useful, but releasing complete transcripts of newly produced high-value exploits could create risk. David Brumley says his team is building further training environments from previously unseen vulnerabilities to reduce memorization. The summary omits exploit mechanics and direct calls to download or run the benchmark.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows60
    • Coding & tools70
    • Models & learning75
    • Infrastructure & compute25
    • Research & evaluation100
    • Safety & security100
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy35

    Share this video

    David Brumley in a blue top on black beside the headline ‘REAL VULNERABILITIES’ and a blue wireframe shield. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 1 August 2026 and duration 27m 17s.

    David Brumley argues that AI security evaluations must reward distinct verified vulnerabilities and meaningful exploit milestones, not simply whether a model can crash a program.