Rise of the HaCRS: Augmenting Autonomous Cyber Reasoning Systems with Human Assistance

by   Yan Shoshitaishvili, et al.

As the size and complexity of software systems increase, the number and sophistication of software security flaws increase as well. The analysis of these flaws began as a manual approach, but it soon became apparent that tools were necessary to assist human experts in this task, resulting in a number of techniques and approaches that automated aspects of the vulnerability analysis process. Recently, DARPA carried out the Cyber Grand Challenge, a competition among autonomous vulnerability analysis systems designed to push the tool-assisted human-centered paradigm into the territory of complete automation. However, when the autonomous systems were pitted against human experts it became clear that certain tasks, albeit simple, could not be carried out by an autonomous system, as they require an understanding of the logic of the application under analysis. Based on this observation, we propose a shift in the vulnerability analysis paradigm, from tool-assisted human-centered to human-assisted tool-centered. In this paradigm, the automated system orchestrates the vulnerability analysis process, and leverages humans (with different levels of expertise) to perform well-defined sub-tasks, whose results are integrated in the analysis. As a result, it is possible to scale the analysis to a larger number of programs, and, at the same time, optimize the use of expensive human resources. In this paper, we detail our design for a human-assisted automated vulnerability analysis system, describe its implementation atop an open-sourced autonomous vulnerability analysis system that participated in the Cyber Grand Challenge, and evaluate and discuss the significant improvements that non-expert human assistance can offer to automated analysis approaches.


page 1

page 2

page 3

page 4


The Coming Era of AlphaHacking? A Survey of Automatic Software Vulnerability Detection, Exploitation and Patching Techniques

With the success of the Cyber Grand Challenge (CGC) sponsored by DARPA, ...

The Coming Era of AlphaHacking?

With the success of the Cyber Grand Challenge (CGC) sponsored by DARPA, ...

ESASCF: Expertise Extraction, Generalization and Reply Framework for an Optimized Automation of Network Security Compliance

The Cyber threats exposure has created worldwide pressure on organizatio...

Learning to Optimize Autonomy in Competence-Aware Systems

Interest in semi-autonomous systems (SAS) is growing rapidly as a paradi...

Development of a System Vulnerability Analysis Tool for Assessment of Complex Mission Critical Systems

A system vulnerability analysis technique (SVAT) for complex mission cri...

Exploring how Component Factors and their Uncertainty Affect Judgements of Risk in Cyber-Security

Subjective judgements from experts provide essential information when as...

Knowledge Rocks:Adding Knowledge Assistance to Visualization Systems

We present Knowledge Rocks, an implementation strategy and guideline for...

Please sign up or login with your details

Forgot password? Click here to reset