SlipCover: Near Zero-Overhead Code Coverage for Python

by   Juan Altmayer Pizzorno, et al.

Coverage analysis is widely used but can suffer from high overhead. This overhead is especially acute in the context of Python, which is already notoriously slow (a recent study observes a roughly 30x slowdown vs. native code). We find that the state-of-the-art coverage tool for Python,, introduces a median overhead of 180 interpreter. Slowdowns are even more extreme when using PyPy, a JIT-compiled Python implementation, with imposing a median overhead of 1,300 This performance degradation reduces the utility of coverage analysis in most use cases, including testing and fuzzing, and precludes its use in deployment. This paper presents SlipCover, a novel, near-zero overhead coverage analyzer for Python. SlipCover works without modifications to either the Python interpreter or PyPy. It first processes a program's AST to accurately identify all branches and lines. SlipCover then dynamically rewrites Python bytecodes to add lightweight instrumentation to each identified branch and line. At run time, SlipCover periodically de-instruments already-covered lines and branches. The result is extremely low overheads – a median of just 5 SlipCover suitable for use in deployment. We show its efficiency can translate to significant increases in the speed of coverage-based clients. As a proof of concept, we integrate SlipCover into TPBT, a targeted property-based testing system, and observe a 22x speedup.


page 8

page 9


Using Relative Lines of Code to Guide Automated Test Generation for Python

Raw lines of code (LOC) is a metric that does not, at first glance, seem...

Predicting Code Coverage without Execution

Code coverage is a widely used metric for quantifying the extent to whic...

Improving Tese Case Generation for Python Native Libraries Through Constraints on Input Data Structures

Modern Python projects execute computational functions using native libr...

Towards exhaustive branch coverage with PathCrawler

Branch coverage of source code is a very widely used test criterion. Mor...

PartiSan: Fast and Flexible Sanitization via Run-time Partitioning

Code sanitizers are used to automatically detect security vulnerabilitie...

Triangulating Python Performance Issues with Scalene

This paper proposes Scalene, a profiler specialized for Python. Scalene ...

FuzzingDriver: the Missing Dictionary to Increase Code Coverage in Fuzzers

We propose a tool, called FuzzingDriver, to generate dictionary tokens f...

Please sign up or login with your details

Forgot password? Click here to reset