The Deep Learning Compiler: A Comprehensive Survey

by   Mingzhen Li, et al.

The difficulty of deploying various deep learning (DL) models on diverse DL hardware has boosted the research and development of DL compilers in the community. Several DL compilers have been proposed from both industry and academia such as Tensorflow XLA and TVM. Similarly, the DL compilers take the DL models described in different DL frameworks as input, and then generate optimized codes for diverse DL hardware as output. However, none of the existing survey has analyzed the unique design architecture of the DL compilers comprehensively. In this paper, we perform a comprehensive survey of existing DL compilers by dissecting the commonly adopted design in details, with emphasis on the DL oriented multi-level IRs, and frontend/backend optimizations. Specifically, we provide a comprehensive comparison among existing DL compilers from various aspects. In addition, we present detailed analysis on the design of multi-level IRs and illustrate the commonly adopted optimization techniques. Finally, several insights are highlighted as the potential research directions of DL compiler. This is the first survey paper focusing on the design architecture of DL compilers, which we hope can pave the road for future research towards DL compiler.


page 11

page 12

page 13

page 17

page 32

page 33

page 34

page 35


Anomalous Instance Detection in Deep Learning: A Survey

Deep Learning (DL) is vulnerable to out-of-distribution and adversarial ...

Bosch Deep Learning Hardware Benchmark

The widespread use of Deep Learning (DL) applications in science and ind...

Deep Insights of Learning based Micro Expression Recognition: A Perspective on Promises, Challenges and Research Needs

Micro expression recognition (MER) is a very challenging area of researc...

Understanding the Limits of Conventional Hardware Architectures for Deep-Learning

Deep learning and hardware for it has garnered immense academic and indu...

FamilySeer: Towards Optimized Tensor Codes by Exploiting Computation Subgraph Similarity

Deploying various deep learning (DL) models efficiently has boosted the ...

A Survey of Modern Compiler Fuzzing

Most software that runs on computers undergoes processing by compilers. ...

Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues

Over recent years, there has been a rapid development of deep learning (...

Please sign up or login with your details

Forgot password? Click here to reset