research
∙
05/04/2023
On the Expressivity Role of LayerNorm in Transformers' Attention
Layer Normalization (LayerNorm) is an inherent component in all Transfor...
share
research
∙
03/01/2023
Diffusing Graph Attention
The dominant paradigm for machine learning on graphs uses Message Passin...
research
∙
06/13/2021
Thinking Like Transformers
What is the computational model behind a Transformer? Where recurrent ne...
research
∙
05/30/2021
How Attentive are Graph Attention Networks?
Graph Attention Networks (GATs) are one of the most popular GNN architec...
research
∙
06/09/2020
On the Bottleneck of Graph Neural Networks and its Practical Implications
Graph neural networks (GNNs) were shown to effectively learn from highly...
research
∙
05/27/2020
A Structural Model for Contextual Code Changes
We address the problem of predicting edit completions based on a learned...
research
∙
04/18/2020
A Formal Hierarchy of RNN Architectures
We develop a formal hierarchy of the expressive capacity of RNN architec...
research
∙
10/30/2019
Learning Deterministic Weighted Automata with Queries and Counterexamples
We present an algorithm for extraction of a probabilistic deterministic ...
research
∙
10/15/2019
Adversarial Examples for Models of Code
We introduce a novel approach for attacking trained models of code with ...
research
∙
09/30/2019
Structural Language Models of Code
We address the problem of any-code completion - generating a missing pie...
research
∙
09/30/2019
Structural Language Models for Any-Code Generation
We address the problem of Any-Code Generation (AnyGen) - generating code...
research
∙
05/20/2019
Towards Neural Decompilation
We address the problem of automatic decompilation, converting a program ...
research
∙
02/25/2019
Neural Reverse Engineering of Stripped Binaries
We address the problem of predicting procedure names in stripped executa...
research
∙
08/04/2018
code2seq: Generating Sequences from Structured Representations of Code
The ability to generate natural language sequences from source code snip...
research
∙
05/13/2018
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
While Recurrent Neural Networks (RNNs) are famously known to be Turing c...
research
∙
03/26/2018
A General Path-Based Representation for Predicting Program Properties
Predicting program properties such as names or expression types has a wi...
research
∙
03/26/2018
code2vec: Learning Distributed Representations of Code
We present a neural model for representing snippets of code as continuou...
research
∙
10/03/2017