research
∙
05/24/2023
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
Recent studies have discovered that Chain-of-Thought prompting (CoT) can...
share
research
∙
03/23/2023
Adaptive Endpointing with Deep Contextual Multi-armed Bandits
Current endpointing (EP) solutions learn in a supervised framework, whic...
research
∙
02/14/2023
A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests
Recently, subgraph GNNs have emerged as an important direction for devel...
research
∙
02/12/2023
3D Molecular Generation via Virtual Dynamics
Structure-based drug design, i.e., finding molecules with high affinitie...
research
∙
02/03/2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
We propose a new class of linear Transformers called FourierLearner-Tran...
research
∙
01/23/2023
Rethinking the Expressive Power of GNNs via Graph Biconnectivity
Designing expressive Graph Neural Networks (GNNs) is a central topic in ...
research
∙
01/15/2023
DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets
Designing an efficient yet deployment-friendly 3D backbone to handle spa...
research
∙
10/09/2022
Online Training Through Time for Spiking Neural Networks
Spiking neural networks (SNNs) are promising brain-inspired energy-effic...
research
∙
10/04/2022
Rethinking Lipschitz Neural Networks for Certified L-infinity Robustness
Designing neural networks with bounded Lipschitz constant is a promising...
research
∙
10/04/2022
One Transformer Can Understand Both 2D 3D Molecular Data
Unlike vision and language data which usually has a unique format, molec...
research
∙
06/09/2022
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Adversarial examples, which are usually generated for specific inputs wi...
research
∙
06/04/2022
Is L^2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
The Physics-Informed Neural Network (PINN) approach is a new and promisi...
research
∙
05/26/2022
Your Transformer May Not be as Powerful as You Expect
Relative Positional Encoding (RPE), which encodes the relative distance ...
research
∙
04/13/2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
We present an efficient method of pretraining large-scale autoencoding l...
research
∙
03/09/2022
Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets
This technical note describes the recent updates of Graphormer, includin...
research
∙
02/28/2022
An Empirical Study of Graphormer on Large-Scale Molecular Modeling Datasets
This technical note describes the recent updates of Graphormer, includin...
research
∙
02/22/2022
VADOI:Voice-Activity-Detection Overlapping Inference For End-to-end Long-form Speech Recognition
While end-to-end models have shown great success on the Automatic Speech...
research
∙
02/16/2022
HousE: Knowledge Graph Embedding with Householder Parameterization
The effectiveness of knowledge graph embedding (KGE) largely depends on ...
research
∙
11/02/2021
Can Vision Transformers Perform Convolution?
Several recent studies have demonstrated that attention-based networks, ...
research
∙
10/13/2021
Boosting the Certified Robustness of L-infinity Distance Nets
Recently, Zhang et al. (2021) developed a new neural network architectur...
research
∙
06/23/2021
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
The attention module, which is a crucial component in Transformer, canno...
research
∙
06/15/2021
First Place Solution of KDD Cup 2021 OGB Large-Scale Challenge Graph-Level Track
In this technical report, we present our solution of KDD Cup 2021 OGB La...
research
∙
06/09/2021
Do Transformers Really Perform Bad for Graph Representation?
The Transformer architecture has become a dominant choice in many domain...
research
∙
05/31/2021
Adversarial Training with Rectified Rejection
Adversarial training (AT) is one of the most effective strategies for pr...
research
∙
05/10/2021
How could Neural Networks understand Programs?
Semantic understanding of programs is a fundamental problem for programm...
research
∙
03/09/2021
Wav2vec-C: A Self-supervised Model for Speech Representation Learning
Wav2vec-C introduces a novel representation learning technique combining...
research
∙
02/27/2021
Transformers with Competitive Ensembles of Independent Mechanisms
An important development in deep learning from the earliest MLPs has bee...
research
∙
02/25/2021
LazyFormer: Self Attention with Lazy Update
Improving the efficiency of Transformer-based language pre-training is a...
research
∙
02/18/2021
Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder
Many real-world applications use Siamese networks to efficiently match t...
research
∙
02/16/2021
Revisiting Language Encoding in Learning Multilingual Representations
Transformer has demonstrated its great power to learn contextual word re...
research
∙
02/10/2021
Towards Certifying ℓ_∞ Robustness using Neural Networks with ℓ_∞-dist Neurons
It is well-known that standard neural networks, even with a high classif...
research
∙
01/31/2021
CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics
Accurate and robust prediction of patient's response to drug treatments ...
research
∙
10/09/2020
A Cross-Level Information Transmission Network for Predicting Phenotype from New Genotype: Application to Cancer Precision Medicine
An unsolved fundamental problem in biology and ecology is to predict obs...
research
∙
09/07/2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Normalization plays an important role in the optimization of deep neural...
research
∙
08/04/2020
Taking Notes on the Fly Helps BERT Pre-training
How to make unsupervised language pre-training more efficient and less r...
research
∙
07/24/2020
Transferred Discrepancy: Quantifying the Difference Between Representations
Understanding what information neural networks capture is an essential p...
research
∙
06/28/2020
Rethinking Positional Encoding in Language Pre-training
How to explicitly encode positional information into neural networks is ...
research
∙
06/28/2020
Rethinking the Positional Encoding in Language Pre-training
How to explicitly encode positional information into neural networks is ...
research
∙
06/10/2020
MC-BERT: Efficient Language Pre-Training via a Meta Controller
Pre-trained contextual representations (e.g., BERT) have become the foun...
research
∙
05/12/2020
Invertible Image Rescaling
High-resolution digital images are usually downscaled to fit various dis...
research
∙
02/17/2020
Incorporating BERT into Neural Machine Translation
The recently proposed BERT has shown great power on a variety of natural...
research
∙
02/12/2020
On Layer Normalization in the Transformer Architecture
The Transformer is widely used in natural language processing tasks. To ...
research
∙
01/08/2020
MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Adversarial training is one of the most popular ways to learn robust mod...
research
∙
11/19/2019
Defective Convolutional Layers Learn Robust CNNs
Robustness of convolutional neural networks has recently been highlighte...
research
∙
10/25/2019
Fast Structured Decoding for Sequence Models
Autoregressive sequence models achieve state-of-the-art performance in d...
research
∙
09/27/2019
On the Anomalous Generalization of GANs
Generative models, especially Generative Adversarial Networks (GANs), ha...
research
∙
09/15/2019
Hint-Based Training for Non-Autoregressive Machine Translation
Due to the unparallelizable nature of the autoregressive factorization, ...
research
∙
08/25/2019
Multilingual Neural Machine Translation with Language Clustering
Multilingual neural machine translation (NMT), which translates multiple...
research
∙
07/28/2019
Representation Degeneration Problem in Training Natural Language Generation Models
We study an interesting problem in training neural network-based models ...
research
∙
06/06/2019