Attention Incorporate Network: A network can adapt various data size

06/06/2018
by   Liangbo He, et al.
0

In traditional neural networks for image processing, the inputs of the neural networks should be the same size such as 224*224*3. But how can we train the neural net model with different input size? A common way to do is image deformation which accompany a problem of information loss (e.g. image crop or wrap). Sequence model(RNN, LSTM, etc.) can accept different size of input like text and audio. But one disadvantage for sequence model is that the previous information will become more fragmentary during the transfer in time step, it will make the network hard to train especially for long sequential data. In this paper we propose a new network structure called Attention Incorporate Network(AIN). It solve the problem of different size of inputs including: images, text, audio, and extract the key features of the inputs by attention mechanism, pay different attention depends on the importance of the features not rely on the data size. Experimentally, AIN achieve a higher accuracy, better convergence comparing to the same size of other network structure

READ FULL TEXT

page 2

page 6

research
03/22/2017

Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection

Traditional speaker change detection in dialogues is typically based on ...
research
08/25/2017

Hierarchical Multi-scale Attention Networks for Action Recognition

Recurrent Neural Networks (RNNs) have been widely used in natural langua...
research
12/01/2019

Not All Attention Is Needed: Gated Attention Network for Sequence Data

Although deep neural networks generally have fixed network structures, t...
research
04/17/2019

DeepNovoV2: Better de novo peptide sequencing with deep learning

We introduce DeepNovoV2, the state-of-the-art neural networks based mode...
research
09/12/2021

Efficient Re-parameterization Residual Attention Network For Nonhomogeneous Image Dehazing

This paper proposes an end-to-end Efficient Re-parameterizationResidual ...
research
02/20/2021

Hard-Attention for Scalable Image Classification

Deep neural networks (DNNs) are typically optimized for a specific input...
research
12/20/2019

ET-USB: Transformer-Based Sequential Behavior Modeling for Inbound Customer Service

Deep-Learning based models with attention mechanism has achieved excepti...

Please sign up or login with your details

Forgot password? Click here to reset