Channel Compression: Rethinking Information Redundancy among Channels in CNN Architecture

by   Jinhua Liang, et al.

Model compression and acceleration are attracting increasing attentions due to the demand for embedded devices and mobile applications. Research on efficient convolutional neural networks (CNNs) aims at removing feature redundancy by decomposing or optimizing the convolutional calculation. In this work, feature redundancy is assumed to exist among channels in CNN architectures, which provides some leeway to boost calculation efficiency. Aiming at channel compression, a novel convolutional construction named compact convolution is proposed to embrace the progress in spatial convolution, channel grouping and pooling operation. Specifically, the depth-wise separable convolution and the point-wise interchannel operation are utilized to efficiently extract features. Different from the existing channel compression method which usually introduces considerable learnable weights, the proposed compact convolution can reduce feature redundancy with no extra parameters. With the point-wise interchannel operation, compact convolutions implicitly squeeze the channel dimension of feature maps. To explore the rules on reducing channel redundancy in neural networks, the comparison is made among different point-wise interchannel operations. Moreover, compact convolutions are extended to tackle with multiple tasks, such as acoustic scene classification, sound event detection and image classification. The extensive experiments demonstrate that our compact convolution not only exhibits high effectiveness in several multimedia tasks, but also can be efficiently implemented by benefiting from parallel computation.


page 1

page 7


Comb Convolution for Efficient Convolutional Architecture

Convolutional neural networks (CNNs) are inherently suffering from massi...

ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions

Convolutional neural networks (CNNs) have shown great capability of solv...

Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution

In natural images, information is conveyed at different frequencies wher...

Orthogonal Convolutional Neural Networks

The instability and feature redundancy in CNNs hinders further performan...

Optimized Vectorizing of Building Structures with Swap: High-Efficiency Convolutional Channel-Swap Hybridization Strategy

The building planar graph reconstruction, a.k.a. footprint reconstructio...

Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphs

Feature spaces in the deep layers of convolutional neural networks (CNNs...

Shift-based Primitives for Efficient Convolutional Neural Networks

We propose a collection of three shift-based primitives for building eff...

Please sign up or login with your details

Forgot password? Click here to reset