Rate-Adaptive Neural Networks for Spatial Multiplexers

by   Suhas Lohit, et al.

In resource-constrained environments, one can employ spatial multiplexing cameras to acquire a small number of measurements of a scene, and perform effective reconstruction or high-level inference using purely data-driven neural networks. However, once trained, the measurement matrix and the network are valid only for a single measurement rate (MR) chosen at training time. To overcome this drawback, we answer the following question: How can we jointly design the measurement operator and the reconstruction/inference network so that the system can operate over a range of MRs? To this end, we present a novel training algorithm, for learning rate-adaptive networks. Using standard datasets, we demonstrate that, when tested over a range of MRs, a rate-adaptive network can provide high quality reconstruction over a the entire range, resulting in up to about 15 dB improvement over previous methods, where the network is valid for only one MR. We demonstrate the effectiveness of our approach for sample-efficient object tracking where video frames are acquired at dynamically varying MRs. We also extend this algorithm to learn the measurement operator in conjunction with image recognition networks. Experiments on MNIST and CIFAR-10 confirm the applicability of our algorithm to different tasks.


page 2

page 3

page 5

page 6

page 7

page 8

page 10


Fast Integral Image Estimation at 1

We propose a framework called ReFInE to directly obtain integral image e...

Adaptive Measurement Network for CS Image Reconstruction

Conventional compressive sensing (CS) reconstruction is very slow for it...

Compressed domain image classification using a multi-rate neural network

Compressed domain image classification aims to directly perform classifi...

Reconstruction from Periodic Nonlinearities, With Applications to HDR Imaging

We consider the problem of reconstructing signals and images from period...

Data driven regularization by projection

We demonstrate that regularisation by projection and variational regular...

Measurement uncertainty and unicity of single number quantities describing the spatial decay of speech level in open-plan offices

The ISO 3382-3 standard (2012) defines single number quantities (SNQs) w...

Please sign up or login with your details

Forgot password? Click here to reset