Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

by   Ian J. Goodfellow, et al.

Recognizing arbitrary multi-character text in unconstrained natural photographs is a hard problem. In this paper, we address an equally hard sub-problem in this domain viz. recognizing arbitrary multi-digit numbers from Street View imagery. Traditional approaches to solve this problem typically separate out the localization, segmentation, and recognition steps. In this paper we propose a unified approach that integrates these three steps via the use of a deep convolutional neural network that operates directly on the image pixels. We employ the DistBelief implementation of deep neural networks in order to train large, distributed neural networks on high quality images. We find that the performance of this approach increases with the depth of the convolutional network, with the best performance occurring in the deepest architecture we trained, with eleven hidden layers. We evaluate this approach on the publicly available SVHN dataset and achieve over 96% accuracy in recognizing complete street numbers. We show that on a per-digit recognition task, we improve upon the state-of-the-art, achieving 97.84% accuracy. We also evaluate this approach on an even more challenging dataset generated from Street View imagery containing several tens of millions of street number annotations and achieve over 90% accuracy. To further explore the applicability of the proposed system to broader text recognition tasks, we apply it to synthetic distorted text from reCAPTCHA. reCAPTCHA is one of the most secure reverse turing tests that uses distorted text to distinguish humans from bots. We report a 99.8% accuracy on the hardest category of reCAPTCHA. Our evaluations on both tasks indicate that at specific operating thresholds, the performance of the proposed system is comparable to, and in some cases exceeds, that of human operators.


page 6

page 7

page 12


Deep Convolutional Neural Networks as Generic Feature Extractors

Recognizing objects in natural images is an intricate problem involving ...

Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues

Recognizing scene text is a challenging problem, even more so than the r...

Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks

Unconstrained text recognition is an important computer vision task, fea...

Very Deep Convolutional Networks for Text Classification

The dominant approach for many NLP tasks are recurrent neural networks, ...

Telugu OCR Framework using Deep Learning

In this paper, we address the task of Optical Character Recognition(OCR)...

Deep Learning for Single-View Instance Recognition

Deep learning methods have typically been trained on large datasets in w...

DeepGeo: Photo Localization with Deep Neural Network

In this paper we address the task of determining the geographical locati...

Please sign up or login with your details

Forgot password? Click here to reset