Bounding the Probability of Error for High Precision Recognition

by   Andrew Kae, et al.

We consider models for which it is important, early in processing, to estimate some variables with high precision, but perhaps at relatively low rates of recall. If some variables can be identified with near certainty, then they can be conditioned upon, allowing further inference to be done efficiently. Specifically, we consider optical character recognition (OCR) systems that can be bootstrapped by identifying a subset of correctly translated document words with very high precision. This "clean set" is subsequently used as document-specific training data. While many current OCR systems produce measures of confidence for the identity of each letter or word, thresholding these confidence values, even at very high values, still produces some errors. We introduce a novel technique for identifying a set of correct words with very high precision. Rather than estimating posterior probabilities, we bound the probability that any given word is incorrect under very general assumptions, using an approximate worst case analysis. As a result, the parameters of the model are nearly irrelevant, and we are able to identify a subset of words, even in noisy documents, of which we are highly confident. On our set of 10 documents, we are able to identify about 6 average without making a single error. This ability to produce word lists with very high precision allows us to use a family of models which depends upon such clean word lists.


page 1

page 2

page 3

page 4


LEXpander: applying colexification networks to automated lexicon expansion

Recent approaches to text analysis from social media and other corpora r...

Omnifont Persian OCR System Using Primitives

In this paper, we introduce a model-based omnifont Persian OCR system. T...

A plug-in approach to maximising precision at the top and recall at the top

For information retrieval and binary classification, we show that precis...

Fused Text Recogniser and Deep Embeddings Improve Word Recognition and Retrieval

Recognition and retrieval of textual content from the large document col...

Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship

We adapt the Higher Criticism (HC) goodness-of-fit test to detect change...

Directional Decision Lists

In this paper we introduce a novel family of decision lists consisting o...

Finding the optimal human strategy for Wordle using maximum correct letter probabilities and reinforcement learning

Wordle is an online word puzzle game that gained viral popularity in Jan...

Please sign up or login with your details

Forgot password? Click here to reset