Faster Binary Embeddings for Preserving Euclidean Distances

10/01/2020
∙
by   Jinjie Zhang, et al.
∙
0
∙

We propose a fast, distance-preserving, binary embedding algorithm to transform a high-dimensional dataset 𝒯⊆ℝ^n into binary sequences in the cube {± 1}^m. When 𝒯 consists of well-spread (i.e., non-sparse) vectors, our embedding method applies a stable noise-shaping quantization scheme to A x where A∈ℝ^m× n is a sparse Gaussian random matrix. This contrasts with most binary embedding methods, which usually use x↦sign(Ax) for the embedding. Moreover, we show that Euclidean distances among the elements of 𝒯 are approximated by the ℓ_1 norm on the images of {± 1}^m under a fast linear transformation. This again contrasts with standard methods, where the Hamming distance is used instead. Our method is both fast and memory efficient, with time complexity O(m) and space complexity O(m). Further, we prove that the method is accurate and its associated error is comparable to that of a continuous valued Johnson-Lindenstrauss embedding plus a quantization error that admits a polynomial decay as the embedding dimension m increases. Thus the length of the binary codes required to achieve a desired accuracy is quite small, and we show it can even be compressed further without compromising the accuracy. To illustrate our results, we test the proposed method on natural images and show that it achieves strong performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
∙ 01/26/2018

Fast binary embeddings, and quantized compressed sensing with structured matrices

This paper deals with two related problems, namely distance-preserving b...
research
∙ 01/24/2016

Fast Binary Embedding via Circulant Downsampled Matrix -- A Data-Independent Approach

Binary embedding of high-dimensional data aims to produce low-dimensiona...
research
∙ 09/17/2020

Binarized Johnson-Lindenstrauss embeddings

We consider the problem of encoding a set of vectors into a minimal numb...
research
∙ 04/08/2022

Fast metric embedding into the Hamming cube

We consider the problem of embedding a subset of ℝ^n into a low-dimensio...
research
∙ 05/13/2014

Circulant Binary Embedding

Binary embedding of high-dimensional data requires long codes to preserv...
research
∙ 05/24/2022

Embedding Neighborhoods Simultaneously t-SNE (ENS-t-SNE)

We propose an algorithm for visualizing a dataset by embedding it in 3-d...
research
∙ 01/11/2020

Embedding Compression with Isotropic Iterative Quantization

Continuous representation of words is a standard component in deep learn...

Please sign up or login with your details

Forgot password? Click here to reset