Online Deep Learning from Doubly-Streaming Data

by   Heng Lian, et al.

This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. The challenges of this problem are two folds: 1) Data samples ceaselessly flowing in may carry shifted patterns over time, requiring learners to update hence adapt on-the-fly. 2) Newly emerging features are described by very few samples, resulting in weak learners that tend to make error predictions. A plausible idea to overcome the challenges is to establish relationship between the pre-and-post evolving feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional media streams with complex feature interplay, which suffers an tradeoff between onlineness (biasing shallow learners) and expressiveness(requiring deep learners). Motivated by this, we propose a novel OLD^3S paradigm, where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building intermediate feature mapping relationship. A key trait of OLD^3S is to treat the model capacity as a learnable semantics, yields optimal model depth and parameters jointly, in accordance with the complexity and non-linearity of the input data streams in an online fashion. Both theoretical analyses and empirical studies substantiate the viability and effectiveness of our proposal.


page 1

page 2

page 3

page 4


Learning with Feature Evolvable Streams

Learning with streaming data has attracted much attention during the pas...

Prediction with Unpredictable Feature Evolution

Feature space can change or evolve when learning with streaming data. Se...

Rebalancing Learning on Evolving Data Streams

Nowadays, every device connected to the Internet generates an ever-growi...

Online Deep Metric Learning via Mutual Distillation

Deep metric learning aims to transform input data into an embedding spac...

Storage Fit Learning with Feature Evolvable Streams

Feature evolvable learning has been widely studied in recent years where...

Adaptive Deep Forest for Online Learning from Drifting Data Streams

Learning from data streams is among the most vital fields of contemporar...

On the Intrinsic Limits to Representationally-Adaptive Machine-Learning

Online learning is a familiar problem setting within Machine-Learning in...

Please sign up or login with your details

Forgot password? Click here to reset