Distributed super point cardinality estimation under sliding time window for high speed network
share
Super point is a special kind of host whose cardinality, the number of contacting hosts in a certain period, is bigger than a threshold. Super point cardinality estimation plays important roles in network field. This paper proposes a super point cardinality estimation algorithm under sliding time window. To maintain the state of previous hosts with few updating operations, a novel counter, asynchronous time stamp (AT), is proposed. For a sliding time window containing k time slices, AT only needs to be updated every k time slices at the cost of 1 more bit than a previous state-of-art counter which requires log_2(k+1) bits but updates every time slice. Fewer updating operations mean that more AT could be contained to acquire higher accuracy in real-time. This paper also devises a novel reversible hash function scheme to restore super point from a pool of AT. Experiments on several real-world network traffic illustrate that the algorithm proposed in this paper could detect super points and estimate their cardinalities under sliding time window in real time.
READ FULL TEXT