A unifying framework for detecting outliers and change points from time series

Jun Ichi Takeuchi, Kenji Yamanishi

Research output: Contribution to journalArticle

180 Citations (Scopus)

Abstract

We are concerned with the issue of detecting outliers and change points from time series. In the area of data mining, there have been increased interest in these issues since outlier detection is related to fraud detection, rare event discovery, etc., while change-point detection is related to event/trend change detection, activity monitoring, etc. Although, in most previous work, outlier detection and change point detection have not been related explicitly, this paper presents a unifying framework for dealing with both of them. In this framework, a probabilistic model of time series is incrementally learned using an online discounting learning algorithm, which can track a drifting data source adaptively by forgetting out-of-date statistics gradually. A score for any given data is calculated in terms of its deviation from the learned model, with a higher score indicating a high possibility of being an outlier. By taking an average of the scores over a window of a fixed length and sliding the window, we may obtain a new time series consisting of moving-averaged scores. Change point detection is then reduced to the issue of detecting outliers in that time series. We compare the performance of our framework with those of conventional methods to demonstrate its validity through simulation and experimental applications to incidents detection in network security.

Original languageEnglish
Pages (from-to)482-492
Number of pages11
JournalIEEE Transactions on Knowledge and Data Engineering
Volume18
Issue number4
DOIs
Publication statusPublished - Apr 1 2006
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'A unifying framework for detecting outliers and change points from time series'. Together they form a unique fingerprint.

  • Cite this