Time Series DFT Signals Clustering

414 views Asked by At

I have a number of time series data sets, which I want to transform to dft signals in order to reduce dimensionality. After transforming to dft, I want to cluster the resulting dft data sets using k-means algorithm.

Since dft signals contain an imaginary number how can one cluster them?

2

There are 2 answers

2
Has QUIT--Anony-Mousse On BEST ANSWER

You could simply treat the imaginary part as another component in your vectors. In other applications, you will want to ignore it!

But you'll be facing other, more severe challenges.

Data mining, and clustering in particular, rarely is as easy as appliyng function a (dft) and function b (k-means) and then you have the result, hooray. Sorry - that is not how exploratory data mining works.

First of all, for many time series, DFT will not be helpful at all. On others, you will first have to do appropriate resampling, or segmentation, or get rid of uninteresting effects such as seasonality. Even if DFT works, it may emphasize artifacts such as the sampling frequency or some interferences.

And then you'll run into one major problem: k-means is based on the assumption that all attributes have the same importance. And DFT is based on the very opposite idea: the first components capture most of the signal, the later ones only minor deviations from it (and that is the very motivation for using this as dimensionality reduction). So based on this intuition, you maybe never should apply k-means on DFT coefficients at all. At the same time, data-mining repeatedly has shown that appfoaches that are "statistical nonsense" can nevertheless provide useful results... so you can try, but verify your resultd with care, and avoid being too enthusiastic or optimistic.

0
eInfochips - Komal C On

With the help of FFT, it converts dataset into dft signals. It helps to calculates DFT for each small data set.