Andrzej Cichocki and Anh Huy Phan
RIKEN - Brain Science Institute, JAPAN
http://www.bsp.brain.riken.jp/~cia
Multi-way Array (Tensor) Factorizations and Decompositions and their Potential Applications
Recent advances in developing high-spatial density array EEG, MEG and fMRI have called for multi-dimensional signal processing techniques (referred here as multi-way analysis (MWA), multiway-array (tensor) factorization/decompositions or dynamic tensor analysis (DTA)) to be employed for brain signals analysis.
Standard matrix factorizations, and component analysis like PCA, SVD, ICA, SCA and NMF and their variants, are invaluable tools for blind source separation, feature selection, dimensionality reduction, noise reduction, and data mining. However, they have only two modes or two-way representations (e.g., channels and time) and therefore have severe intrinsic limitations.
In comprehensive multichannel and multimodal brain data structures often contain higher-order ways (modes) such as trials, tasks conditions, subjects, and groups in addition to the intrinsic dimensions of space, time and frequency. In fact, specific mental tasks or stimuli are often presented repeatedly in a sequence of trials leading to a large volume stream of data encompassing many dimensions: Channels (space), time-frequency, trials, and conditions. For such kind of data two-way matrix factorizations (e.g., regular ICA, NMF) or "flat-world view" may be insufficient for analysis of complex data. In order, to obtain more natural representations of the original multidimensional data structure, it is necessary to use tensor decomposition and factorization approaches since additional dimensions or modes can be retained only in multi-linear models to produce structures that are unique and which admit interpretations that are neurophysiologically meaningful.
The most promising decompositions/factorization for N-th order tensors are the Tucker tensor decomposition and the more restricted PARAFAC and slice oriented decomposition (SOD) models. These models can be viewed as generalizations of the 2D factor/component analysis (e.g., PCA, ICA, NMF) for data with more than two modalities by imposing some additional constraints such as orthogonality, mutual independence, non-negativity and/or sparsity of hidden factors. Especially, nonnegativity constraints leads to Nonnegative matrix factorization (NMF), Nonnegative Tensor Factorization (NTF), and nonnegative Tucker decomposition (NTD), which in conjunction with sparse coding have recently been given much attention due to easy interpretable and meaningful representation. The advantages of the sparse NTF/NMF/NTD based feature extraction approach lies in its capability to yield components which are common across the space, time and/or frequency domains and at the same time discriminative across different conditions without prior knowledge of the discriminative frequency bands and temporal windows for a specific subject. In practice, various oscillatory activities might overlap, but the sparse and nonnegative representations of the tensor data given e.g., by the time-frequency-space transformation enables the decompositions to isolate each oscillatory behavior well even when these activities are not well-separated in the space-time domain alone.
In this talk we will discuss a family of efficient algorithms for NMF/NTF/NTD/SOD as well as sparse nonnegative coding and representation that have many potential applications in computational neuroscience, multi-sensory processing, compressed sensing and multidimensional data analysis. We have developed a class of optimized local algorithms which are referred as Hierarchical Alternating Least Squares (HALS) algorithms. For these purposes, we have performed sequential constrained minimization on a set of squared Euclidean distances. We then extend this approach to robust cost functions using the alpha and beta divergences and derive flexible update rules. The discussed algorithms are locally stable and work well for NMF-based blind source separation (BSS) not only for the over-determined case but also for an under-determined (over-complete) case (i.e., for a system which has less sensors than sources) if data are sufficiently sparse. The NMF learning rules are extended and generalized for higher order nonnegative tensor factorization (NTF). Moreover, these algorithms can be tuned to different noise statistics by adjusting a single parameter. Extensive experimental results confirm the accuracy and computational performance of the developed algorithms, especially, with usage of multi-layer hierarchical NMF approach.
We believe that higher-order tensor decomposition is a promising approach for analysis of complex data especially analysis of brain signals because the factors are physically meaningful (for example, scalp plots, temporal patterns, trial-to-trial variability, and so forth) and researchers can interpret them relatively easily from a physical or a physiological context. We are also typically dealing with very large, high-dimensional data matrices in which we need to efficiently reduce the number of parameters for estimation and multiway array decomposition is an efficient way to do this. Finally, tensor decompositions can impose many objectives like discrimination, multiway independence, smoothness, sparse representation, and multiway clustering.
Biographies
Andrzej CICHOCKI received the M.Sc. (with honors), Ph.D. and Dr.Sc. (Habilitation) degrees, all in electrical engineering from Warsaw University of Technology (Poland). Since 1972, he has been with the Institute of Theory of Electrical Engineering, Measurement and Information Systems, Faculty of Electrical Engineering at the Warsaw University of Technology, where he obtain a title of a full Professor in 1995. He spent several years at University Erlangen-Nuerenberg, at the Chair of Applied and Theoretical Electrical Engineering directed by Professor Rolf Unbehauen, as an Alexander-von-Humboldt Research Fellow and Guest Professor. In 1995-1997 he was a team leader of the laboratory for Artificial Brain Systems, at Frontier Research Program RIKEN, in the Brain Information Processing Group. He is currently the head of the laboratory for Advanced Brain Signal Processing, at RIKEN Brain Science Institute (JAPAN) in the Brain-Style Computing Group directed by Professor Shun-ichi Amari. He is co-author of more than 100 technical papers and three internationally recognized monographs (two of them translated to Chinese): Adaptive Blind Signal and Image Processing (Wiley, April 2003 -revised edition); CMOS Switched-Capacitor and Continuous-Time Integrated Circuits and Systems (Springer-Verlag, 1989); Neural Networks for Optimizations and Signal Processing (Teubner-Wiley,1994).
Anh Huy PHAN graduated in 2002 in Electronic Engineering, HoChiMinh City University of Technologies, Vietnam. From 2002 to 2007, he worked as vice head of R&D department, Broadcast Research and Application Center, Vietnam Television (VTV). His group has done science projects for VTV focusing on video enhancement and restoration degraded by random noises (such as Gaussian noise, Poisson noise, impulse noise) and by corruptions (blotch, scratch, hair, dust, shaking, blinking), methods for video stabilization, nonlinear advertisement in digital video, also developed value-added services in digital television such as electronic program guide (EPG) supporting Vietnamese languages, broadcast data and application over DVB-T based on DSMCC and MHP (Multimedia Home Platform) standard.
Since 2007, he has been member of the Laboratory for Advanced Brain Signal Processing, RIKEN, JAPAN and he doing research on blind source separation, nonnegative matrix and tensor factorizations. He is leading coauthor of the new versions of MATLAB toolboxes ICALAB ver. 3.0, NMFLAB and MULTIWAY-LAB.
