著者

Akisato Kimura Masashi Sugiyama Hitoshi Sakano Hirokazu Kameoka

雑誌

情報処理学会論文誌数理モデル化と応用(TOM) (ISSN:18827780)

巻号頁・発行日

vol.6, no.1, pp.136-145, 2013-03-12

---

It is well known that dimensionality reduction based on multivariate analysis methods and their kernelized extensions can be formulated as generalized eigenvalue problems of scatter matrices, Gram matrices or their augmented matrices. This paper provides a generic and theoretical framework of multivariate analysis introducing a new expression for scatter matrices and Gram matrices, called Generalized Pairwise Expression (GPE). This expression is quite compact but highly powerful. The framework includes not only (1) the traditional multivariate analysis methods but also (2) several regularization techniques, (3) localization techniques, (4) clustering methods based on generalized eigenvalue problems, and (5) their semi-supervised extensions. This paper also presents a methodology for designing a desired multivariate analysis method from the proposed framework. The methodology is quite simple: adopting the above mentioned special cases as templates, and generating a new method by combining these templates appropriately. Through this methodology, we can freely design various tailor-made methods for specific purposes or domains.

著者

Tuan Duong NGUYEN Marthinus Christoffel DU PLESSIS Takafumi KANAMORI Masashi SUGIYAMA

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE Transactions on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E97.D, no.7, pp.1822-1829, 2014 (Released:2014-07-01)

参考文献数

22

被引用文献数

2 4

---

We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.

著者

Masashi SUGIYAMA

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE Transactions on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E93.D, no.10, pp.2690-2701, 2010-10-01 (Released:2010-10-01)

参考文献数

36

被引用文献数

20 36

---

Kernel logistic regression (KLR) is a powerful and flexible classification algorithm, which possesses an ability to provide the confidence of class prediction. However, its training—typically carried out by (quasi-)Newton methods—is rather time-consuming. In this paper, we propose an alternative probabilistic classification algorithm called Least-Squares Probabilistic Classifier (LSPC). KLR models the class-posterior probability by the log-linear combination of kernel functions and its parameters are learned by (regularized) maximum likelihood. In contrast, LSPC employs the linear combination of kernel functions and its parameters are learned by regularized least-squares fitting of the true class-posterior probability. Thanks to this linear regularized least-squares formulation, the solution of LSPC can be computed analytically just by solving a regularized system of linear equations in a class-wise manner. Thus LSPC is computationally very efficient and numerically stable. Through experiments, we show that the computation time of LSPC is faster than that of KLR by two orders of magnitude, with comparable classification accuracy.

著者

Masashi SUGIYAMA Taiji SUZUKI

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE Transactions on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E94.D, no.6, pp.1333-1336, 2011-06-01 (Released:2011-06-01)

参考文献数

10

被引用文献数

6 10

---

Identifying the statistical independence of random variables is one of the important tasks in statistical data analysis. In this paper, we propose a novel non-parametric independence test based on a least-squares density ratio estimator. Our method, called least-squares independence test (LSIT), is distribution-free, and thus it is more flexible than parametric approaches. Furthermore, it is equipped with a model selection procedure based on cross-validation. This is a significant advantage over existing non-parametric approaches which often require manual parameter tuning. The usefulness of the proposed method is shown through numerical experiments.

著者

Masashi SUGIYAMA Makoto YAMADA

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE Transactions on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E95.D, no.10, pp.2564-2567, 2012-10-01 (Released:2012-10-01)

参考文献数

27

被引用文献数

2 9

---

The Hilbert-Schmidt independence criterion (HSIC) is a kernel-based statistical independence measure that can be computed very efficiently. However, it requires us to determine the kernel parameters heuristically because no objective model selection method is available. Least-squares mutual information (LSMI) is another statistical independence measure that is based on direct density-ratio estimation. Although LSMI is computationally more expensive than HSIC, LSMI is equipped with cross-validation, and thus the kernel parameter can be determined objectively. In this paper, we show that HSIC can actually be regarded as an approximation to LSMI, which allows us to utilize cross-validation of LSMI for determining kernel parameters in HSIC. Consequently, both computational efficiency and cross-validation can be achieved.

著者

Masashi SUGIYAMA Ichiro TAKEUCHI Taiji SUZUKI Takafumi KANAMORI Hirotaka HACHIYA Daisuke OKANOHARA

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E93-D, no.3, pp.583-594, 2010-03-01

---

Estimating the conditional mean of an input-output relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of conditional density estimation that is suitable for multi-dimensional continuous variables. The basic idea of the proposed method is to express the conditional density in terms of the density ratio and the ratio is directly estimated without going through density estimation. Experiments using benchmark and robot transition datasets illustrate the usefulness of the proposed approach.

著者

Akisato Kimura Masashi Sugiyama Takuho Nakano Hirokazu Kameoka Hitoshi Sakano Eisaku Maeda Katsuhiko Ishiguro

雑誌

情報処理学会論文誌数理モデル化と応用(TOM) (ISSN:18827780)

巻号頁・発行日

vol.6, no.1, pp.128-135, 2013-03-12

---

Canonical correlation analysis (CCA) is a powerful tool for analyzing multi-dimensional paired data. However, CCA tends to perform poorly when the number of paired samples is limited, which is often the case in practice. To cope with this problem, we propose a semi-supervised variant of CCA named SemiCCA that allows us to incorporate additional unpaired samples for mitigating overfitting. Advantages of the proposed method over previously proposed methods are its computational efficiency and intuitive operationality: it smoothly bridges the generalized eigenvalue problems of CCA and principal component analysis (PCA), and thus its solution can be computed efficiently just by solving a single eigenvalue problem as the original CCA.

著者

Hisashi KASHIMA Tsuyoshi IDE Tsuyoshi KATO Masashi SUGIYAMA

出版者

The Institute of Electronics, Information and Communication Engineers

雑誌

IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)

巻号頁・発行日

vol.E92-D, no.7, pp.1338-1353, 2009-07-01

被引用文献数

16

---

Kernel methods such as the support vector machine are one of the most successful algorithms in modern machine learning. Their advantage is that linear algorithms are extended to non-linear scenarios in a straightforward way by the use of the kernel trick. However, naive use of kernel methods is computationally expensive since the computational complexity typically scales cubically with respect to the number of training samples. In this article, we review recent advances in the kernel methods, with emphasis on scalability for massive problems.