Please use this identifier to cite or link to this item: http://hdl.handle.net/1783.1/6715

Multi-task learning using generalized t process

Authors Zhang, Yu
Yeung, Dit-Yan
Issue Date 2010
Source Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), Sardinia, Italy, 13-15 May 2010, pp.964-971
Summary Multi-task learning seeks to improve the generalization performance of a learning task with the help of other related learning tasks. Among the multi-task learning methods proposed thus far, Bonilla et al.’s method (Bonilla et al., 2008) provides a novel multi-task extension of Gaussian process (GP) by using a task covariance matrix to model the relationships between tasks. However, learning the task covariance matrix directly has both computational and representational drawbacks. In this paper, we propose a Bayesian extension by modeling the task covariance matrix as a random matrix with an inverse-Wishart prior and integrating it out to achieve Bayesian model averaging. To make the computation feasible, we first give an alternative weight-space view of Bonilla et al.’s multi-task GP model and then integrate out the task covariance matrix in the model, leading to a multi-task generalized t process (MTGTP). For the likelihood, we use a generalized t noise model which, together with the generalized t process prior, brings about the robustness advantage as well as an analytical form for the marginal likelihood. In order to specify the inverse-Wishart prior, we use the maximum mean discrepancy (MMD) statistic to estimate the parameter matrix of the inverse-Wishart prior. Moreover, we investigate some theoretical properties of MTGTP, such as its asymptotic analysis and learning curve. Comparative experimental studies on two common multi-task learning applications show very promising results.
Subjects
Language English
Format Conference paper
Access
Files in this item:
File Description Size Format
Multitask.pdf 1.6 MB Adobe PDF