We extend our recent work on relevant subtask learning, a new variant of multi-task learning where the goal is to learn a good classifier for a task-of-interest with too few training samples, by exploiting "supplementary data" from several other tasks. It is crucial to model the uncertainty about which of the supplementary data samples are relevant for the task-of-interest, that is, which samples are classified in the same way as in the task-of-interest. We have shown that the problem can be solved by careful mixture modeling: all tasks are modeled as mixtures of relevant and irrelevant samples, and the model for irrelevant samples is flexible enough so that the relevant model only needs to explain the relevant data. Previously we used simple maximum likelihood learning; now we extend the method to variational Bayes inference more suitable for high-dimensional data. We compare the method experimentally to a recent multi-task learning method and two naive methods.

J. Peltonen and S. Kaski belong to Helsinki Institute for Information Technology HIIT and to the Adaptive Informatics Research Centre. The work was supported by the Academy of Finland, decision 207467. Y. Yaslan was supported by the Center for International Mobility CIMO. This work was also supported in part by the PASCAL2 Network of Excellence of the European Community. This publication only reflects the authors' views. We thank Y. Xue for providing SMTL code.