A Survey on Transfer Learning

This survey categorizes and reviews the latest progress on transfer learning in different data mining and machine learning areas. Transfer learning emerged to overcome the problem that assumes training and test set data can’t be in different feature space and should have the same distribution. Therefore, models don’t need to be rebuilt after using new data which is expensive and sometimes impossible. The main focus of the paper is on transfer learning for classification, regression and clustering problems.

Traditional supervised and semi-supervised methods can remove noisy data or use cost-sensitive learning but most of them assume that the labeled and unlabeled data have the same distributions. However, transfer learning lets the training and test data distribution to be different. Traditional machine learning techniques try to learn each task from scratch, while transfer learning techniques try to transfer the knowledge from some previous task to a target task when the latter has fewer high-quality training data.

In transfer learning, we have the following three main research issues: 1) what to transfer, 2) how to transfer, 3) when to transfer. “What to transfer” asks which part of knowledge can be

transferred across domains or tasks. “how to transfer “corresponds to learning algorithms need to be developed to transfer the knowledge “When to transfer” asks in which situations, transferring skills should be done.

categorize transfer learning under three subsettings, inductive transfer learning, transductive transfer learning, and unsupervised transfer learning, based on different situations between the source and target domains and tasks.