Recently, the machine learning team led by Prof. Chen Hong from College of Science of HZAU, published a research paper titled “Multi-task Additive Models for Robust Estimation and Automatic Structure Discovery” in the “34th Conference on Neural Information Processing Systems” (NeurIPS 2020).
Targeting at learning conditional means, most existing interpretable models for high-dimensional data are built under the single-task learning framework, which are often not directly applicable to multi-task data and suffer from performance degradation in data processing with non-Gaussian noise. In particular, traditional group sparse interpretable models rely heavily on the prior information of variable structure. To tackle these problems, this paper, by integrating the mode regression, addictive model and structural penalty item into a multitask and bilevel optimization framework, proposes a new class of addictive models, called Multi-task Additive Models (MAM), which could achieve robust estimation of data with complex noise as well as automatic discovery of potential variable group structure in the data. Considering non-convex and non-smooth characteristics of MAM, researchers propose a class of smooth iterative optimization strategies based on semi-quadratic optimization and forward-backward segmentation algorithms, and give the convergence analysis of the optimization algorithm. Experiments on simulations and coronal mass ejection verify the competitive performance of the constructed model from multiple perspectives, such as the capability of estimation error and structure discovery.
The difference between the multi-task additive model and the traditional group sparse model is shown in Figure 1:
Figure 1: (a) Multi-task data generation process (b) Traditional group sparse model (c) Multi-task addictive Model
Built on the work of Prof. Chen’s early CCF A artificial intelligence conference (H. Chen, X. Wang and H. Huang, NIPS, 2017; X. Wang, H. Chen and H. Huang, NIPS,2017；G. Liu, H. Chen and H. Huang, ICML, 2020) and the work of artificial intelligence top journals (H. Chen, Y. Wang, et al., TNNLS 2020), this research wins a fund by the General Program of National Natural Science Foundation of China. Wang Yingjie, a Ph.D. student from College of Informatics of HZAU is the first author and Prof. Chen serves as the corresponding author, with others contributing their parts to this research including Zheng Feng, an associate researcher from Southern University of Science and Technology and Chen Yanhong, a researcher in the National Space Science Center.
Source: http://news.hzau.edu.cn/2020/1208/59026.shtml
Translated by: Shang Meng
Supervised by: Xie Lujie
