一种抽样二阶随机算法

数学理论与应用 ›› 2022, Vol. 42 ›› Issue (1): 92-103.

一种抽样二阶随机算法

王静, 王湘美^*

贵州大学数学与统计学院, 贵阳, 550025

出版日期:2022-03-31 发布日期:2022-03-31
通讯作者: 王湘美 (1972−), 副教授, 博士, 从事黎曼流形上凸优化算法研究; E−mail: xmwang2@gzu.edu.cn
基金资助:
国家自然科学基金项目 (11661019) 资助

A Sampled Secondorder Stochastic Algorithm

Wang Jing, Wang Xiangmei^*

College of Mathematics and Statistics, Guizhou University, Guiyang 550025, China

Online:2022-03-31 Published:2022-03-31

摘要/Abstract

摘要：

本文针对机器学习中的大规模优化问题,将Lissa算法和SSN算法结合起来，给出一种抽样二阶随机算法(SSN-Lissa)，并在目标函数是光滑且强凸的条件下, 证明该算法的线性收敛性. 数值例子表明SSN-Lissa算法比Lissa算法和SSN算法更有效.

关键词:

机器学习, 凸优化 , 一阶随机算法, 二阶随机算法, 泰勒近似

Abstract:

In this paper a sampled second-order stochastic algorithm (SSN-Lissa) is presented by combining the Lissa and SSN algorithms for large-scale machine learning optimizations. The linear convergence of the algorithm is established under the assumption that the objective functions are smooth and strongly convex. Numerical experiments show that the new algorithm is more effective than the Lissa and SSN.

Key words:

Machine learning, Convex optimization, First-order stochastic method, Second-order stochastic method, Taylor approximation

王静, 王湘美.

一种抽样二阶随机算法

[J]. 数学理论与应用, 2022, 42(1): 92-103.

Wang Jing, Wang Xiangmei.

A Sampled Secondorder Stochastic Algorithm

[J]. Mathematical Theory and Applications, 2022, 42(1): 92-103.