Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization
Jiacai Liu and Chaojie Wang and Chris Yuhao Liu and Liang Zeng and Rui Yan and Yiwen Sun and Yang Liu and Yahui Zhou. arXiv:2412.18279, 2024.
CV
Education
Ph.D in Reinforcement Learning, Fudan University, (2023-2027)
Supervisor : Ke Wei
M.S. in Statistics, Fudan University, (2021 - 2023)
B.M. in Shanghai University of Finance and Economic, (2017 - 2021)
I rank 1st in GaoKao in Guangyuan City, Sichuan Province
Work experience
- Game AI Intern in NETEASE GAMES AI LAB, 2023.02 - 2023.07.
- Research on the sample efficiency of on-policy algorithms (such as PPO).
- Research on the Imitation Learning with GAN series algorithms.
- Alignment Intern in Skywork AI LAB.
- Research on the post-training.
Publications
$phi$-Update: A Class of Policy Update Methods with Policy Convergence Guarante
Wenye Li, Jiacai Liu and Ke Wei. $\phi$-Update: A Class of Policy Update Methods with Policy Convergence Guarante. In The Thirteen International Conference on Learning Representations, ICLR 2025, Singapore.
Elementary Analysis of Policy Gradient Methods
Jiacai Liu, Wenye Li, and Ke Wei. Elementary Analysis of Policy Gradient Methods. arXiv:2404.03372, 2024.
On the Convergence of Projected Policy Gradient for Any Constant Step Sizes
Jiacai Liu, Wenye Li, and Ke Wei. On the Convergence of Projected Policy Gradient for Any Constant Step Sizes. arXiv:2311.0110, 2023.
On the Linear Convergence of Policy Gradient under Hadamard Parametrization
Jiacai Liu, Jinchi Chen, and Ke Wei. On the Linear Convergence of Policy Gradient under Hadamard Parameterization. arXiv:2305.19575, 2023.
Talks
Some Progress on the Convergence of Policy Gradient Methods
Remote research talk at Csaba Szepesvari's research group, University of Alberta
Projected Policy Gradient Converges in a Finite Number of Iterations
Presentation at Applied Math Ph.D. Seminar, Rm1801,Guanghua East Tower
Awards
- 2023.06, IJCAI 2023 AI Olympics Competition, Champion.
- 2021.06, Chinese Collegiate Computing Competition, 1st Prize.
- 2020.06, Chinese Collegiate Computing Competition, 3rd Prize.
- 2016.03, China National High School Mathematics League, Sichuan Province, 2nd Prize.