• Financial Reinforcement Learning (FinRL)

Research history and contributions

FinRL is a collaborative open-source research effort in financial reinforcement learning. Its research lineage includes the 2018 workshop study Practical Deep Reinforcement Learning Approach for Stock Trading, which investigated a deep reinforcement learning agent using historical data for 30 stocks. Xiao-Yang Liu is an author of this work and the subsequent FinRL papers listed below.

From trading agents to a reusable framework

The 2020 FinRL library paper introduced reusable market environments, trading agents, tutorials, and backtesting tools for automated stock trading and portfolio allocation. Xiao-Yang Liu and Hongyang Yang are identified as equal contributors in the paper. The 2021 ICAIF paper extended the framework presentation around a modular three-layer architecture: market environments, deep reinforcement learning agents, and financial applications. Together, these works document contributions to environment design, strategy development, evaluation, customization, and reproducible experimentation.

Market environments and benchmarks

FinRL-Meta, published in the NeurIPS 2022 Datasets and Benchmarks Track, expanded this direction toward data-driven market environments and benchmarks. It described a DataOps approach to turning financial datasets into Gym-style environments, alongside demonstrations and educational resources. This research record preserves the original papers and their authorship while providing links to evolving community software.

Software and documentation

Explore the FinRL and FinRL-Meta repositories and FinRL documentation for code, examples, and implementation details. These external resources evolve separately from the historical publication record maintained here.

FinRL repository

FinRL-Meta repository

FinRL documentation

FinRL library paper (2020): Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. Liu and Yang contributed equally.

Competitions and community resources

We've been developing Financial Reinforcement Learning (FinRL), following the Model Openness Framework (MOF).

IEEE, FinAI Contest 2025, FinRL Competition 2025,  Github repo,  Tutorial Web. 

IEEE IDS 2025, Special Track: Financial Reinforcement Learning and Foundation Models (FinRLFM)

ACM ICAIF,  FinRL Competition 2023,  FinRL Competition 2024

Historical development note: The earlier FinAI Lab page recorded that the lab had been inactive on the AI4Finance FinRL GitHub repository since November 2023. This records lab involvement at that time; external community software evolves separately.

Join us: Be part of a collaborative effort, your expertise and enthusiasm will shape the future of open finance.

Open Financial Reinforcement Learning (FinRL)

• Xiao-Yang Liu, Z. Xia, H. Yang, J. Gao, D. Zha, M. Zhu, C.D. Wang, Z. Wang, and J. Guo. Dynamic Datasets and Market Environments for Financial Reinforcement Learning. Machine Learning Journal, Springer Nature, 2023.
• Xiao-Yang Liu, Z. Xia, J. Rui, J. Gao, H. Yang, M. Zhu, C.D. Wang, Z. Wang, and J. Guo. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 2022.
• Xiao-Yang Liu, Hongyang Yang, J. Gao, and C. D. Wang. FinRL: Deep reinforcement learning framework to automate trading in quantitative finance. ACM International Conference on AI in Finance (ICAIF), 2021.
• Xiao-Yang Liu, Z. Xiong, S. Zhong, Hongyang Yang, and Anwar Walid. Practical deep reinforcement learning approach for stock trading. Workshop on Challenges and Opportunities for AI in Financial Services, NeurIPS 2018.
• Keyi Wang, Nikolaus Holzer, Ziyi Xia, Yupeng Cao, Jiechao Gao, Anwar Walid, Kairong Xiao, Xiao-Yang Liu Yanglet. FinRL Contests: Benchmarking Data-driven Financial Reinforcement Learning Agents. AI for Engineering, Wiley, 2025.
• Kent Wu, Ziyi Xia, Shuaiyu Chen, and Xiao-Yang Liu. Curriculum Learning from Smart Retail Investors: Towards Financial Open-endedness. Agent Learning in Open-Endedness Workshop, NeurIPS 2023.
• Berend Jelmer D. Gort, Xiao-Yang Liu*, J. Gao, Shuaiyu Chen, and Christina Dan Wang. Deep reinforcement learning for cryptocurrency trading: Practical approach to address backtest overfitting. ACM International Conference on AI in Finance (ICAIF), Workshop on Benchmarks for AI in Finance, 2022; Also appeared at AAAI’23 Bridge on AI for Financial Services, 2023.  FinRL-Crypto.
• M. Guan and Xiao-Yang Liu*. Explainable deep reinforcement learning for portfolio management: An empirical approach. ACM International Conference on AI in Finance (ICAIF),  2021.
• Hongyang Yang, Xiao-Yang Liu*, S. Zhong, and Anwar Walid. Deep reinforcement learning for automated stock trading: an ensemble strategy. ACM International Conference on AI in Finance (ICAIF),  2020.

High-Performance GPU Optimization for FinRL

• Keyi Wang, Kairong Xiao, and Xiao-Yang Liu Yanglet. Parallel Market Environments for FinRL Contests. IEEE 11th International Conference on Intelligent Data and Security (IDS), pp. 65-67. IEEE, 2025.
• Xiao-Yang Liu, Zechu Li, Zhuoran Yang, Jiahao Zheng, Zhaoran Wang, Anwar Walid, Jian Guo, and Michael I. Jordan. ElegantRL-Podracer: Scalable and elastic library for cloud-native deep reinforcement learning. Deep Reinforcement Learning Workshop, NeurIPS, 2021.
• Zechu Li, Xiao-Yang Liu, Jiahao Zheng, Zhaoran Wang, Anwar Walid, Jian Guo. FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance. ACM international conference on AI in finance. 2021.
• Nikolaus Holzer, Keyi Wang, Kairong Xiao, Xiao-Yang Liu Yanglet. Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024, 2025.