Building "Large" AI Models for Finance

Abstract: Goal-Oriented Portfolio Management Through Transformer-Based Reinforcement Learning

We directly optimize arbitrary objectives of portfolio management via Transformer-based reinforcement learning---an alternative framework to conventional supervised-learning paradigms that routinely entail first-step estimations of return distributions or risk premia. We further develop multi-sequence, self-attention-based neural-networks tailored for the distinguishing features of financial big data, while allowing interactions with the market states and training without labels. The resulting AlphaPortfolio model, as the first ``large'' AI models in finance, yields superior out-of-sample performances (e.g., Sharpe ratio above two and over 13% risk-adjusted alpha with monthly rebalancing) that are robust under various market conditions and economic restrictions (e.g., exclusion of small and illiquid stocks). We further demonstrate AlphaPortfolio's flexibility to incorporate transaction costs, state interactions, and alternative objectives, before applying polynomial-feature-sensitivity analysis to uncover key drivers of investment performance, including their rotation and nonlinearity. Overall, we highlight the utility of deep reinforcement learning in finance and ``economic distillation'' for interpreting large models, while demonstrating how economic objectives can effectively guide the training of AI models. Time permitting, we discuss how the framework can be extended to provide a data-driven-robust-control approach to studying corporate finance, complementing reduced-form models and structural estimations.