Backtesting AI Portfolios

In the paper I want to present, we address look-ahead bias in financial analysis using LLMs. We propose a straightforward methodology to quantify model memorization by querying LLMs without contextual prompts, establishing clear bias benchmarks. Empirical results reveal that look-ahead bias is most pronounced at lower data frequencies and in aggregate indices, requiring exclusion of substantial observations for accurate backtesting. Conversely, higher-frequency and granular data exhibit minimal memorization. Smaller models show significantly reduced bias due to limited parameter capacity. Our method enables precise identification and exclusion of biased observations, preserving statistical power and enhancing backtesting reliability, providing practical guidelines for financial analysis using advanced LLMs.