Finance and economics
Machine learning for factor investing
Factor investing uses machine learning to combine many firm characteristics, such as value, momentum and quality, into one predicted return or rank, to capture interactions linear models miss, and to explain which characteristics drive a model. Results are judged across many formation dates against classic factor models.
How machine learning is used in factor investing
Choosing which characteristics matter
Researchers have proposed hundreds of firm characteristics that relate to returns, and many overlap. Regularised methods such as the lasso select the ones that add independent information and allow nonlinear effects.
Real exampleDissecting characteristics nonparametrically (Freyberger, Neuhierl and Weber)Used the adaptive group lasso to test which characteristics provide independent information about expected returns, and found that many previously identified predictors do not.
Combining characteristics with nonlinear models
Trees and neural networks combine many characteristics into one expected return and capture interactions, such as momentum behaving differently in small and large firms, that linear factor models miss.
Real exampleEmpirical asset pricing via machine learning (Gu, Kelly and Xiu)Compared linear models, trees and neural networks for measuring stock risk premia, and traced the gains of trees and neural networks to nonlinear interactions between predictors.
Estimating factor models with deep learning
Neural networks can estimate a pricing model that uses many firm characteristics and macroeconomic series at once, with economic restrictions built into the training objective.
Real exampleDeep learning in asset pricing (Chen, Pelger and Zhu)Estimated an asset pricing model with deep neural networks, using the no-arbitrage condition as the training objective and many macroeconomic time series to describe the state of the economy.
Correcting for data mining
When hundreds of signals have been tested on the same history, some look significant by chance. Factor researchers raise the bar for evidence and report how many alternatives were tried.
Real exampleTesting many factors (Harvey, Liu and Zhu)Argued that, given the hundreds of factors already tested, a newly discovered factor should clear a t-ratio above 3.0 rather than the usual 2.0.
Replicating published signals
Before combining signals with a model, researchers rebuild them from source data and check the results against the original papers. Shared replication data makes that faster and comparable.
Real exampleOpen Source Asset Pricing (Chen and Zimmermann)Publishes test asset returns and signals replicated from the academic asset pricing literature, with code and Python and R packages for downloading the data.
Can you learn machine learning for factor investing?
Yes, if you are comfortable with regression or willing to learn it. Factor research is mostly careful panel data work, so learn Python and statistics first, then classic factor models and portfolio sorts, then machine learning methods that combine many characteristics.
- Python and panel dataBuild a monthly panel of stocks and characteristics, lag every characteristic so it is known at the formation date, and form portfolios by sorting.Learn it in Python
- Regression and multiple testingCross-sectional regressions, t-statistics and corrections for testing many signals at once.Learn it in Statistics
- Linear algebra for factor modelsCovariance matrices and principal components, which underlie statistical factor models and IPCA.Learn it in Maths
- Regularised and tree modelsLasso, elastic net, random forests and gradient boosting fitted on the cross-section and evaluated on later years.Learn it in Machine Learning
- Neural networks for expected returnsFeed-forward networks with early stopping and ensembling, compared with the linear model on the same out-of-sample period.Learn it in Deep Learning
Moving into machine learning for factor investing
If you already work in the field
Equity analysts and quant researchers already know the classic factors, portfolio sorts and how turnover raises costs. Add Python and machine learning, then replicate one published signal from the Open Source Asset Pricing data before combining signals with a model.
If you come from data science or ML
Data scientists should learn asset pricing basics: what a factor is, how portfolio sorts and Fama-MacBeth regressions work, and why look-ahead and survivorship bias distort results. Reading a replication study and the original Fama and French papers gives the vocabulary.
Roles that do this workQuantitative researcher, equitiesFactor research analystSystematic portfolio managerQuantitative analyst
Tools you will use
| Tool | What you use it for |
|---|---|
| pandas | Align characteristics and returns by formation date without look-ahead. |
| statsmodels | Run time-series factor regressions and test alphas. |
| linearmodels | Run Fama-MacBeth cross-sectional regressions with appropriate standard errors. |
| LightGBM | Fit gradient-boosted models that combine many characteristics into one return forecast. |
| Alphalens (reloaded) | Analyse a factor's information coefficient, quantile returns and turnover. |
| SHAP | Explain which characteristics drive a tree model's predictions. |
Public data to practise on
- Kenneth R. French Data Library: Free factor returns and characteristic-sorted portfolios, the standard benchmark for factor research.
- Open Source Asset Pricing: Replicated predictor signals and portfolio returns from the academic asset pricing literature, with Python and R download packages.
- AQR data sets: Factor and portfolio return series from AQR research papers, such as quality minus junk and betting against beta.
Your first project
Download the Kenneth French 25 size and book-to-market portfolios with the factor returns, explain monthly portfolio returns with the three-factor model, and compare it with the CAPM baseline using R² and average absolute alpha.
Common questions
What is factor investing?
Factor investing builds portfolios that tilt towards characteristics associated with returns or risk, such as value, momentum, size, quality and low volatility. A factor is usually defined by sorting stocks on a characteristic and comparing the top and bottom groups.
Does machine learning improve factor investing?
Academic studies report that flexible models combining many characteristics predict returns better than single linear factors in historical data. Whether that holds after costs, turnover and changing markets is uncertain, so treat the results as research, not advice.
Why do so many factors fail out of sample?
Researchers have tested hundreds of signals on the same historical data, so some look strong by chance. Higher significance thresholds, replication on new periods and markets, and accounting for costs filter many of them out.
What data do I need for factor research?
Monthly stock returns, firm characteristics built from accounting data, and factor return series to compare against. Kenneth French's data library and Open Source Asset Pricing are free starting points.
Courses for these skills
- Practical Machine Learning in PythonConnect inspection to model evaluation and leakage.51 published lessons · more lessons in production
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnReconcile reporting periods, identifiers and units.23 published lessons · all planned lessons open
- Statistics and Probability for Machine LearningAssess variation between samples instead of treating a ranking as fixed.41 published lessons · all planned lessons open
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.