Finance and economics
Machine learning for stock selection
In stock selection, machine learning ranks a universe of stocks by expected relative return using fundamentals, prices and alternative data, combines many weak signals into one score, and shows which features drive the ranking. Evaluation compares ranks across many dates rather than forecasting one price.
How machine learning is used in stock selection
Ranking stocks against each other
Instead of forecasting one price, models estimate each stock's chance of beating the market or its peers over the next period, then rank the whole universe by that score.
Real exampleDeep neural networks, gradient-boosted trees, random forests (Krauss, Do and Huck)Trained deep neural networks, gradient-boosted trees, random forests and their ensembles on lagged returns of S&P 500 stocks to estimate each stock's probability of outperforming the market the next day, then ranked stocks by it.
Combining many weak signals
Valuation, momentum, profitability, liquidity and volatility each carry a little information. A model trained across many dates combines them into one score and shows which signals carry the most weight.
Real exampleEmpirical asset pricing via machine learning (Gu, Kelly and Xiu)Found that linear, tree and neural network models agreed on the same dominant predictive signals, which included variations on momentum, liquidity and volatility.
Explaining each score
Portfolio managers need to know why a stock ranks highly before acting on it. Feature attribution methods break a model's score into contributions from each input.
Real exampleSHAP (Lundberg and Lee)Introduced SHAP, a unified framework that assigns each feature an importance value for a particular prediction.
Using text from company filings
Annual and quarterly reports are long and change slowly, so changes in their wording can be informative. Text similarity and language model features turn filings into inputs for a ranking model.
Real exampleLazy Prices (Cohen, Malloy and Nguyen)Measured year-on-year changes in companies' 10-K filings and found that the changes predict future earnings, profitability, news announcements and firm-level bankruptcies.
Can you learn machine learning for stock selection?
Yes. Machine learning for stock selection is a ranking problem on a panel of stocks over time, and the skills are Python, statistics, supervised learning and careful point-in-time data handling. Fundamental analysts and data scientists can both learn it in that order.
- Python and panel dataBuild a table of stocks by date with returns and lagged characteristics, using only information public on each date.Learn it in Python
- Statistics for rankingsRank correlation, information coefficients and their variation across dates, which show whether a ranking is consistent.Learn it in Statistics
- Supervised learning across datesGradient boosting and regularised regression trained on past dates and tested on later ones, with ranking objectives as an option.Learn it in Machine Learning
- Model explanationSHAP values and partial dependence plots to show which characteristics drive each score.
- Text featuresEmbeddings or language model scores from filings and transcripts, added as features and tested like any other signal.Learn it in Generative AI
Moving into machine learning for stock selection
If you already work in the field
Equity analysts already know the companies, the accounting and why a cheap stock can stay cheap. Add Python and a supervised model, then test on later data whether a screen you use today ranks stocks better on its own or combined with other characteristics.
If you come from data science or ML
Data scientists should learn financial statements, how characteristics such as value and profitability are built, point-in-time data and why survivorship bias matters. Asset pricing replication datasets let you check your pipeline against published results.
Roles that do this workQuantitative equity researcherEquity research analystQuantitative portfolio managerInvestment data scientist
Tools you will use
| Tool | What you use it for |
|---|---|
| pandas | Build cross-sectional feature tables keyed by stock and date. |
| LightGBM | Train ranking or regression models on many stock characteristics. |
| scikit-learn | Rank-normalise features and fit linear baselines per date. |
| Alphalens (reloaded) | Measure a ranking's information coefficient and quantile spreads across dates. |
| SHAP | Explain which features push a stock up or down the ranking. |
Public data to practise on
- Numerai dataset: Free, obfuscated point-in-time stock features and targets grouped by era, built for training and validating stock-ranking models.
- Open Source Asset Pricing: Replicated predictor signals and portfolio returns from published research, for testing which characteristics rank stocks.
Your first project
Download the Numerai dataset, rank stocks on its target with LightGBM, and compare it with a single-feature baseline using mean per-era Spearman correlation on the validation eras.
Common questions
What is the difference between stock selection and stock prediction?
Stock prediction usually forecasts one stock's price or return, while stock selection ranks many stocks against each other on the same date. Ranking removes the common market move, which is the hardest part to predict.
Which features are used in machine learning stock selection?
Common inputs are past returns over several horizons, valuation ratios, profitability, investment, liquidity and volatility, plus text and analyst data where available. Each is lagged so it is known on the ranking date.
How is a stock ranking model evaluated?
By the rank correlation between scores and later returns on each date, averaged over many dates, and by the spread between top- and bottom-ranked groups, with turnover and costs reported alongside.
Can I use this page to choose stocks to buy?
No. This page teaches research methods with historical data. It does not recommend securities, and patterns found in research studies do not guarantee future results.
Courses for these skills
- Practical Machine Learning in PythonFit models to a defined financial target.51 published lessons · more lessons in production
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnReconcile reporting periods, identifiers and units.23 published lessons · all planned lessons open
- Statistics and Probability for Machine LearningAssess correlations, uncertainty and changes in the sample.41 published lessons · all planned lessons open
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.