Survivorship bias can ruin your backtesting results by creating a false sense of success. It happens when you only analyze assets that survived (like current stocks in an index) while ignoring those that failed, went bankrupt, or were delisted. This oversight inflates performance metrics, such as returns and Sharpe ratios, making weak strategies look profitable.
Key Points:
- What it is: Survivorship bias focuses only on successful assets, excluding those that failed.
- Why it matters: Ignoring failed assets skews backtesting results, hiding risks and inflating metrics like returns and Sharpe ratios.
- Real-world effects: Studies show excluding delisted stocks can quadruple returns and drastically improve Sharpe ratios.
- How to avoid it:
By addressing survivorship bias, you’ll ensure your backtests reflect market realities and avoid overestimating your strategy’s potential.
What is Survivorship Bias?
Definition
Survivorship bias refers to the tendency to focus only on assets or entities that have endured over time while ignoring those that didn’t make it. In trading, this happens when you analyze stocks or cryptocurrencies that are still active today, overlooking the ones that failed or were delisted. This selective focus distorts the reality of the market.
When backtesting, excluding failed assets gives you an unrealistic advantage by relying on hindsight. Essentially, the dataset is already filtered for success before you even begin testing. As researchers from Vontobel explain:
"Survivorship bias removes the losers and makes the past look far rosier than reality".
The real-world challenge of trading is dealing with uncertainty – you never know which assets will fail. Survivorship bias skews backtesting results, making them unreliable for real-world application.
Why It Matters for Backtesting
Understanding survivorship bias is crucial because it can drastically alter performance metrics during backtesting.
When you ignore failed assets, the results aren’t just slightly inaccurate – they can be completely misleading. For example, Vontobel illustrated this in October 2025 with a five-stock momentum strategy. Including a delisted stock (Stock E) resulted in a Sharpe ratio of 0.09 and a return of 0.50%. However, when only surviving stocks were considered, the Sharpe ratio soared to 0.66, and returns quadrupled to 2.00%.
This difference is significant. A strategy that seems mediocre when accounting for failures suddenly looks like a winner when failures are excluded. But this doesn’t reflect reality. Without considering delisted assets, you’re essentially testing a strategy in an idealized scenario where every choice succeeds – a far cry from actual market conditions.
This issue is more than just inflated returns. Backtesting is often seen as a critical tool for traders. As QuantifiedStrategies points out:
"The significance of backtesting in trading cannot be overstated. It’s a tool that allows traders to… make informed decisions based on analysis rather than guesswork".
However, if the analysis is based on biased data, those so-called "informed decisions" are built on an illusion, not reality.
sbb-itb-e64548c
Survivorship Bias In Trading And Backtesting
How Survivorship Bias Distorts Backtesting Results
Impact of Survivorship Bias on Backtesting Performance Metrics
Inflated Performance Numbers
Testing a strategy with only currently active stocks can paint an overly rosy picture of its success. For instance, excluding just one delisted asset might inflate a strategy’s average quarterly return from 0.50% to 2.00% and push the Sharpe ratio from 0.09 to 0.66 – an 86% jump in performance metrics.
This stark difference reveals the impact of survivorship bias. A strategy that initially seems mediocre, with a Sharpe ratio of 0.09, can appear highly promising when the failures are ignored. The problem doesn’t stop there. The issue becomes more pronounced with multiple testing. Research shows that after testing just seven different strategy configurations, there’s a high likelihood of finding at least one backtest with a 2-year Sharpe ratio exceeding 1.0 – even if the actual expected out-of-sample Sharpe ratio is zero. As Marcos Lopez de Prado explains:
"After trying only 7 strategy configurations, a researcher is expected to identify at least one 2-year long backtest with an annualized Sharpe ratio of over 1, when the expected out of sample Sharpe ratio is 0".
Hidden Risks
Survivorship bias doesn’t just inflate performance metrics – it also hides the true risks of a strategy. Stocks that are delisted often experience significant price declines before their removal. Ignoring these failed assets erases periods of high volatility and steep losses, creating an equity curve that looks deceptively smooth.
This selective omission undermines the reliability of key metrics like the Sharpe ratio. In a study of 888 algorithmic trading strategies, these metrics were found to have minimal predictive value (R² < 0.025) for out-of-sample performance. In other words, the inflated figures tell us very little about how a strategy will hold up in actual market conditions.
Examples from Market History
The effects of survivorship bias aren’t limited to theoretical scenarios – they’ve played out in real market events. Crises like the dot-com bubble burst and the 2008 financial meltdown demonstrate how excluding failed companies can obscure substantial risks, making strategies seem more robust than they truly are.
Thomas Wiecki of Quantopian Inc. highlighted this phenomenon, noting:
"The more backtesting a quant has done for a strategy, the larger the discrepancy between backtest and out-of-sample performance".
Common Sources of Survivorship Bias
Testing with Current Index Members
A frequent pitfall in backtesting arises when using today’s index composition to test strategies on past markets. For example, downloading the current S&P 500 list and applying it to a backtest spanning 2000–2025 introduces hindsight bias. The companies on today’s list have weathered events like the dot-com crash, the 2008 financial crisis, and other market downturns. However, your backtest assumes these companies were always part of the index, ignoring the fact that many didn’t survive.
Indices are regularly updated. Companies like Lehman Brothers and Enron were once dominant but were removed after their collapse. Ignoring these failures can drastically skew results. Consider this: a momentum strategy backtested on the Nasdaq 100 from 1993 to 2020, using only current members, showed a staggering CAGR of 46% with a 41% drawdown. But when delisted companies from the dot-com bubble were included, the CAGR dropped to 16.4%, and the drawdown soared to 83%.
Research by Hendrik Bessembinder highlights another striking fact: between 1926 and 2015, only 42.1% of common stocks outperformed short-term Treasuries, and the median lifespan of a stock was just seven years. In one study, backtesting the 20 smallest companies in the S&P 500 with current members showed growth more than five times faster than when using a historical dataset that included delisted companies.
Free and Retail Data Sources
Another source of bias comes from using free or retail data sources. Platforms like Yahoo Finance only include active, tradable stocks, leaving out bankrupt, merged, or delisted companies. This omission skews backtest results by ignoring the failures that your strategy needs to account for.
The impact of this bias grows over time. For instance, a dataset covering the last 10 years in North America could exclude up to 75% of the stocks that were actually trading during that period. Brian Stanley, Founder of QuantRocket, explains:
"Brokers are focused on helping their customers trade; since you can’t trade delisted stocks, most brokers don’t include them in their data feeds".
This oversight can have real consequences. In 2010, Quantitative Investment Management (QIM) found that a strategy they expected to yield 20% actually returned only 8% after properly accounting for survivorship bias. Similarly, during the 2008 financial crisis, survivorship bias led to an annual overestimation of mutual fund performance by 2.1%.
Cryptocurrency Backtesting
The fast-paced and volatile cryptocurrency market faces similar challenges. Many backtesting tools rely on data from active exchanges, which often purge price histories for delisted coins. This makes it nearly impossible to recreate an accurate picture of the market’s past.
Crypto markets are highly volatile – about three times more so than traditional stocks. Projects frequently fail, and many tokens didn’t exist before 2019. Coins like BitConnect and OneCoin once attracted massive attention before collapsing in 2017–2018 due to regulatory issues. If these failed tokens are excluded from backtesting, the results paint an overly rosy picture of the market.
As CoinAPI puts it:
"Most crypto backtesting is like [testing a car in a wind tunnel with only gentle breezes]… when they hit real markets – with slippage, liquidity gaps, and order book dynamics – the strategy crumbles".
How to Avoid Survivorship Bias
To address the distortions caused by survivorship bias, consider these practical strategies:
Use Comprehensive Historical Data
Ensure your dataset includes all companies from the relevant test period – not just those that survived. This means incorporating firms that went bankrupt, merged, or were delisted. Databases like CRSP (Center for Research in Security Prices) and Compustat are designed to maintain these records, helping to avoid survivorship bias entirely.
Free data sources often focus only on active stocks, leaving out the failures that significantly impact real-world trading outcomes. To accurately reflect the market’s history, professional-grade data is essential.
Additionally, make sure your asset universe corresponds to what was historically available during the period under analysis.
Select Assets Based on Their Historical Availability
When backtesting, avoid using today’s index members to represent historical periods. Instead, opt for a time-varying universe that mirrors the companies actually available at each point in time. This prevents your strategy from benefitting unfairly from hindsight.
To further reduce bias, timestamp all input data with its original release date. This ensures you’re not inadvertently introducing survivorship or look-ahead bias into your analysis.
Leverage Backtesting Tools with Bias Detection Features
Advanced backtesting platforms can play a crucial role in identifying and correcting bias. Look for tools that calculate metrics like the Deflated Sharpe Ratio (DSR), which adjusts for performance inflation caused by testing multiple strategy variations. These platforms should also determine the Minimum Backtest Length (MinBTL) based on the number of configurations tested. Research indicates that testing just seven strategy variations can produce at least one 2-year backtest with an annualized Sharpe ratio over 1.0 – even if the true expected performance is zero.
MillionMachine is one such platform that addresses these challenges. It offers Monte Carlo simulations to test for overfitting and supports backtesting across various asset classes, including stocks, futures, currencies, ETFs, and cryptocurrencies. The platform also allows users to visually design and optimize strategies while analyzing performance metrics beyond the traditional Sharpe ratio. This helps you distinguish genuine opportunities from statistical noise.
As Marcos López de Prado aptly points out:
"The maddening thing about backtesting is that the better you become at it, the more likely false discoveries will pop up".
Best Practices for Accurate Backtesting
Avoiding survivorship bias is just one part of creating dependable trading strategies. To truly build strategies you can trust, you need to focus on data quality, consider different market conditions, and maintain transparency in your research. Let’s start with the foundation: your data.
Check Your Data Quality
The quality of your historical data can make or break your backtesting results. Databases such as CRSP (Center for Research in Security Prices) and Compustat are excellent options because they provide a complete history of securities within your test period. These sources also timestamp data with its initial public release date, helping you avoid look-ahead bias. Plus, they include records of companies that went bankrupt, merged, or were delisted – critical details for accurate testing.
When working with accounting data, it’s important to apply proper time lags. This mimics the real-world delay between the close of a fiscal period and when the data becomes publicly available.
The impact of ignoring these details can be dramatic. In October 2025, researchers Daniel Höchle and Sandro Felicioni illustrated this with a momentum strategy involving five stocks. By excluding the delisting of just one stock, they inflated average quarterly returns from 0.50% to 2.00%. Narrowing the backtesting window to favorable periods further exaggerated the Sharpe ratio from 0.1 to an eye-popping 10.6 – turning a weak strategy into what appeared to be a stellar one.
Accurate data is the backbone of testing strategies across various market scenarios.
Test in Different Market Conditions
A strategy that thrives in one market environment might fail in another. That’s why it’s essential to test your strategy in a variety of conditions, including bull markets, bear markets, high volatility periods, and low volatility environments. This helps ensure your approach isn’t overly dependent on specific market trends.
Stress tests are another valuable tool. By simulating diverse scenarios, you can see how your strategy performs under different conditions – not just the historical path the market took. If your strategy only succeeds during specific economic cycles, it’s likely a statistical fluke, not a reliable trading edge.
As Marcos López de Prado wisely points out:
"The purpose of a backtest is to discard bad models, not to improve them".
In other words, use backtesting to identify and eliminate weak strategies early on. Avoid the temptation to tweak parameters just to make the numbers look better – that’s a fast track to overfitting.
Document Your Assumptions
Transparency is key to distinguishing solid research from accidental data mining. Make it a habit to document every backtest you perform. This isn’t just good practice – it’s essential for calculating the Deflated Sharpe Ratio and determining whether your results reflect genuine alpha or selection bias.
David H. Bailey doesn’t mince words on this:
"Not reporting the number of trials involved in identifying a successful backtest is a similar kind of fraud".
Before running any tests, start with a clear economic hypothesis. Why should this strategy work? What market inefficiency are you targeting? Defining these questions upfront prevents you from retrofitting explanations to random patterns.
Document everything: the criteria for selecting your asset universe, how you handled delisted securities, the parameters you tested, and your data sources. Don’t forget to account for realistic transaction costs and bid-ask spreads. For instance, a strategy with a 0.5% roundtrip cost might see its Sharpe ratio fall to zero compared to a frictionless simulation.
MillionMachine.com is designed for research, education, and strategy development. It does not offer personalized investment advice, trading advice, or financial guidance. MillionMachine does not recommend or evaluate the suitability of any strategy, trade, or investment.
Users are solely responsible for assessing their trading decisions and risks. Simulations, backtests, and performance metrics generated by MillionMachine are hypothetical and not guarantees of future results. Actual trading outcomes may vary significantly from simulated results.
MillionMachine is not registered as a Commodity Trading Advisor (CTA), Investment Advisor, or Broker-Dealer with the NFA, CFTC, SEC, or any other regulatory body. While the founder was formerly registered as a CTA with the National Futures Association (NFA), that registration is no longer active, and MillionMachine does not engage in regulated advisory activities.
MillionMachine does not execute trades, manage customer funds, or provide access to live trading accounts. Broker API integrations are strictly for user-initiated automation. Users are responsible for ensuring compliance with all relevant laws, regulations, and broker requirements.
Market data, charts, signals, and analytics provided by MillionMachine are for informational purposes only. MillionMachine does not verify the accuracy or completeness of market data and assumes no responsibility for errors, delays, or omissions.
Trading financial instruments – including futures, stocks, cryptocurrencies, and derivatives – carries significant risk and may not be suitable for everyone. Losses can exceed your initial investment. Past performance, whether actual or simulated, is not indicative of future results.
Conclusion
Survivorship bias can seriously distort backtesting results, creating a misleading picture of success. When strategies are tested using only assets that survived – while excluding bankrupt companies, delisted stocks, or failed cryptocurrencies – it paints an overly optimistic view that ignores the inevitable failures present in real-world trading.
This issue isn’t just theoretical; it’s backed by hard data. For example, testing multiple strategy configurations can yield seemingly stellar results even when actual performance is zero. Similarly, excluding delisted stocks can inflate key metrics, turning a losing strategy into one that falsely appears profitable. These discrepancies aren’t minor – they can fundamentally alter the perception of a strategy’s viability.
Addressing survivorship bias requires diligence and the right approach. Using databases that include delisted securities, applying statistical corrections for multiple testing, and evaluating strategies under various market conditions are critical steps. Additionally, documenting every test thoroughly is essential for transparency. As David H. Bailey noted, failing to report testing details misrepresents performance evaluations.
MillionMachine supports traders in tackling these challenges with tools designed for rigorous backtesting. Features like Monte Carlo simulations for overfitting detection and advanced performance analysis across asset classes help ensure a disciplined approach. This methodology is key to turning theoretical strategies into practical, real-world success.
Backtesting isn’t just about generating impressive charts – it’s about identifying and eliminating flawed strategies. Reliable results stem from quality data, statistical precision, and an honest evaluation of outcomes. These principles are your best defense against survivorship bias and other pitfalls in the backtesting process.
MillionMachine.com is a platform for research, education, and developing trading strategies. It does not provide personalized investment advice, financial guidance, or solicitations to trade any financial instrument. MillionMachine makes no recommendations and does not assess the suitability of any strategy, trade, or investment.
Users are solely responsible for their trading decisions and the risks involved. All simulations, backtests, and performance metrics generated by MillionMachine are hypothetical and should not be interpreted as guarantees of future results. Hypothetical performance has inherent limitations and often diverges significantly from actual trading outcomes.
MillionMachine is not registered as a Commodity Trading Advisor (CTA), Investment Advisor, or Broker-Dealer with regulatory bodies such as the NFA, CFTC, or SEC. While the founder was previously registered as a CTA with the National Futures Association (NFA), that registration is no longer active, and MillionMachine does not engage in regulated advisory activities.
The platform does not execute trades, manage customer funds, or provide access to live trading accounts. Any broker API integrations are solely for user-initiated and controlled automation. Users are fully responsible for ensuring compliance with applicable laws, regulations, and broker requirements.
All market data, charts, signals, and analytics provided by MillionMachine are for educational and informational purposes only. MillionMachine does not guarantee the accuracy or completeness of market data and assumes no responsibility for errors, delays, or omissions.
Trading financial instruments – including stocks, futures, cryptocurrencies, and derivatives – carries substantial risk and may not be suitable for all investors. Losses can exceed your initial investment. Past performance, whether real or simulated, is not indicative of future results.
FAQs
What is survivorship bias, and how does it impact backtesting accuracy?
Survivorship bias happens when backtesting overlooks assets or strategies that have failed, been delisted, or otherwise removed from the dataset. This skews the results, painting an overly optimistic picture by focusing solely on the ‘survivors,’ which can artificially boost returns and risk-adjusted metrics.
To conduct accurate backtesting, it’s crucial to work with a comprehensive dataset that includes both successful and failed assets. This approach offers a more realistic assessment of how a strategy performs across full market conditions.
How can I avoid survivorship bias when backtesting trading strategies?
Survivorship bias happens when backtesting only includes data from assets that are still active, overlooking those that were delisted, merged, or went bankrupt. This can paint an overly rosy picture of performance. To avoid this pitfall, make sure your dataset covers all assets – both active and inactive – during the testing period.
Choose datasets that include delisted and failed securities, along with their final prices, delisting dates, and any settlements. Ensure your analysis accounts for total-return calculations, factoring in dividends and stock splits for all securities, not just the ones that survived. It’s also a good idea to validate your data by cross-referencing it with exchange records to confirm accuracy.
Platforms like MillionMachine can simplify this process by offering access to data free from survivorship bias, along with tools to thoroughly analyze performance. Combining high-quality data with out-of-sample testing will give you more dependable and realistic insights into your trading strategies.
Why should delisted or failed assets be included in backtesting results?
When conducting backtests, it’s crucial to include delisted or failed assets to prevent survivorship bias. This bias skews results by focusing solely on assets that are still active, painting an unrealistically positive picture of a strategy’s performance.
By factoring in assets that underperformed or failed, your backtest delivers a more realistic assessment of the risks and challenges involved in trading. This approach ensures your strategy evaluation is grounded in the complexities of actual market conditions.
