In academic research, the pursuit of truth requires keen observation and a steadfast commitment to objectivity.
However, what if our search for knowledge is compromised by an unseen deception?
Imagine a scenario where only successful outcomes are highlighted, while failures are conveniently overlooked. This phenomenon, known as survivorship bias, can distort our understanding of reality and lead to misleading conclusions. By focusing solely on those who have succeeded, we may overlook valuable insights from those who have encountered setbacks.
In this article, we explore the implications of survivorship bias in research and provide practical tips to avoid its effects. By remaining vigilant and ensuring that all outcomes are considered, we can uphold the integrity of our research and draw more accurate conclusions.
Survivorship Bias
When researchers concentrate on people, organizations, or situations that have made it through some form of selection process while ignoring those who have not, it is known as survivorship bias. Survivorship bias might lead researchers to draw inaccurate conclusions since they only analyse a subset of the population. Survivorship bias is an example of selection bias.
The term “survivorship bias” refers to the error that happens when a dataset focuses primarily on “surviving” observations, failing to account for those who did not endure or survive.
It results from the exclusion of non-surviving elements in the dataset, leading to an incomplete and distorted view of reality.
Survivorship bias can be found in various fields such as business, finance, history, and especially in research.
In research, this bias arises when only the data from successful participants or interventions are analyzed, disregarding those who dropped out or failed to meet the criteria. To address survivorship bias, researchers must acknowledge its presence and actively incorporate diverse perspectives and data into their analyses.
Here’s an example of survivorship bias.
Imagine a hospital conducting research on trauma patients admitted to the emergency room (ER) to determine the most effective procedures.
However, researchers can only start their studies if a patient is stable enough to provide consent. This trial excludes individuals who didn’t survive their injuries or are too unwell to consent.
Consequently, survivorship bias may skew the results, as only the experiences of survivors are considered. This bias isn’t limited to research and can also affect decision-making in fields like finance, medicine, and business.
Impact of Survivorship Bias in Research
The risks of survivorship bias can result in a distorted perception of reality, which can lead to bad judgements and decisions.
We may overestimate the likelihood of success, underestimate risks, and make incorrect assumptions or strategies if we ignore the failures or unsuccessful situations.
Examining both survivors and non-survivors is essential to preventing this bias, as is examining the entire distribution of results rather than just the most extreme ones.
Examples of survivorship bias
Survivorship bias in research are prevalent across various fields, including health and medical studies.
survivorship bias in medical research
One prominent example of this can be found in medical research studies that only include patients who have successfully completed an intervention or treatment.
This bias becomes evident in disease diagnoses, particularly when examining post-diagnosis survival rates.
The bias arises because patients with more favorable prognoses, such as younger age, healthier medical histories, and higher fitness levels, tend to exhibit higher survival rates. However, patients with less favorable medical histories who do not survive are frequently excluded out of the calculation of survival rates, which causes an overrepresentation of people in good health in the data.
Accurately analysing the virus’s impact in the context of global events such as the COVID-19 pandemic is difficult.
Medical experts and epidemiologists advise against depending only on survival rate calculations since they do not provide a complete picture. For instance, deaths that occur without a COVID-19 test result are not eligible to be counted among the official number of deaths associated with the virus.
When analyzing data relating to the condition, this constraint can cause survivorship bias, especially in areas where testing capacity and infrastructure are overwhelmed. This could lead to incomplete data and possible errors in survival rate calculations.
survivorship bias in plane research World War II
While studying bomber planes during World War II, statistician Abraham Wald and his coworkers at Columbia University came across a striking illustration of survivorship bias. Their job was to determine where on the aircraft strengthening was needed by analysing the damage caused by planes that were returning.
They started by focusing on the places that had sustained the greatest damage, with the intention of reinforcing those areas to improve their survival.
Wald, however, correctly identified survivorship bias in their analysis. He saw that they were ignoring the planes that had been shot down and were only examining at the ones that had made it back safely.
Their strategy changed as a result of this realisation. Wald recognised that the parts of the returning planes with the most damage were the ones that could sustain it and yet survive.
In contrast, the planes that were shot down most certainly suffered damage in a variety of areas. As a result, survivorship bias skewed their analysis, causing it to overlook planes that did not survive.
Wald’s team suggested reinforcing the areas of the planes that were underrepresented in the data, especially the cockpit and motors, to address this bias. They realised that the planes that were hit in these crucial locations never came back, proving the need of reinforcement there. This insight enabled them to make more precise predictions, perhaps saving countless lives.
Why is Survivorship bias important?
Survivorship bias is significant for several reasons:
- When survivorship bias is present, only a portion of the data is considered, leading to overly optimistic conclusions. This can create a false perception that circumstances are more favorable or success is more common than it actually is. For instance, focusing solely on successful entrepreneurs may lead to underestimating the challenges and risks involved in starting a business.
- Survivorship bias can result in the misinterpretation of correlation, where a cause-and-effect relationship is perceived where none exists. For example, assuming that high school or college dropouts become successful entrepreneurs because of leaving school, when in reality, success might be due to other factors unrelated to dropping out.
- By ignoring those who didn’t “survive” or succeed, important information about failures or unsuccessful outcomes is missed. This incomplete decision-making can lead to misguided strategies or missed opportunities, as crucial lessons from failures are overlooked.
implications of survivorship bias In research
Survivorship bias can seriously affect research in a number of ways, which may compromise the validity and reliability of the results.
Below are several significant implications of survivorship bias on research:
1. Incomplete understanding
The presence of survivorship bias may lead to an incomplete understanding of the research topic. When researchers ignore or exclude failures or non-survivors, they lose out on important information that might provide important details about the underlying mechanisms or variables influencing results.
2. Publication bias
Publication bias can potentially affect study findings. Research with significant or positive results may be more likely to be published in journals and other publications, while research with non-significant or negative outcomes may be overlooked.
The skewed representation of the research environment that this publication bias might provide can result in an overemphasis on successful or positive outcomes.
3. Biased sample selection
Survivorship bias could affect the selection of participants or samples. Researchers may accidentally bias their sample by only including individuals or entities that have survived or succeeded, leaving out those that have not. This type of bias leads to distorted results and limits the generalizability of the research findings.
4. Impaired generalizability and replication:
Research findings may have difficulty being replicated as a result of survivorship bias. It becomes challenging to reproduce the results with the same degree of success or generalize the findings to other populations or circumstances when only successful or surviving cases are taken into account. This reduces the study’s external validity and robustness.
5. Skewed statistical analysis
By ignoring examples that fail or do not survive, survivorship bias can skew statistical analysis. This bias may cause statistical relationships to be misleading, skewed distributions, or an overestimation of effect size. As a result, interpretations drawn from the statistical significance and results could be false or deceptive.
6. Misleading conclusions
Survivorship bias emerges from a basic misunderstanding of correlation and causation, leading to skewed conclusions and misinterpretations of reality. When researchers exclusively consider the outcomes that survive or succeed, they might formulate incorrect generalizations or assert unfounded assertions regarding the entire population or phenomenon being investigated.
7. Misguided decision-making and policy
Survivorship bias in research can have a knock-on effect on misguided policy and decision-making processes. Policy decisions based on study findings that are skewed by survivorship bias may not be beneficial because they do not take into account the whole range of experiences and outcomes.
Strategies to Avoid Survivorship Bias
Having understood the risks posed by survivorship bias, it’s essential to equip ourselves with the awareness and tactics necessary to evade its influence.
Here are ten invaluable tips to help us liberate ourselves from its deceptive hold:
- Define sample or population: Ensure a clear definition of the population or sample from which data is gathered, encompassing all relevant cases, not solely the successful ones. Avoid excluding subgroups, outliers, or non-responders, as they might offer crucial insights.
- Collect data on non-responders or non-survivors: Gather information on non-survivors or non-responders, including their characteristics and reasons for dropout or non-response. This comprehensive approach offers a fuller understanding of the studied phenomenon and identifies potential biases.
- Use several data sources: Relying on a single data source increases the likelihood of survivorship bias. Using several data sources, such as alternative datasets, surveys, or interviews, allows you to gain a more thorough and diversified viewpoint while reducing any biases.
- Use random sampling techniques: Employ random sampling methods like random sampling, stratified sampling, or cluster sampling to provide each population or sample member an equal chance of inclusion. This helps mitigate selective sampling and survivorship bias risks.
- Report limitations transparently: Be open and truthful about the limitations on your research, including any possibility of survivor bias. Acknowledge any biases, limitations, or difficulties encountered during the research process, and be sure to address them in your conclusions. This can improve the integrity and credibility of the research you’ve done.
- Include multiple time points: Incorporate multiple time points when examining dynamic phenomena to capture changes comprehensively. Avoid focusing solely on the endpoint, as it may overlook the experiences of those who didn’t reach it, potentially introducing survivorship bias.
- Apply sensitivity analysis: To test how resilient the results are to various hypotheses or scenarios, use sensitivity analysis. This can assist in determining the possible influence of survivorship bias on the findings and offer a more precise and trustworthy estimation of the phenomenon being investigated.
- Challenge assumptions: Pay attention to the assumptions that are established during the study process, such as assuming that successful cases or survivors are typical of the total population. Challenge these assumptions and assess their viability closely to prevent survivorship bias.
- Seek different perspectives: Colleagues, specialists, and participants from all backgrounds are good places to start when looking for feedback and suggestions. This might assist in finding potential biases that could go unnoticed from one perspective, such as survivorship bias.
- Consider several outcomes: Avoid focusing solely on one outcome or measure. To fully encompass all potential outcomes, take into account a variety of outcomes, both positive and negative. By avoiding cherry-picking or highlighting just the positive results, this helps lessen the negative effects of survivorship bias.
Conclusion
Awareness of survivorship bias is crucial because it influences our perceptions, judgments, and the quality of our conclusions.
By understanding and addressing survivorship bias, we can make more informed decisions and draw more accurate conclusions based on a comprehensive understanding of the data.
