Defense Counsel Journal

Statistically Significant Association: Preventing the Misuse of the Bradford Hill Criteria to Prove Causation in Toxic Tort Cases

February 12, 2020

William R. Andrichik

William R. Andrichik

William R. Andrichik is an associate at Steptoe & Johnson LLP who counsels clients through a wide variety of complex commercial challenges, helping navigate potential disputes and litigating active matters. He litigates in Federal and State courts across the country, handling breach of contract, toxic tort and product liability claims, mass torts, and class actions.

Jeremy S. Goldkind

Jeremy S. Goldkind

Jeremy Goldkind is also a partner in Steptoe’s Chicago office. His practice concentrates on complex commercial disputes and product liability matters, including toxic tort.

Derek C. Smith

Derek C. Smith

Derek C. Smith is a partner at Steptoe & Johnson LLP who represents chemical, pharmaceutical and manufacturing clients in challenging commercial, product liability, and toxic tort matters. He regularly counsels these clients in environmental, health, and safety matters and also defends them against personal injury and property damage.

ONE of the highest hurdles for a plaintiff to clear in a toxic tort or product liability case is proof of causation. This is because other key elements of the plaintiff’s case, such as product identification, product use and even exposure, typically only require lay testimony. Proof of causation, on the other hand, requires expert testimony – and requires proof “to a reasonable degree of scientific certainty.”

The legal system’s insistence on the “reasonable degree of scientific certainty” standard is nothing more than a legal fiction created to give courts, counsel and jurors alike the false sense of security that opinions are devoid of scientific guesswork. In reality, however, science is often anything but certain. If it were, experts would offer scientific fact, not opinion. 

Lack of certainty is never more evident than in the field of epidemiology, where researchers study the health of human populations. Through a variety of statistical tools, most notably mortality and incidence rates, epidemiologists seek to better understand potential risk factors associated with diseases in such populations. These studies, however, are not designed to achieve certainty in the observed associations, but rather to mitigate factors which could impact those associations, such as chance, bias and other confounding factors (e.g., smoking). Simply put, the less likely that chance, bias and other confounding factors may explain an observed association, the more confident epidemiologists become in their conclusions about the association. As a result, epidemiology is more about confidence in a finding than it is about certainty in a finding. 

Having a well-thought-out plan to address an expert’s causation opinion—especially in cases where other elements are not (or cannot be) challenged—is vital to avoiding an unfavorable verdict against your client. This article addresses the use of one common methodology used by epidemiologists in toxic tort cases to prove causation: the Bradford Hill criteria.

I. What are the Bradford Hill criteria?

To reach a conclusion of causation, epidemiologists often consider a set of “criteria” created by Sir Bradford Hill.  Sir Bradford Hill was a British physician who studied the association between smoking and lung cancer.¹1 A. Bradford Hill, The Environment and Disease: Association or Causation?, 58 Proc. Royal Soc. Med. 295, 295-300 (1965). To help study associations in epidemiological studies, Sir Bradford Hill developed nine criteria (or “viewpoints” as he described them):²2 See, e.g., DeGidio v. Centocor Ortho Biotech, Inc., 3 F.Supp.3d 674, 678 (N.D. Ohio 2014).

• Strength of association: whether the association is strong and statistically significant.
• Consistency of the association: whether multiple studies show the same (or substantially similar) results.
• Specificity of the Association: whether the observed effect has only one known cause.
• Temporality of the Association: whether the purported cause precedes the observed effect.
• Biological gradient (aka “Dose-response”): whether a stronger or greater exposure leads to greater amounts of harm.
• Plausibility: there must be both a rational and theoretical basis for the result.
• Coherence: whether the cause-effect relationship conflicts with what is already known and whether there are other competing hypotheses.
• Experiment: experimental evidence strengthens a causal inference and makes it more plausible.
• Analogy: whether a commonly accepted phenomenon in one area can be applied to another area.

These criteria represent a generally-accepted methodology that assists “epidemiologists [in] mak[ing] judgments about whether causation may be inferred from an association.”³3 Id.

There are nine criteria in all, but Sir Bradford Hill cautioned that none of them can “bring indisputable evidence for or against the cause-and-effect hypothesis.”⁴4 Hill, The Environment and Disease: Association or Causation? 58 Proc. Royal Soc’y Med. at 299. To the contrary, Hill did not believe anyone could “usefully lay down some hard-and-fast rules of evidence that must be obeyed” before accepting cause and effect relationships.⁵5 Id. As such, none of his “viewpoints” are required as a sine qua non of causation.⁶6 Id. Instead, his “viewpoints” can only, “with greater or less strength,” help answer the fundamental question – “is there any other way of explaining the set of facts before us, is there any other answer equally, or more, likely than cause and effect?”⁷7 Id.

The fact that the Bradford Hill criteria are merely a set of guidelines, and not a formula for causation, means that two different experts applying the same criteria may reach two different opinions on causation, namely, one for a cause-and-effect relationship and the other against such a relationship. This often makes it difficult to exclude expert testimony on the grounds that an expert did not faithfully apply the criteria, since application of the criteria to epidemiology studies, as well as the weight given to each criterion after application, is largely subjective. As a result, most disputes over the application of the criteria go to the weight of the expert’s testimony, not its admissibility.

This is particularly true in jurisdictions employing the Frye standard, because it is the scientific community, not the court, which is the gatekeeper of the evidence.⁸8 See Melnick v. Consolidated Edison, Inc., 959 N.Y.S.2d 609, 620 (N.Y. Sup. Ct.  2013) (“General acceptance does not come from the number of jurors that can be convinced of a novel theory at trial, especially where there is scant evidence to support that new theory or principle.”); Goeb v. Tharaldson, 615 N.W.2d 800 (Minn. 2000) (“[T]he Frye general acceptance standard ensures that the persons most qualified to assess scientific validity of a technique have the determinative voice.”) (citation omitted). Consequently, as long as an expert employs a method that the scientific community considers generally acceptable, then the court must admit the opinion.

The Daubert standard is more flexible, with the court, not the scientific community, serving as gatekeeper.  Although general acceptance remains a factor pursuant to Daubert, it is not the only factor, which means courts can admit opinion testimony “even if the particular methods [the experts] have used in arriving at their opinion are not yet accepted as canonical in their branch of the scientific community.”⁹9 Rosen v. Ciba-Geigy Corp., 78 F.3d 316, 318 (7th Cir. 1996).

There is no dispute that the Bradford Hill criteria are canonical in the field of epidemiology, but they should not be used as a means of turning the courtroom into a petri dish for culturing unproven scientific theories and personal beliefs.¹⁰10 Melnick., 959 N.Y.S.2d at 618 (“Courts are not medical or scientific laboratories in which to experiment with novel theories of causative factors or disease or medical conditions.”). As Judge Posner so aptly explained, the “[l]aw lags science; it does not lead it.”¹¹11 Rosen, 78 F.3d at 319; see also Tamraz v. Lincoln Elec. Co., 620 F.3d 665, 677 (6th Cir. 2010). This is easier said than done, particularly when experts wrap their unproven theories and hypotheses in the cloak of the Bradford Hill criteria. In such instances, we must feel confident that an objective measure exists for ensuring that courts are not paving new trails that science has not yet walked. With respect to the Bradford Hill criteria, this objective measure already exists.

II. Prerequisites to applying Bradford Hill criteria

Though the Bradford Hill criteria are commonly employed by epidemiologists to show causation, their acceptance is not a given. Before an epidemiologist can opine regarding causation, he or she must clear other, preliminary hurdles. As Sir Bradford Hill himself explained, his criteria are only applied after an association that is “perfectly clear-cut and beyond what we would care to attribute to the play of chance” exists.¹²12 See Hill, supra note 2, at 295 (emphasis in original). Consequently, if an expert cannot establish that a statistically significant association exists in the epidemiological literature between the disease and the exposure at issue, then the Bradford Hill criteria do not apply. 

First, the expert must show an association between the purported cause and the perceived effect. Epidemiologists have a number of tools at their disposal to identify a potential association between a disease, on the one hand, and a risk factor such as exposure to a chemical or other agent, on the other hand.¹³13 Federal Judicial Center, Reference Manual on Scientific Evidence, 566 (Third Ed., 2011). Some of the most commonly used analyses include relative risk, odds ratio, and attributable risk.

For example, when applying an odds ratio to show an association, a ratio of 1.0 means the risk or odds of disease in exposed individuals is the same as the risk or odds in unexposed individuals.¹⁴14 Id. at 567. If the ratio is greater than 1.0, the risk or odds of disease in exposed individuals is greater than the risk or odds in unexposed individuals, which suggests there is a positive association between exposure to the agent and the disease.¹⁵15 Id. Ratios less than 1.0 suggest that a negative association exists between the agent and the disease.¹⁶16 Id.

In some instances, however, “[a] study may find a positive association (relative risk greater than 1.0) when there is no true association.”¹⁷17 Id. at 572.   Conversely, a study may suggest there is no association when one in fact does exist.¹⁸18 Id. To increase confidence that observed associations are real, epidemiologists will seek to show that exposure to the agent and disease “occur together more frequently than one would expect by chance.”¹⁹19 Id. at 566.

There are two techniques for assessing chance: p-values and confidence intervals.²⁰20 Id. at 573, 577, 580-581. A “p-value” represents the probability of observing an association at least as large as the association observed in the study, despite there being no actual association.²¹21 Id. at 577. Put more (and possibly over-) simply, it is the probability of finding a false positive. A common p-value used in epidemiological studies is 0.05, which “means that the probability is 5% of observing an association at least as large as that found in the study when in truth there is no association.”²²22 Id.

A confidence interval, on the other hand, provides the range within which the risk would likely fall if the study were repeated numerous times.²³23 Id. at 573.  “If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the same population.”²⁴24 Id. at 580. If the range includes 1.0 (e.g. .8 – 1.3), the result is not statistically significant at the 0.05 level.²⁵25 Id. at 581.

Understanding p-values and confidence intervals is important because the statistical significance of a study has nothing to do with the size of the risk.²⁶26 Id. at 573 (“[S]tatistical significance is not about the size of the risk found in a study.”). In other words, a study’s findings are not statistically significant merely because the risk is large (e.g., 2.0 or 2.5 or 3.0).²⁷27 Id. Rather, we must have some level of confidence that the observed risk is real.  Confidence intervals and p-values provide us with this confidence. As a result, a study’s findings may be statistically significant when only a slightly increased risk of 1.05 is observed, but not when a higher risk of 2.0 is observed. 

However, the existence of a statistically significant association to trigger the Bradford Hill criteria does not necessarily mean that statistically significant associations are proof of causation. For example, while we may be confident in a statistically significant risk of 1.05, but not 2.0, we cannot say that the extremely small, but statistically significant, increase from 1.0 to 1.05 is proof of causation. As Sir Bradford Hill explained, tests of significance “can, and should, remind us of the effects that the play of chance can create, and they will instruct us in the likely magnitude of those effects…, [but] [b]eyond that they contribute nothing to the ‘proof’ of our hypothesis.”²⁸28 Hill, supra n. 2, at 299 (emphasis added). Thus, epidemiologists can use the Bradford Hill criteria to bridge the gap from association to causation.

III. Courts agree – statistically significant associations are a condition precedent to use of the Bradford Hill criteria

Modern jurisprudence confirms Sir Bradford Hill’s reasoning that the Bradford Hill criteria cannot be applied until a statistically significant association exists in the epidemiological literature.²⁹29 Soldo v. Sandoz Pharmaceuticals Corporation, 244 F. Supp.2d 434, 569 (W.D. Pa. 2003) (“Review of the criteria themselves, as set forth in the seminal remarks of Dr. Bradford-Hill, shows that an epidemiologic foundation is a prerequisite for application of his criteria…. [and] because plaintiff’s experts have not demonstrated any statistically-significant epidemiologic study showing an increased risk of postpartum stroke in women using Parlodel^®, application of the Bradford-Hill criteria is unwarranted.”); Wagoner v. Exxon Mobil Corp., 813 F. Supp.2d 771, 803 (E.D. La. 2011) (“It is true that the set of criteria known as the Bradford Hill criteria has been widely acknowledged as providing an appropriate framework for assessing whether a causal relationship underlies a statistically significant association between an agent and a disease.”) (citations omitted); Frischhertz v. Smithkline Beecham Corporation, 2012 WL 6697124, at *3 (E.D. La. December 21, 2012) (“The Bradford-Hill criteria can only be applied after a statistically significant association has been identified.”); McMunn v. Babcock & Wilcox Power Generation Group, Inc., 2013 WL 3487560, at *15 (W.D. Pa. July 12, 2013); In re: Lipitor (Atorvastatin Calcium) Marketing, Sales Practices and Products Liability Litigation, 174 F. Supp.3d 911, 924-926 (D. S.C. 2016), aff’d, 892 F.3d 624, 642 (4th Cir. 2018) (“[I]t is well established that the Bradford Hill method used by epidemiologists does require that an association be established through studies with statistically significant results.”) (emphasis in original) (citations omitted); see also Jones v. Novartis Pharmaceuticals Corporation, 235 F. Supp.3d 1244, 1272 (N.D. Ala. 2017) (holding that an association must first be established before an expert may rely on the Bradford Hill methodology to form a general causation opinion and then excluding an expert’s general causation opinion because the expert admitted that no study finding a statistically significant association between the drug and disease at issue existed in the peer-reviewed literature.); Federal Judicial Center, Reference Manual on Scientific Evidence, 598-599 (Third Ed., 2011) (“We emphasize that these [Bradford Hill] guidelines are employed only after a study finds an association to determine whether that association reflects a true causal relationship.”) (emphasis in original) (citations omitted). As the court in McMunn v. Babcock & Wilcox Power Generation Group explained, the use of the Bradford Hill criteria is a two-step process.³⁰30 McMunn, 2013 WL 3487560, at *15. “Step one looks to whether there is a statistically significant association between a substance and a specific disease.”³¹31 Id. (citation omitted). If no such association exists, “the analysis should end there.”³²32 Id. (citation omitted). If one does exist, “the second step applies the Bradford Hill criteria to assess whether the relationship is causal.”³³33 Id. (citation omitted).

At this point, it is important to clarify that we are only discussing the methodology associated with the Bradford Hill criteria. Statistically significant studies are not a prerequisite to proving causation in every case.³⁴34 Dunn v. Sandoz Pharmaceuticals Corporation, 275 F. Supp.2d 672, 680 (M.D. N.C. 2003) (“Requiring [plaintiff’s expert] to have a statistically significant epidemiological study as the beginning point for application of the Bradford Hill criteria does not require [him] to have a statistically significant study in order to prove causation.”); In re Lipitor, 174 F. Supp.3d at 924. To the contrary, experts can render causation opinions based on other evidence – they just cannot do so through the use of the Bradford Hill criteria without first showing that a statistically significant association exists in the epidemiological literature.³⁵35 Dunn, 275 F. Supp.2d at 680; In re Lipitor, 174 F. Supp.3d at 924-925.

The bottom line is that if an expert’s methodology for establishing causation is the Bradford Hill criteria, then a prerequisite to the use of that criteria should be the existence of a statistically significant association in the epidemiological literature. If an expert cannot show such an association, then another methodology for establishing causation must be used.

IV. How to effectively use (and defend against) the Bradford Hill criteria

So what does this mean for your case? Attorneys representing plaintiffs would be well-advised to consider the application of the Bradford Hill criteria to their client’s claim at the outset of the case; in other words, before they have made a significant investment and before fact discovery is underway. The first step is to determine whether your client’s claim requires epidemiological proof and, if so, whether the Bradford Hill criteria are implicated. Next, attorneys should develop a good understanding of their specific jurisdiction’s treatment of the Bradford Hill criteria—paying special attention to the requirement in most jurisdictions that an expert must first show an association before applying the criteria. Then, rather than waiting for court-imposed deadlines or the completion of fact discovery, attorneys should retain an expert early in the case to help consult with discovery necessary to avoid challenges to issues such as association, confidence in the association, and the application of the Bradford Hill criteria.

Defendants in toxic tort cases should follow a similar blueprint— starting with thinking about the causation piece of the puzzle early, and well before fact discovery is completed. Defense counsel would also benefit greatly from retaining an expert early in the case to consult and help craft discovery requests, requests for admission, and themes for the plaintiff’s or plaintiff’s experts’ depositions. Knowing the law of the jurisdiction is equally important for defense counsel, especially the common requirement that an association is a pre-requisite for application of the Bradford Hill criteria. This requirement often allows for successful dispositive motion practice, despite the typically fact-based nature of medical and legal causation issues. Having a well-developed plan and a strong command of your jurisdiction’s law on this issue can mean the difference between winning or losing on summary judgment, or a defense verdict versus a potentially large plaintiff’s verdict in toxic tort, environmental, and product liability matters.

View Article

Back