Aggregation, Multilevel Data Title: Ecological inference and the ecological fallacy Author: David A. Freedman Date: March 1999 Pub: PDF Url: http://www.stat.berkeley.edu/~census/549.pdf Abstract: This paper reviews several methods for making ecological inferences, that is, inferring the behavior of individuals from aggregate data. Also considered is the ecological fallacy, which is the idea that relationships observed for groups necessarily hold for individuals. Title: On "Solutions" to the Ecological Inference Problem Authors: D. A. Freedman, S. P. Klein, M. Ostland and M. Roberts Date: April 1998 Pub: PDF Url: http://www.stat.berkeley.edu/~census/515.pdf Abstract: In his 1997 book, King announced "A Solution to the Ecological Inference Problem". This paper tests King's method on data sets where truth is known. His announcement is premature. In the test data, his method produces results that are far from truth, and diagnostics are unreliable. Ecological regression makes estimates that are similar to King's, while the neighborhood model is much more accurate. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Causal Inference Title: From Association to Causation: Some Remarks on the History of Statistics Author: David Freedman Date: June 1998 Pub: PDF Url: http://www.stat.berkeley.edu/~census/521.pdf Abstract: The "numerical method" in medicine goes back to Pierre Louis' study of pneumonia (1835), and John Snow's book on the epidemiology of cholera (1855). Snow took advantage of natural experiments and used convergent lines of evidence to demonstrate that cholera is a waterborne infectious disease. More recently, investigators in the social and life sciences have used statistical models and significance tests to deduce cause-and-effect relationships from patterns of association; an early example is Yule's study on the causes of poverty (1899). In my view, this modeling enterprise has not been successful. Investigators tend to neglect the difficulties in establishing causal relations, and the mathematical complexities obscure rather than clarify the assumptions on which the analysis is based. Formal statistical inference is, by its nature, conditional. If maintained hypotheses A, B, C, ... hold, then H can be tested against the data. However, if A, B, C, ... remain in doubt, so must inferences about H. Careful scrutiny of maintained hypotheses should therefore be a critical part of empirical work--a principle honored more often in the breach than the observance. Snow' work on cholera will be contrasted with modern studies that depend on statistical models and tests of significance. The examples may help to clarify the limits of current statistical techniques for making causal inferences from patterns of association. Cancer clusters present analytic problems of their own. There are many routes of exposure, and many forms of pathology. Distinguishing between real associations and patterns created by chance may be especially difficult in that context. Epidemiology has made enormous contributions to the control of disease, but remains an inexact science where judgment matters more than statistical technique. There is ample room for differences of opinion. Title: From association to causation via regression Author: David A. Freedman Date: April 22, 1994 Abstract: For nearly a century, investigators in the social sciences have used regression models to deduce cause-and-effect relationships from patterns of association. Path models and automated search procedures are more recent developments. In my view, this enterprise has not been successful. The models tend to neglect the difficulties in establishing causal relations, and the mathematical complexities tend to obscure rather than clarify the assumptions on which the analysis is based. Formal statistical inference is, by its nature, conditional. If maintained hypotheses A, B, C, ... hold, then H can be tested against the data. However, if A, B, C, ... remain in doubt, so must inferences about H. Careful scrutiny of maintained hypotheses should therefore be a critical part of empirical work-- a principle honored more often in the breach than the observance. I will discuss modeling techniques that seem to convert association into causation. The object is to clarify the differences among the various uses of regression, and the difficulties in making causal inferences by modeling. Title: On Specifying Graphical Models for Causation Author: David A. Freedman Date: June 2001 Pub: PDF Url: http://www.stat.berkeley.edu/~census/601.pdf Abstract: In this paper, which is mainly expository, I will try to set up graphical models for causation, having a bit less than the usual complement of hypothetical counterfactuals. Assuming the invariance of error distributions seems to be essential for causal inference, but the errors themselves need not be invariant. Graphs can be interpreted using conditional distributions, so that we can better address connections between the mathematical framework and causality in the world. I will state the identification issue in terms of conditionals, proceeding mainly by example. Title: Salt and Blood Pressure: Conventional Wisdom Reconsidered Author: D. A. Freedman and D. B. Petitti Date: April 2000 Pub: PDF Url: http://www.stat.berkeley.edu/~census/573.pdf Abstract: The "salt hypothesis" is that higher levels of salt in the diet lead to higher levels of blood pressure, with attendant risk of cardiovascular disease. Intersalt was designed to test the hypothesis, with a cross-sectional study of salt levels and blood pressures in 52 populations. The study is often cited to support the salt hypothesis, but the data are somewhat contradictory. Thus, four of the populations (Kenya, Papua, and two Indian tribes in Brazil) have very low levels of salt and blood pressure. Across the other 48 populations, however, blood pressures go down as salt levels go up-- contradicting the salt hypothesis. Regressions of blood pressure on age indicate that for young people, blood pressure is inversely related to salt intake-- another paradox. This paper discusses the Intersalt data and study design, looking at some of the statistical issues and identifying respects in which the study failed to follow its own protocol. Also considered are human experiments bearing on the salt hypothesis. The effect of salt reduction is stronger for hypertensive subjects than normotensives. Even the effect of a large reduction in salt intake on blood pressure is modest, and publication bias is a concern. To determine the health effects of salt reduction, a long-term intervention study would be needed, with endpoints defined in terms of morbidity and mortality; dietary interventions seem more promising. Funding agencies and medical journals have taken a stronger position favoring the salt hypothesis than is warranted by the evidence, raising questions about the interaction between the policy process and science. Title: The swine flu vaccine and Guillain-Barre syndrome Author: D.A. Freedman and P.B. Stark Date: February 1999 Pub: PDF Url: http://www.stat.berkeley.edu/~census/546.pdf Abstract: Epidemiologic methods were developed to prove general causation: identifying exposures that increase the risk of particular diseases. Courts often are more interested in specific causation: on balance of probabilities, was the plaintiff's disease caused by exposure to the agent in question? Some authorities have suggested that a relative risk greater than 2.0 meets the standard of proof for specific causation. Such a definite criterion is appealing, but there are difficulties. Bias and confounding are familiar problems; individual differences must be considered too. The issues are explored in the context of the swine flu vaccine and Guillain-Barre syndrome. Title: Are There Algorithms that Can Discover Causal Structure? Authors: David Freedman and Paul Humphreys Date: May 1998 Pub: PDF Url: http://www.stat.berkeley.edu/~census/514.pdf Abstract: For nearly a century, investigators in the social and life sciences have used regression models to deduce cause-and-effect relationships from patterns of association. Path models and automated search procedures are more recent developments. However, these formal procedures tend to neglect the difficulties in establishing causal relations, and the mathematical complexities tend to obscure rather than clarify the assumptions on which the analysis is based. This paper focuses on statistical procedures that seem to convert association into causation. Formal statistical inference is, by its nature, conditional. If maintained hypotheses A, B, C, .... hold, then H can be tested against the data. However, if A, B, C, .... remain in doubt, so must inferences about H. Careful scrutiny of maintained hypotheses should therefore be a critical part of empirical work---a principle honored more often in the breach than the observance. Spirtes, Glymour, and Scheines have developed algorithms for causal discovery. We have been quite critical of their work. Korb and Wallace, as well as SGS, have tried to answer the criticisms. This paper will continue the discussion. The responses may lead to progress in clarifying assumptions behind the methods, but there is little progress in demonstrating that the assumptions hold true for any real applications. The mathematical theory may be of some interest, but claims to have developed a rigorous engine for inferring causation from association are premature at best. The theorems have no implications for samples of any realistic size. Furthermore, examples used to illustrate the algorithms are diagnostic of failure rather than success. There remains a wide gap between association and causation. Technical Report No. 446 Title: Concordance between rats and mice in bioassays for carcinogenesis Author: David A. Freedman, Lois S. Gold and Tony H. Lin Date: March 15, 1996; revised April 15, 1996 Abstract: According to current policy, chemicals are evaluated for possible cancer risk to humans at low dose by testing in bioassays, where high doses of the chemical are given to rodents. Thus, risk is extrapolated from high dose in rodents to low dose in humans. The accuracy of these extrapolations is generally unverifiable, since data on humans are limited. However, it is feasible to examine the accuracy of extrapolations from mice to rats. If mice and rats are similar with respect to carcinogenesis, this provides some evidence in favor of inter-species extrapolations; conversely, if mice and rats are different, this casts doubt on the validity of extrapolations from mice to humans. One measure of inter-species agreement is concordance, the percentage of chemicals that are classified the same way as to carcinogenicity in mice and rats. Observed concordance in NCI/NTP bioassays is about 75%, which may seem on the low side-- because mice and rats are closely related species tested under the same experimental conditions. However, observed concordance could under-estimate true concordance, due to measurement error in the bioassays-- a possibility demonstrated by Piegorsch et al. Expanding on this work, we show that the bias in observed concordance can be either positive or negative: an observed concordance of 75% can arise if the true concordance is anything between 20% and 100%. In particular, observed concordance can seriously over-estimate true concordance. Statistical Assumptions as Empirical Commitments Richard A. Berk David A. Freedman http://www.stat.berkeley.edu/~census/berk2.pdf http://www.stat.berkeley.edu/~census/alderman.txt