LDS Church Growth, Member Activity, and Convert Retention:
Review and Analysis By David Stewart

Return to Table of Contents

Chapter III-16: Limitations of Data

The natural human tendency for certainty has often led to overextended conclusions from inadequate data, ranging from ancient myths that attributed unexplained natural phenomena to the will of the gods, to contemporary theories about the origin of the universe that make detailed interpolations far beyond that which can be demonstrated by existing data. Mankind has historically been unable to admit the tentative nature of dominant theories, and seems to prefer even erroneous conclusions over admission of ignorance. Even in the modern age, the strength of evidence for prevailing theories in a variety of disciplines is, in my view, often overstated.

The study of church growth is particularly inclined to agenda and overstatement, in view of both the limited quality of available data and the emotionally charged nature of the topic. Although a wide array of statistical indicators from various sources, it is important to keep in mind the limitations of the data and to recognize what conclusions cannot be drawn in addition to those that can. When conclusions can be drawn, it is necessary to assess the strength of the evidence to distinguish between well-documented conclusions and more tentative ones. This requires objectivity in allowing the data to determine the conclusion, rather than approaching the data with a specific agenda. It also requires transparency in candidly presenting data that may not support one's conclusion alongside with data that does, and carefully considering limitations or potential flaws of a theory. Public peer review is also an important element, as even thoughtful scholars trying to be objective may overlook limitations or inadequacies which may be readily apparent to others. Not every objection is necessarily meritorious, but an open discourse is important in determining which conclusions can be defended and which cannot.

The number of members who express belief in church teachings, the number of members worldwide who attend church each week, and the number who adhere to other church teachings, cannot be ascertained definitively from any combination of official and independent data. Various indicators -- especially when collaborated from supporting sources -- may lead us to achieve reasonably close approximations on a variety of topics, and to conclude with a high probability that a figure which we seek to ascertain lies within a narrow range. Yet even well-founded approximations are just that. Just as the Heisenberg uncertainty principle in physics asserts that a particle's velocity and location cannot be simultaneously known, even if it can be approximated by a probability distribution, so too in the study of religious sociology we find intrinsic limitations which allow helpful approximations but often preclude absolute precision. This does not mean that such data are not relevant or important, nor does it allow conclusions based on compelling data to be lightly dismissed, but it does require awareness of the data's limitations as well as strengths.