Problems & Bias

Each of these five charts contains fundamentally the same information. But, by simply increasing the size of parts in the chart, a bias can clearly be seen and felt. This problem of inherent bias must be accepted by anyone attempting to understand or build a religion chart. Once recognized and understood, Charting Pluralism can function as it was intended: as a tool for interfaith dialogue. By starting with the Core, the problem of inherent bias, while still present, will be minimized. This is precisely the reason that the Core must be the starting point in Charting Pluralism, so that “common ground” can be attained.

The Problem of Cross-Cultural Bias

Even if chart-makers do their best to safeguard against bias in the size and position of items in their charts, a particular visual representation may not even then be free of bias. In addition to biases maintained by chart-makers, we must also be aware that the audience of these charts brings bias to the viewing. Any particular person looking at a particular chart may, indeed, interpret that chart according to a set of biases not necessarily shared by the chart-maker. The interpreting eye of a viewer may very likely impose bias, whether consciously or not, upon a chart.

One common bias, for example, that many people in Western culture may bring to this chart is their very concept of religion, itself. Many people who grew up in the Christianity-dominated West understand religion instinctively—and logically so—on the model of Christianity. Due to the denominational structure of Protestantism, many Westerners understand “religion” as a particular, autonomous aspect of daily life, and they understand the nature of “belonging to a religion” as elective and mutually exclusive. These conceptions are heavily influenced by the ideas of “freedom of religion” and “separation of Church and State.” Under the conception of religion held by many Westerners, people are understood to have a choice in what religion they practice. A Catholic can convert and become a Methodist. A Baptist can elect to stop going to the Baptist church and go, instead, to the Episcopalian church. Any particular Christian, whether Orthodox, Catholic or Protestant can decide to leave Christianity altogether through conversion to Judaism, Islam, or any other non-Christian faith. While certainly this freedom is highly valued by Western culture, and rightly so according to many Westerners, the perhaps inevitable result of this freedom to move among the multiplicity of “religions” is that religion is often conceived of as mutually exclusive. Each religion or denomination is viewed as essentially a “club” where you’re either a member or you’re not, and in order to belong to one particular of these “clubs” you can’t belong to any others.

For example, a Lutheran who converts to Roman Catholicism is considered now to be Catholic. Imagine being in a conversation with someone at a party and asking, “What religion are you?” and getting the answer “I’m Catholic and Lutheran.” While some cultural identifiers in Western culture are not conceived of as mutually exclusive (e.g. “I’m Polish and Irish”), religion is considered so by most people. If you’re in liberal enough company, you might be able to say, “Well, I don’t consider myself any particular religion. I go to my Catholic parish every other week, but on off weeks I like to stop by the Methodist church sometimes, the Russian Orthodox church about once a month, the mosque during Ramadan and the synagogue several times a year.” The company at that party might respect or even empathize with your sentiment, but if asked by a third party what religion you are, the attendees of this party would most likely answer, “I think she’s Catholic, but she’s kind of a free spirit.” The liberal party goers would likely not identify you as “belonging” to all these religions.

Additionally, the fact that a question like “What religion are you?” would even be asked in common company in Western culture points to the conception of religion as a separate and autonomous part of daily life. In the minds of Westerners, it occupies certain territory in one’s life. Other religions can be swapped into or out of this territory in a person’s life without having much effect on the non-religious territory. Two school boys in contemporary America can be best of friends, even if one is Jewish and one is Catholic, without much conflict in their daily lives. They attend the same classes at public school, learning the same history, reading the same books, working together on a common science project. They might both ride their bikes to and from school and stop on the way home at their favorite comic book shop. They might spend the afternoon at one or the other’s house alternatively playing video games and doing homework (or playing video games when they really ought to be doing homework). When they get a little older, they might attend the same college, pick the same major, frequent the same campus bar, graduate and get jobs at the same company. Religion would certainly enter their lives along the way. They would realize the differences between their lives at an early age when one would go to church on the weekends and the other to synagogue; they would notice in December that one celebrated Hanukkah and the other celebrated Christmas; the Jewish boy might attend a party thrown in honor of the Catholic boy’s First Communion, and Catholic boy would very likely get an invitation to his friend’s Bar Mitzvah; if the Jewish family keeps kosher, then issues with food might arise when these best friends inevitably stayed over for dinner at each other’s houses; once they grew up and started dating, one or both might limit his potential dates based on shared religion in a way he would not limit his male friends. But these are predictable religious “zones” in Western culture. These two fictional boys can share most of their everyday lives with religion occupying its expected, separate, autonomous zone.

Non-Western “religions” do not always fit this conception so neatly. In fact, an attempt to apply the Western conception of religion to other faith systems can be awfully messy. In India, for instance, practices among those adherents commonly grouped under Hinduism show tremendous variation. Different people honor different deities, or live their lives according different “ways” of faith, but these choices aren’t necessarily mutually exclusive. And they certainly don’t mark one person as being a different “religion” than the other. Parceling out particular sections of the Hindu population based on particular faiths and practices, and calling those sections different “religions” is erroneous at best. On the other hand, some scholars have argued that even attempting to group the multi-various practices of people in India under the heading of “Hinduism” is itself highly problematic. The fluidity among these faiths and practices is so complex that an attempt either to separate them or unite them under categories following the Western concept of “religion” doesn’t work. Additionally, the practice of what Westerners call “religion” in many non-Western cultures is so pervasive in people’s daily lives that calling part of those lives “religious” and part “secular” is also problematic. Doing so would reflect the Western bias. In reality, many so-called “religions” outside of Western culture are so thoroughly woven into the daily lives of the people who practice these faiths that separating out the “religious” parts is impossible; in these contexts it is perhaps easier to think of these faiths as a “way of life” rather than “religion” in the Western sense. Moreover, in some areas of the world where named categories for particular belief systems do exist, these “religions” are not necessarily mutually exclusive. It is very common in Japan, due to a variety of cultural circumstances, for people to practice both Buddhism and Shinto, and combining these practices is not a fringe movement, but rather a regularity.

The non-applicability of the Western concept of “religion” to other cultures goes to show how easily any classification system can lead to misunderstanding. The term “religion” doesn’t need to be tossed out altogether in the discussion of cross-cultural phenomena, but the Western bias surrounding this term does need to be recognized before its meaning can be widened by both those who invoke the term and those who comprehend it in context. In creating and comprehending charts as part of the Charting Pluralism project, all participants must be aware of where bias might exist in their own understandings of the charts. For example, following from the explanation above, it is useful to point out that each branch of a particular chart doesn’t necessarily represent a different “religion” in the western sense. Take, for example, the varieties of Hinduism represented on the main chart:

These branches represent different ways in which Hinduism is practiced. Think of them as different facets of the same gem. It might, however, be easy for Westerners to confuse these for different and mutually exclusive “religions” because the Christian arm of the chart represents a reality of denominational categories to which they have become acculturated:

Just as on the Hinduism area of the chart, the Christian area represents different facets of Christianity. While these different facets hold the connotation of mutual exclusivity within the context of Western culture, that doesn’t make it any more or less accurate a representation of the varieties of Christian practice. The arms of Christianity on this chart will not necessarily have a one-to-one conceptual correlation for any other grouping on the chart. In fact, even within Christianity, the facets of Orthodoxy represented on the chart can’t be said to represent the same conceptual categories as the facets or Protestantism or of Catholicism. Likewise, we cannot use this chart to generalize the conceptual categories of Western religion outside the Western context. Or even sometimes within it. Being a Conservative Jew does not necessarily mean the same thing within the context of Judaism as being a Methodist means within the context of Christianity, nor does either of these conceptual categories apply neatly to being a Shi’ite within the Islamic context, or practicing Bhakti within the Hindu context.

It would be easy, however, for a person viewing the main chart through a Western, Christian bias to generalize his or her own conception of Western Christianity to other areas of the chart. It would be logical, in fact, for a viewer of Western Christian bias to infer from this chart that Brahmanism is one among three major Hindu religions, and that being a Sunni as opposed to a Shi’ite is just like being a Catholic as opposed to a Lutheran. While these categories are not without similarity, remembering that the correspondence cannot be one-to-one is vitally important for recognizing one’s own biases in viewing and understanding these charts.

The best way to combat bias, both from the chart-maker and from the chart-viewer to remain in constant dialog, to acknowledge potential bias up front and always to be on the look-out for biases we might not have noticed or realized previously. Perfect scholarship is impossible, both in the study of comparative religions and most any other academic discipline one can imagine. Bias is always brought to the table, whether that bias is strong, subtle or even unknown to the scholar and/or his audience. But if we shrank from bias, dialog and learning would never happen. The Charting Pluralism project seeks to be as upfront as possible with the methodological problems inherent in our endeavor and to be self-reflective in searching for and exposing bias, especially in ourselves. Exposing our own biases is a victory for mutual discussion and understanding, and it is an essential part of the Charting Pluralism endeavor.

Methodological Issues in Compiling Statistics on Adherent Populations

The gathering of statistics on numbers of adherents to particular religions may seem like a straightforward, scientific endeavor that would not be clouded by bias. However, like any method of investigating human culture, statistics gathering can contain just as many biases as more seemingly subjective forms of cultural exploration. Bias can enter into statistics gathering both on the parts of the researcher and of the subjects.

The researcher setting out to count religious adherents, for example, must make a number of decisions on how to compile statistics, including what methods and what sources to use. Chosing a representative sample of respondents is an imperfect science and fraught with potential for bias; mistakes on the part of the researcher can skew the sample, but so can simple problems of access. Imagine, for example, a cult that keeps all or part of its population sheltered from the wider world. Using a method like a telephone survey would be useless for collecting data to include religions that eschew modern technology. Looking a records of church membership will yield different results than surveying a sample of the local population. The questions chosen by the researcher to ask in such a survey will also result in a variety of statistical responses; imagine the difference in answers you’d get for a question like “What religion are you?” and a question like “Do you go to church?”

Such decisions can and must be made, however, and the careful researcher can balance the elements of a survey to minimize bias. Looking at the 2001 American Religious Identification Survey (Kosmin, Mayer, Keysar), we can see how this balancing act plays out in one example. Decisions had to be made in the methodology due to issues of practicality and funding. The survey’s principal investigators made the decision to conduct their survey by phone, even though they acknowledge the limitations of this method. They highlight the problem especially when dealing with recent immigrant groups. The researchers point out to their readers that the survey data may be at a disadvantage in calculating the adherent statistics for recent immigrants because of two main factors: 1) because the answering of a phone survey can be a strange and alienating experience for a new immigrant having come from a culture unaccustomed to such events and 2) many new immigrants still speak primarily their native languages and due to the limits of funding the survey had to be conducted in English alone. Despite these acknowledged limitations, the survey authors argue that the disadvantages afforded by these factors are not statistically significant enough to interfere with the usefulness of the survey data.

Respondent bias must also be accounted for, even by the most fastidious researcher with a survey plan designed to provide checks and balances. One of the problems of gathering statistics on world religions is that the self-perception of religious affiliation on the part of respondents can clash with the researcher’s conception of what aspects of religious affiliation that he/she is intending to investigate. One common example is the concern on the part of researchers that the numbers of adherents for certain religions in their surveys will become inflated because they are so widespread or well established. For example, the researcher studying religious affiliation in a certain region of the world might survey participants to asking simply that… What religion are you? Are you Christian? Muslim? Jewish? Buddist? Hindu? Something else? Perhaps the question would even aim at a more specific answer, asking for the particular religious tradition or denomination… Baptist Christian? Conservative Jew? Sunni Muslim? No matter the specificity, the concern remains the same: Will people tell “the truth”? Or will people answer based on tradition or habit?

In regions of the world where Christianity is the dominant religion, so statistics speak truly to its dominance? Or does Christianity’s historical entrenchment inflate the numbers. If presented with a survey asking for religious affiliation, how will people answer? Practicing Christians who attend church regularly and partake of the majority of rituals and sacraments will answer “Christian” or some denomination thereof. This is certainly not a problem for researchers. Where the problem comes in is when researchers ask themselves: Do all respondents who answer “Christian” fit this profile? In Christian dominant countries, chances are the answer is “no.” The likelihood is that many respondents will answer “Christian” even if they only ever go to church on Easter and Christmas.