Appendix 1: Methodology
Navigate this page:
This is the third time the Pew Forum
has measured restrictions on religion around the globe.8 This report, which includes data through the
year ending in mid-2010, follows the same methodology as the Pew Forum’s
December 2009 report, “Global Restrictions on Religion,” and its August 2011
report, “Rising Restrictions on Religion,” with one major difference: Instead
of reporting two-year averages, as the previous reports did, this report
assesses restrictions on an annual basis. Future studies in this series also
will report annual data.
The Pew Forum uses two 10-point
indexes – the Government Restrictions Index (GRI) and the Social Hostilities
Index (SHI) – to rate 197 countries and self-governing territories on their
levels of restrictions.9 The August
2011 report assessed “substantial” changes in restrictions by comparing each
country’s average scores for the two-year period from mid-2006 to mid-2008 with
its average scores for the overlapping two-year period from mid-2007 to
mid-2009.10 Using overlapping two-year periods was a
precaution intended to minimize minor, annual fluctuations until the Pew Forum
built up data from a sufficient number of years to be able to distinguish
important, longer-term trends. That threshold has now been crossed. This report
analyzes changes in restrictions on an annual basis, looking at four years,
ending in mid-2007, mid-2008, mid-2009 and mid-2010. It categorizes the amount
of change in each country’s scores in two ways, numerically and by percentile.
First, countries are grouped into
categories depending on the size of the numeric change in their scores from
year to year on the two indexes: changes of two points or more in either
direction; changes of at least one point but less than two points; changes of
less than one point; or no change at all. (See chart at right and three charts in the Summary of Findings: Changes in Government Restrictions, Changes in Social Hostilities and Changes in Overall Restrictions.)
Changes in overall levels of
restrictions are calculated for each country by comparing its scores on both
indexes (the GRI and the SHI) from year to year. When a country’s scores on the
GRI and the SHI changed in the same direction (both increased or both
decreased), the greater amount of change determined the category. For instance,
if the country’s GRI score increased by 0.8 and its SHI score increased by 1.5,
the country was put into the overall “1.0-1.9 increase” category. When a
country’s score increased on one index but decreased on the other, the
difference between the amounts of change determined the grouping. For example,
if the country’s GRI score increased by 2.0 and its SHI score decreased by 1.5,
the country went into the overall “0.1-0.9 increase” category. When a country’s
score on one index stayed the same, the amount of change on the other index was
used to assign the category.
Second, this report categorizes the
levels of government restrictions and social hostilities in each country by
percentiles. As the benchmark, it uses the results from the baseline year (the
year ending in mid-2007). Scores in the top 5% on each index in mid-2007 were
categorized as “very high.” The next highest 15% of scores were categorized as
“high,” and the following 20% were categorized as “moderate.” The bottom 60% of
scores were categorized as “low.” See the table above for the index score thresholds
as determined from the mid-2007 data. These thresholds are applied to all
subsequent years of data.
Overview of Procedures
The methodology used by the Pew Forum
to assess and compare restrictions on religion was developed by senior
researcher and director of cross-national data Brian J. Grim in consultation
with other members of the Pew Research Center staff, building on a methodology
that Grim and professor Roger Finke developed while at Penn State University’s
Association of Religion Data Archives.11 The goal was
to devise quantifiable, objective and transparent measures of the extent to
which governments and societal groups impinge on the practice of religion. The
findings were used to rate 197 countries and self-governing territories on two
indexes that are reproducible and can be periodically updated.
This research goes beyond previous
efforts to assess restrictions on religion in several ways. First, the Pew
Forum coded (categorized and counted) data from 19 published cross-national
sources, providing a high degree of confidence in the findings. The Pew Forum
coders looked to the sources for only specific, well-documented facts, not
opinions or commentary.
Second, the Pew Forum staff used
extensive data-verification checks that reflect generally accepted best
practices for such studies, such as double-blind coding (coders do not see each
other’s ratings), inter-rater reliability assessments (checking for consistency
among coders) and carefully monitored protocols to reconcile discrepancies
among coders.
Third, the Pew Forum coding took into
account whether the perpetrators of religion-related violence were government
or private actors. The coding also identified how widespread and intensive the
restrictions were in each country.
Fourth, one of the most valuable
contributions of the indexes and the questions used to construct them (see the
section on The Coding Instrument) is their ability to chart change
over time.
Countries and Territories
The 197 countries and
self-administering territories covered by the study contain more than 99.5% of
the world’s population. They include 191 of the 192 member states of the United
Nations as of mid-2010 plus six self-administering territories – Kosovo, Hong
Kong, Macau, the Palestinian territories, Taiwan and Western Sahara.12 Reporting on
these territories does not imply any position on what their international
political status should be, only recognition that the de facto situations in
these territories require separate analysis.
Although the 197 countries and
territories vary widely in size, population, wealth, ethnic diversity,
religious makeup and form of government, the study does not attempt to adjust
for such differences. Poor countries are not scored differently on the indexes
than wealthy ones. Countries with diverse ethnic and religious populations are
not “expected” to have more social hostilities than countries with more
homogeneous populations. And democracies are not assessed more leniently or
harshly than authoritarian regimes.
Information Sources
The Pew Forum identified 19 widely
available, frequently cited sources of information on government restrictions
and social hostilities involving religion around the world. This study includes
four sources that were not used in the baseline report on religious
restrictions. (See Information Sources for more details on the new information sources.)
The primary sources, which are listed
below, include reports from U.S. government agencies, several independent,
nongovernmental organizations and a variety of European and United Nations
bodies. Although most of these organizations are based in Western countries,
many of them depend on local staff to collect information across the globe. As
previously noted, the Pew Forum did not use the commentaries, opinions or
normative judgments of the sources; the sources were combed only for factual
information on specific policies and actions.
Primary Sources
1. Country
constitutions
2. U.S. State
Department annual reports on International Religious Freedom
3. U.S.
Commission on International Religious Freedom annual reports
4. U.N. Special
Rapporteur on Freedom of Religion or Belief reports
5. Human Rights
First reports in first and second years of coding; Freedom House reports
in third and fourth years of coding
6. Hudson
Institute publication: “Religious Freedom in the World” (Paul Marshall)
7. Human Rights
Watch topical reports
8. International
Crisis Group country reports
9. United
Kingdom Foreign & Commonwealth Office annual report on human rights
10. Council of
the European Union annual report on human rights
11. Amnesty
International reports
12. European
Network Against Racism Shadow Reports
13. United
Nations High Commissioner for Refugees reports
14. U.S. State
Department annual Country Reports on Terrorism
15.
Anti-Defamation League reports
16. U.S. State
Department Country Reports on Human Rights Practices
17. U.S. National
Counterterrorism Center’s Worldwide Incident Tracking System
18. Uppsala
University’s Uppsala Conflict Data Program, Armed Conflict Database
19. Human Rights
Without Frontiers “Freedom of Religion or Belief” newsletters
U.S. government reports with
information on the situation in the United States
20. U.S.
Department of Justice “Religious Freedom in Focus” newsletters and reports
21. FBI Hate
Crime Reports
As noted, this study includes four
sources that were not included in the Pew Forum’s first report on global
restrictions on religion: Freedom House reports; Uppsala University’s Armed
Conflict Database; the “Freedom of Religion or Belief” newsletters of Human
Rights Without Frontiers; and the U.S. government’s Worldwide Incident Tracking
System (WITS).
The Freedom House reports have
replaced Human Rights First reports, which have not been updated since
mid-2008. The Uppsala Armed Conflict Database provides information on the
number of people affected by religion-related armed conflicts, supplementing
other sources. The Human Rights Without Frontiers “Freedom of Religion or
Belief” newsletters have partially replaced the Hudson Institute publication,
“Religious Freedom in the World” (by Paul Marshall), which has not been updated
since its release in 2008. Human Rights Without Frontiers is a nongovernmental
organization based in Brussels, with affiliated offices throughout the world.
The Hudson Institute publication still offers useful background on certain
standing laws but no longer provides information on new or changing
restrictions. The U.S. government’s WITS database has provided greater detail
on the number of people affected by religion-related terrorism than either the
State Department’s International Religious Freedom reports or the State
Department’s annual Country Reports on Terrorism. Until May 2012, WITS was a
publicly available database maintained by the U.S. National Counterterrorism
Center, a government organization that is part of the Office of the Director of
National Intelligence; it is no longer available online.
While some of the increases in
religious restrictions noted in this study could reflect the use of more
up-to-date and/or better information sources, Pew Forum staff monitor the
impact of source information variability each year and have found no evidence
of overall informational bias. (For additional discussion, see the Potential
Biases section.)
The Coding Instrument
As explained in more detail below,
Pew Forum staff developed a battery of questions similar to a survey
questionnaire. Coders consulted the primary sources in order to answer the
questions separately for each country. While the State Department’s annual
reports on International Religious Freedom generally contained the most
comprehensive information, the other sources provided additional factual detail
that was used to settle ambiguities, resolve contradictions and help in the
proper scoring of each question.
The
questionnaire, or coding instrument, generated a set of numerical measures on
restrictions in each country. It also made it possible to see how government
restrictions intersect with broader social tensions and incidents of violence
or intimidation by private actors. The coding instrument with the list of
questions used for this report is shown in the Summary of Results.
The coding process required the
coders to check all the sources for each country. Coders determined whether
each source provided information critical to assigning a score; had supporting
information but did not result in new facts; or had no available information on
that particular country. Multiple sources of information were available for all
countries and self-administering territories with populations greater than 1
million. More than three-in-four of the countries and territories analyzed by
the Pew Forum were multi-sourced; only small, predominantly island, countries
had a single source, namely, the State Department reports.
Coding the United States presented a
special problem since it is not included in the State Department’s annual
reports on International Religious Freedom. Accordingly, Pew Forum coders also
looked at reports from the U.S. Department of Justice and the FBI on violations
of religious freedom in the United States, in addition to consulting all the
primary sources, including reports by the United Nations, Human Rights Watch,
Freedom House, the International Crisis Group and the U.K. Foreign & Commonwealth
Office, many of which contain data on the United States.
The Coding Process
The Pew Forum employed strict
training and rigorous coding protocols to make its coding as objective and
reproducible as possible. Coders worked directly under a senior researcher’s
supervision, with additional direction and support provided by other Pew Forum
researchers. The coders underwent an intensive training period that included a
thorough overview of the research objectives, information sources and
methodology.
Countries were double-blind coded by
two coders (coders did not see each other’s ratings), and the initial ratings
were entered into an electronic survey with details on each incident cataloged
in Microsoft Excel spreadsheets. The coders began by filling out the coding
instrument for each country using the information source that had the most
comprehensive information, typically the State Department reports. The protocol
for each coder was to answer every question on which information was available
in the initial source. Once a coder had completed that process, he or she then
turned to the other sources. As new information was found, this was also coded
and the source duly noted. Whenever ambiguities or contradictions arose, the
source providing the most detailed, clearly documented evidence was used.
After two coders had separately
completed the coding instrument for a particular country, their scores were
compared by a senior researcher. Areas of discrepancy were discussed at length
with the coders and were reconciled in order to arrive at a single score on
each question for each country. The data for each country were then combined
into a master file, which was imported into a relational database.
Throughout this process, the coding
instrument itself was continually monitored for possible defects. The questions
were designed to be precise, comprehensive and objective so that, based on the
same data and definitions, the coding could be reliably reproduced by others
with the same results.
Pew Forum staff generally found few
cases in which one source contradicted another. When contradictions did arise –
such as when sources provided differing estimates of the number of people
displaced due to religion-related violence – the source that cited the most
specific documentation was used. The coders were instructed to disregard broad,
unsubstantiated generalizations regarding abuses and to focus on reports that
contained clear, precise documentation and factual details, such as names,
dates and places where incidents occurred.
Inter-rater reliability statistics
were computed by comparing the coders’ independent, blind ratings. The Pew
Forum took scores from one coder for the 197 countries and compared them with
another coder’s scores for the same questions, computing the degree to which
the scores matched. These measures were very high, with an average score of 0.8
or above on the key variables. Scores above 0.8 on a zero-to-one scale are
generally considered very good, and scores around 0.7 are generally acceptable.
The Pew Forum’s overall inter-rater reliability average across all the
variables coded was greater than 0.8 for each year.
The data-verification procedures went
beyond the inter-rater reliability statistics. They also involved comparing the
answers on the main measures for each country with other closely related
questions in the data set. This provided a practical way to test the internal
reliability of the data.
Pew Forum staff also checked the
reliability of the coded data by comparing them with similar, though more
limited, religious restrictions data sets. In particular, published government
and social regulation of religion index scores are available from the
Association of Religion Data Archives (for three years of data) and the Hudson
Institute (for one year of data), which makes them ideal measures for
cross-validation. The review process found very few significant discrepancies
in the coded data; changes were made only if warranted by a further review of
the primary sources.
Restriction of Religion Indexes
The Government Restrictions Index is
based on 20 indicators of ways that national and local governments restrict
religion, including through coercion and force. The Social Hostilities Index is
based on 13 indicators of ways in which private individuals and social groups
infringe on religious beliefs and practices, including religiously biased
crimes, mob violence and efforts to stop particular religious groups from
growing or operating. The study also counted the number and types of documented
incidents of religion-related violence, including terrorism and armed conflict.
Government Restrictions Index
Coding multiple indicators makes it
possible to construct a Government Restrictions Index of sufficient gradation
to allow for meaningful cross-national comparisons. An additional advantage of
using multiple indicators is that it helps mitigate the effects of measurement
error in any one variable, providing greater confidence in the overall measure.
The Pew Forum coded 20 indicators of
government restrictions on religion (see the Summary of Results). These 20
items were added together to create the GRI. In two cases, these items
represent an aggregation of several closely related questions: Measures of five
types of physical abuses are combined into a single variable (GRI Q.19); and
seven questions measuring aspects of government favoritism are combined into an
overall favoritism scale (GRI Q.20 is a summary variable showing whether a
country received the maximum score on one or more of the seven questions).
A test of whether the 20 items were
statistically reliable as a single index produced a scale reliability
coefficient greater than 0.9 for each year. Since coefficients of 0.7 or higher
are generally considered acceptable, it was appropriate to combine these 20
items into a single index.
The GRI is a fine-grained measure
created by adding the 20 items on a 0-to-10 metric, with zero indicating very
low government restrictions on religion and 10 indicating extremely high
restrictions. This involved two general calculations. First, the 20 questions
that form the GRI were standardized so that each variable had an identical
maximum value of one point, while gradations among the answers allowed for
partial points to be given for lesser degrees of the particular government
restriction being measured. Second, the overall value of the index was
proportionally adjusted – so that it had a maximum value of 10 and a possible
range of zero to 10 – by dividing the sum of the variables by two.
Social Hostilities Index
In addition to government
restrictions, violence and intimidation in societies also can limit religious
beliefs and practices. Accordingly, Pew Forum staff tracked more than a dozen
indicators of social impediments on religion. Once again, coding multiple
indicators made it possible to construct an index that shows gradations of
severity or intensity and allows for comparisons among countries. The Summary
of Results contains the 13 items used by Pew Forum staff to create the Social
Hostilities Index.
As with the Government Restrictions
Index, various types of violence and intimidation were combined. A test of
whether these 13 items were statistically reliable as a single index produced a
scale reliability coefficient of 0.9 or higher for each year. Since
coefficients of 0.7 or higher are generally considered acceptable, it was
statistically appropriate to combine these items into a single index.
The SHI was constructed by adding
together the 13 indicators based on a 0-to-10 metric, with zero indicating very
low social impediments to religious beliefs and practices and 10 indicating
extremely high impediments. This involved two general calculations. First, the
various questions that form the index were standardized so that each variable
had an identical maximum value of one point, while gradations among the answers
allowed for partial points to be given for lesser degrees of the particular
hostilities being measured. Second, the indicators were added together and set
to have a possible range of zero to 10 by dividing the sum of the variables by
1.3.
Notes on Fluctuations in Certain
Results
Some fluctuations on individual
measures have resulted from minor variations in coding procedures and are not
as significant as they may appear. This was especially the case for GRI Q.3 and
SHI Q.4.
As shown in the Summary of Results
for GRI Q.3 (“Taken together, how do the constitution/basic law and other
national laws and policies affect religious freedom?”), the number of countries
with a score of zero on that question (indicating no restrictions) increased
from 63 in the year ending in mid-2007 to 96 in the year ending in mid-2009. It
then dropped to 75 countries in the year ending in mid-2010. However, these
fluctuations may be largely attributable to variations in the coding procedures
across the years. Retrospective analysis indicates that during the first year
coded (year ending in mid-2007), the coders were more likely to give countries
a partial score (0.33) on this question than in subsequent years. Some
recalibration in the most recent year brought the coding closer to the criteria
used in the initial year. The retrospective analysis suggests that in the year
ending in mid-2009, the coders had a higher bar for assigning a score of 0.33
(they considered restrictive laws or policies alone to be insufficient; there
had to be clear harassment or abuses of religious groups or individuals). In
the most recent year (year ending in mid-2010), coders assigned 0.33 if there
were restrictive laws or policies only at the local level, which is consistent
with the intent of the question. The effect of these variations in coding
criteria is relatively small: A difference of 0.33 from year to year on this
question produces a change of just 0.17 on the Government Restrictions Index
because each question on the GRI is worth a half point (0.33/2 = 0.17).
As noted earlier in the methodology,
some of the increase in religion-related terrorism (SHI Q.4) found in this
study could reflect the use of new source material that provided greater detail
on terrorist activities than the sources used in the baseline report.
Specifically, in coding terrorist activities that occurred from mid-2008 to
mid-2009 and from mid-2009 to mid-2010, coders used the Worldwide Incidents
Tracking System (WITS), the U.S. National Counterterrorism Center’s database of
terrorist incidents. Because the same sources were used for both periods, these
are the most appropriate years to compare.
Finally, it is important to note that
situations within countries may have changed since the end of the periods
studied. For instance, the Arab Spring uprisings in the Middle East and North
Africa occurred in late 2010 and 2011, after the period covered by this study.
Also, the formal division of Sudan into two separate countries took place in
2011. Subsequent Pew Forum reports on global restrictions on religion will
assess Sudan and South Sudan separately, but this report covers the formerly
undivided country.
Note on New Analysis in This Report
An analysis of government
restrictions and social hostilities confirms a pattern initially discussed in
the baseline report of this study. When all 197 countries and
self-administering territories are plotted on a chart comparing their average
scores on the Government Restrictions Index and the Social Hostilities Index
for the four-year period from mid-2006 to mid-2010, it is apparent that the two
measures tend to move together. Running through the graph is the so-called
regression line, which plots how scores on one index are related to scores on
the other index.
As in the baseline report, the upward
slope of the line indicates that higher scores on one index generally are
associated with higher scores on the other. Many countries are clustered in the
lower left corner, showing that they are low on both types of restrictions.
Though the remaining countries are fairly dispersed, most still follow the
direction taken by the regression line, and very few are located in the upper
left or lower right corners of the graph. This means that, in general, it is
rare for countries that are high in social hostilities to be low on government
restrictions, or for those that are high on government restrictions to be low
in social hostilities.
The association between government
restrictions and social hostilities is also visible when comparing how each
question on the indexes relates to the overall level of restrictions or
hostilities. As shown in the chart in the Summary of Findings of the report, the average level
of social hostilities involving religion for the four-year period from mid-2006
to mid-2010 tends to be higher in countries with each type of
government restriction on religion than in countries without the
government restrictions. The average level of social hostilities among the
countries with very high levels of government favoritism toward a particular
religion or religions (SHI = 4.8) is much higher than the average level of
social hostilities among countries without high levels of government favoritism
(1.3). Other government actions that are strongly associated with social
hostilities involving religion are the use of force against religious groups
(SHI = 4.7), failing to intervene to stop religious discrimination (4.7) and
limiting conversion from one religion to another (4.1).13 This same
pattern is present to varying degrees for all 20 GRI questions. (See further
discussion on GRI Q.20 below.)
The chart also provides information
on situations where social hostilities are lowest. Specifically, the right-hand
column shows that social hostilities were lowest among countries where
governments do not harass or intimidate religious groups (SHI = 0.7), national
laws and policies protect religious freedom (0.8), governments do not interfere
with religious worship or practices (0.9) and governments do not use force
against religious groups (1.0).
Likewise, certain types of social
hostilities involving religion are more likely to be associated with higher
government restrictions on religion. Sectarian or communal violence between
religious groups has the strongest association with government restrictions on
religion. The average level of government restrictions among the countries with
sectarian violence (GRI = 5.0) is much higher than among countries without such
violence (2.4), as shown in the chart in the Summary of Findings. Other social hostilities that
are strongly associated with government restrictions are hostilities over
conversion from one religion to another (GRI = 4.9), violence or the threat of
violence to enforce religious norms (4.8), religion-related terrorist violence
(4.8) and groups coercively dominating public life with their perspective on
religion (4.8).14
This chart also provides information
on situations where government restrictions are lowest. Specifically, the
right-hand column of the chart shows that government restrictions are, on
average, lowest in countries where there are no violent acts resulting from
tensions between religious groups (GRI = 1.4), no crimes or malicious acts
motivated by religious hatred (1.6), no groups dominating public life with
their perspective on religion (1.8) and no incidents of violence stemming from
hostility over religious conversion (1.9).
These charts show that the presence
of any of the 20 types of government restrictions on religion is, on average,
associated with higher overall social hostilities involving religion. Likewise,
the presence of any of the 13 types of social hostilities is, on average,
associated with higher overall government restrictions. Given the general
association between government restrictions and social hostilities discussed at
the beginning of this section, some association between the different types of
government restrictions and social hostilities was expected. But it was not
necessarily expected to find such a consistent pattern among all the questions.
This does not necessarily mean that increased government restrictions cause
social hostilities or that increased hostilities cause government restrictions.
It may be that increases in government restrictions trigger social hostilities,
and that increases in social hostilities trigger government restrictions.15
As part of this new analysis, the
study also introduces a new way of analyzing GRI Q.20, which is an overall
government favoritism scale created by combining seven questions measuring
aspects of government favoritism of religion. Since GRI Q.20 is a scale with
many possible values rather than a single question with distinct categories, it
is not possible to conduct the types of comparisons described in this section
unless the scale is converted into categories. Following the example of putting
GRI and SHI into categorical levels, GRI Q.20 is also treated as a Government
Favoritism Index. Accordingly, the study categorizes the levels of government
favoritism by percentiles. For this analysis, countries with average scores
across the four years of the study in the top 5% on GRI Q.20 (11 countries)
were categorized as “very high.” The next highest 15% of scores (30 countries)
were categorized as “high,” and the following 20% were categorized as
“moderate” (39 countries). The bottom 60% of scores were categorized as “low”
(188 countries). This categorization permits the same analysis of this question
as for the other component questions of the indexes, as shown in the following
chart.
Note on the Effects of Consolidating
to a New Database
For all four years of this study,
information on the number, types and locations of incidents of government force
and social violence toward religious groups as well as deference to religious
authorities in matters of law were coded at the province level. (See example of
data coding for India (PDF) from the December 2009 baseline report.) Each year,
the province numbers were summed and put into separate country-level files.
Since the publication of the August 2011 report, Pew Forum staff have created a
database that integrates all four years of province- and country-level data on
religious restrictions. During this process, Pew Forum staff reviewed any
discrepancies between province files and the sums that had been transferred to
the country files and made appropriate corrections. The adjustments made were
relatively minor and had small effects on index scores for countries, on
average less than 0.005 points on the 10-point indexes. Consolidating the four
years of data into a database also entailed a review of the data on harassment
of religious groups. In particular, instances of harassment from the year
ending in mid-2007 were stored as open-ended questions, and in a few cases they
were recoded to match the categories used in the subsequent years.
Additional Analysis in the Study
As in the 2011 report, this study
reports a further summary of the number of countries where specific religious
groups faced government or social harassment. This is essentially a
cross-tabulation of GRI Q.11 (“Was there harassment or intimidation of
religious groups by any level of government?”) and the first type of religious
hatred or bias measured in SHI Q.1a. (“Did individuals face harassment or
intimidation motivated by religious hatred or bias?”). For purposes of this
study, the definition of harassment includes any mention in the primary sources
of an offense against an individual or group based on religious identity. Such
offenses may range from physical attacks and direct coercion to more subtle
forms of discrimination. But prejudicial opinions or attitudes, in and of
themselves, do not constitute harassment unless they are acted upon in a
palpable way.
As noted above, this study provides
data on the number of countries in which different religious groups are
harassed or intimidated. But the study does not assess either the severity or
the frequency of the harassment in each country. Therefore, the results should
not be interpreted as gauging which religious group faces the most harassment
or persecution around the world.
Religion-Related Terrorism and Armed
Conflict
Terrorism and war can have huge
direct and indirect effects on religious groups, including destroying religious
sites, displacing whole communities and inflaming sectarian passions.
Accordingly, the Pew Forum tallied the number, location and consequences of
religion-related terrorism and armed conflict around the world, as reported in
the same primary sources used to document other forms of intimidation and
violence. However, war and terrorism are sufficiently complex that it is not
always possible to determine the degree to which they are religiously motivated
or state sponsored. Out of an abundance of caution, this study does not include
them in the Government Restrictions Index. They are factored instead into the
index of social hostilities involving religion, which includes one question
specifically about religion-related terrorism and one question specifically
about religion-related war or armed conflict. In addition, other measures in
both indexes are likely to pick up spillover effects of war and terrorism on
the level of religious tensions in society. For example, hate crimes, mob
violence and sectarian fighting that occur in the aftermath of a terrorist
attack or in the context of a religion-related war would be counted in the
Social Hostilities Index, and laws or policies that clearly discriminate
against a particular religious group would be registered on the Government
Restrictions Index.
For the purposes of this study, the
term “religion-related terrorism” is defined as premeditated, politically
motivated violence perpetrated against noncombatant targets by subnational
groups or clandestine agents that have some identifiable religious ideology or
religious motivation. It also includes acts carried out by groups that have a
nonreligious identity but target religious personnel, such as clergy. Readers
should note that it is the political character and motivation of the groups,
not solely the type of violence, that is at issue here. For instance, a bombing
would not be classified as religion-related terrorism if there was no clearly
discernible religious ideology or bias behind it unless it was directed at
religious personnel. Religion-related war or armed conflict is defined as armed
conflict (a conflict that involves sustained casualties over time or more than
1,000 battle deaths) in which religious rhetoric is commonly used to justify
the use of force, or in which one or more of the combatants primarily
identifies itself or the opposing side by religion.
Potential Biases
As noted earlier, the primary sources
indicate that the North Korean government is among the most repressive in the
world, including toward religion. Because of independent observers’ lack of
regular access to North Korea, however, the sources are unable to provide the
kind of specific, timely information that forms the basis of this report.
Therefore, North Korea is not included on either index.
This raises two important issues
concerning potential information bias in the sources. The first is whether
other countries that limit outsiders’ access and that may seek to obscure or
distort their record on religious restrictions were adequately covered by the
sources. Countries with relatively limited access have multiple primary sources
of information that the Pew Forum used for its coding. Each is also covered by
other secondary quantitative data sets on religious restrictions that have used
a similar coding scheme, including earlier years of coded State Department
report data produced by Grim at Penn State’s Association of Religion Data
Archives (ARDA) project (three data sets); independent coding by experts at the
Hudson Institute’s Center for Religious Liberty using indexes also available
from ARDA (one data set); and content analysis of country constitutions
conducted by the Becket Fund for Religious Liberty (one data set). Pew Forum
staff used these for cross-validation. Thus, contrary to what one might expect,
even most countries that limit access to information tend to receive fairly extensive
coverage by groups that monitor religious restrictions.
The second key question – the
flipside of the first – is whether countries that provide freer access to
information receive worse scores simply because more information is available
on them. As described more fully in the methodology in the baseline report, Pew
Forum staff compared the length of State Department reports on freer-access
countries with those of less-free-access countries. The comparison found that
the median number of words was approximately three times as large for the
limited-access countries as for the open-access countries. This suggests that
problems in freer-access countries are generally not overreported in the State
Department reports.
Only when it comes to religion-related
violence and intimidation in society do the sources report more problems in the
freer-access countries than in the limited-access ones. However, the Social
Hostilities Index includes several measures – such as SHI Q.8 (“Did religious
groups themselves attempt to prevent other religious groups from being able to
operate?”) and SHI Q. 11 (“Were women harassed for violating religious dress
codes?”) – that are less susceptible to such reporting bias because they
capture general social trends or attitudes as well as specific incidents. With
these limitations in mind, it appears that the coded information on social
hostilities is a fair gauge of the situation in the vast majority of countries
and a valuable complement to the information on government restrictions.
Data on social impediments to
religious practice can more confidently be used to make comparisons among
countries with sufficient openness, which includes more than nine-in-ten
countries covered in the coding. An analysis by Grim and Richard Wike,
associate director of the Pew Research Center’s Global Attitudes Project,
tested the reliability of the State Department reports on social impediments to
religious practice by comparing public opinion data with data coded from the
reports in previous years by Grim and experts at Penn State. They concluded
that “the understanding of social religious intolerance embodied in the State
Department reports is comparable with the results of population surveys and
individual expert opinion.”16
Footnotes:
8 See the methodology of the Pew
Forum’s 2009 report, “Global Restrictions on Religion,” for a discussion of the
conceptual basis for measuring restrictions on religion. (return to text)
9 Previous reports provided a score for
the territory of Northern Cyprus and therefore included 198 countries and
territories. According to the U.S. State Department, only one country – Turkey
– recognizes the separate status of Northern Cyprus. Thus, future reports will
score Northern Cyprus as part of the Republic of Cyprus. The exclusion of
Northern Cyprus in this report has a negligible effect on the global and
regional findings. (return to text)
10 The 2011 report referred to a change
in a country’s score as “substantial” only if it was at least 1.5 standard
deviations above or below the mean amount of change among all countries and
territories. The change also had to be in the same direction, meaning that it
had to rise or fall both in the period from mid-2006 to mid-2008 and in the
overlapping period from mid-2007 to mid-2009. See the methodology in the Pew
Forum’s August 2001 report “Rising Restrictions on Religion” for more details. (return to text)
11 See Grim, Brian J. and Roger Finke.
2006. “International Religion Indexes: Government Regulation, Government
Favoritism, and Social Regulation of Religion.” Interdisciplinary Journal of
Research on Religion, vol. 2, article 1. (return to text)
12 The one member state of the United
Nations not included in the study is North Korea. The sources clearly indicate
that North Korea’s government is among the most repressive in the world with
respect to religion as well as other civil and political liberties. (The U.S.
State Department’s 2010 Report on International Religious Freedom, for example,
says that “Genuine freedom of religion does not exist” in North Korea.) But
because North Korean society is effectively closed to outsiders and independent
observers lack regular access to the country, the sources were unable to
provide the kind of specific, timely information that the Pew Forum categorized
and counted (“coded,” in social science parlance) for this quantitative study.
Therefore, the report does not include scores for North Korea. (return to text)
13 Ordering is based on second decimal
place when scores are tied. (return to text)
14 Ordering is based on second decimal
place when scores are tied. (return to text)
15 For a more advanced statistical
analysis of the association between government restrictions and social
hostilities, see Grim, Brian J. and Roger Finke. 2011. “The Price of Freedom
Denied: Religious Persecution and Conflict in the 21st Century.” New York:
Cambridge University Press, and their related 2007 article, “Religious
Persecution in Cross-National Context: Clashing Civilizations or Regulated
Economies?” American Sociological Review, vol. 72, no. 3: 633-658. (return to text)
16 See Grim, Brian J. and Richard Wike.
2010. “Cross-Validating Measures of Global Religious Intolerance: Comparing
Coded State Department Reports with Survey Data and Expert Opinion.” Politics
and Religion, vol. 3, issue 1: 102-129. (return to text)