Appendix C: Survey Methodology

Results for the survey are based on face-to-face interviews conducted under the direction of Opinion Research Business in Iraq, Morocco and Tunisia and Princeton Survey Research Associates International in the other 36 countries. Findings are reported exclusively for Muslims; however, the survey is based on national samples that did not screen out non-Muslims, except in Thailand, where a sample of only Muslims was fielded in five southern provinces. In certain instances, regions of countries with high levels of insecurity or limited access were also excluded from the national samples. Oversamples of Muslims were conducted in two countries: Bosnia-Herzegovina and Russia. In both countries, oversampling was achieved by disproportionately sampling regions or territories known to have higher concentrations of Muslims.

In all countries, surveys were administered through face-to-face interviews conducted at a respondent’s place of residence. All samples are based on area probability designs, which typically entailed proportional stratification by region and urbanity, selection of primary sampling units (PSUs) proportional to population size, and random selection of secondary and tertiary sampling units within PSUs. Interview teams were assigned to designated random routes at the block or street level and followed predetermined skip patterns when contacting households. Within households, adult respondents were randomly selected by enumerating all adults in the household using a Kish grid or selecting the adult with the most recent birthday.

The questionnaire administered by survey interviewers was designed by the Pew Forum staff in consultation with subject matter experts and advisers to the project. The questionnaire was translated into the vernacular language(s) of each survey country, checked through back-translation and pretested prior to fieldwork. In total, the survey was conducted in more than 80 languages.

Conducting opinion polls in diverse societies necessitates adapting the survey to local sensitivities. In some countries, pretest results indicated the need to suppress certain questions to avoid offending respondents and/or risking the security of the interviewers. In other countries, interviewers considered some questions too sensitive to pretest. Thus, not all questions were asked in all countries.

For example, in 2008-2009, Muslims and Christians in sub-Saharan Africa were asked whether the sacred texts of their respective religions are the literal word of God. In 2011-2012, pretest results indicated that respondents found this item offensive because it “questioned the unquestionable.” Therefore, it was removed from the questionnaire. 

In another example, interviewers in Afghanistan said they were uncomfortable asking respondents whether they believe there is only one God and that Muhammad is God’s Prophet, as respondents could interpret the question to be blasphemous. Therefore, the question was omitted in Afghanistan.

Following fieldwork, survey performance for each country was assessed by comparing the results for key demographic variables with reliable, national-level population statistics. For each country, the data were weighted to account for different probabilities of selection among respondents in each sample. Additionally, where appropriate, data were weighted through an iterative procedure to more closely align the samples with official population figures for characteristics such as gender, age, education and ethnicity. The reported sampling errors and the statistical tests of significance used in analysis take into account the effect of both types of weighting. The reported sampling errors and statistical tests of significance also take into account the design effects associated with each sample.

The table below shows the sample size and margin of sampling error for Muslim respondents in each country. For results based on the Muslim sample in the countries surveyed, one can say with 95% confidence that the error attributable to collecting data from some, rather than all, members of the Muslim population is plus or minus the margin of error. This means that in 95 out of 100 samples of the same size and type, the results we obtain would vary by no more than plus or minus the margin of error for the country in question.

It should be noted that practical difficulties in conducting multinational surveys can introduce error or bias into the findings of opinion polls. In some countries, the achieved samples suffered from imbalances in the number of women or men interviewed, while in some countries a lack of adequate, national-level statistics made it difficult to assess the accuracy of educational characteristics among the sampled population. Specific difficulties encountered were:

Gender Imbalances: In Afghanistan and Niger, the survey respondents are disproportionately male, while in Thailand, Azerbaijan and Uzbekistan they are disproportionately female.

In each of these countries interviewers faced practical difficulties in reaching additional male or female respondents. In Afghanistan, despite strict gender matching, cultural norms frequently limited the ability of interviewers to contact women in certain areas. In Niger, difficulties associated with recruiting enough female interviewers affected gender matching and may have discouraged the participation of women in the survey.

Surveying in active conflict zones posed particular challenges for interviewers. In southern Thailand, security concerns limited the number of interviews that could be conducted in the evening hours, leading in part to fewer interviews with men, who often are out of the house during daytime hours.

In Azerbaijan and Uzbekistan, large-scale labor migration patterns may have contributed to fewer interviews with male respondents.

Education: In many countries, census statistics on education are unavailable, dated or disputed by experts. The lack of reliable national statistics limits the extent to which survey samples can be assessed for representativeness on this measure. 

In Albania, the Palestinian territories and Tajikistan, the surveys appear to overrepresent highly educated respondents compared with the last available national census. In each of these cases, however, official education statistics are based on, or estimated from, censuses conducted five or more years prior to the survey and thus were not used for the purposes of weighting.

In Niger, the sample is disproportionately well-educated compared with the last available Demographic and Health Survey (2006), but no education census statistics are available to assess the representativeness of the sample.

In addition to sampling error and other practical difficulties, one should bear in mind that question wording can also have an impact on the findings of opinion polls.

For details about the surveys conducted in 15 sub-Saharan African countries in 2008-2009, see the Pew Forum’s 2010 report “Tolerance and Tension: Islam and Christianity in Sub-Saharan Africa.”

The survey questionnaire and a topline with full results for the 24 countries surveyed in 2011-2012 is included in Appendix D.

Afghanistan
Sample design: Stratified area probability sample of all 34 Afghan provinces (excluding nomadic populations) proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Baloch, Dari, Hazara, Pashto, Uzbek
Fieldwork dates: Nov. 27–Dec. 17, 2011
Representative: Nationally representative of 94% of the adult population.
Design effect: 3.4

Albania
Sample design: Stratified area probability sample of all three regions proportional to population size and urban/rural population. Some difficult-to-reach areas were excluded.
Mode: Face-to-face adults 18+
Languages: Albanian
Fieldwork dates: Oct. 24–Nov. 13, 2011
Representative: Nationally representative of 98% of the adult population.
Design effect: 2.3

Azerbaijan
Sample design: Stratified area probability sample of eight of 11 oblasts (excluding Upper-Karabakh, Nakhchivan and Kalbacar-Lacin) and city of Baku proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Azeri, Russian
Fieldwork dates: Dec. 4–Dec. 25, 2011
Representative: Nationally representative of 85% of the adult population.
Design effect: 3.3

Bangladesh
Sample design: Stratified area probability sample of all seven administrative divisions proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Bangla
Fieldwork dates: Nov. 21, 2011–Feb. 5, 2012
Representative: Nationally representative of the adult population.
Design effect: 3.8

Bosnia-Herzegovina
Sample design: Stratified area probability sample of all seven regions proportional to population size and urban/rural population. In addition, an oversample of Muslims was conducted in majority-Bosniak areas. Some difficult-to-reach areas were excluded.
Mode: Face-to-face adults 18+
Languages: Bosnian, Croatian, Serbian
Fieldwork dates: Nov. 3–Nov. 20, 2011
Representative: Nationally representative of 98% of the adult population.
Design effect: 1.8

Egypt
Sample design: Stratified area probability sample of 24 of 29 governorates proportional to population size and urban/rural population. The five frontier provinces, containing 2% of the overall population, were excluded.
Mode: Face-to-face adults 18+
Languages: Arabic
Fieldwork dates: Nov. 14–Dec. 18, 2011
Representative: Nationally representative of 98% of the adult population.
Design effect: 2.6

Indonesia
Sample design: Stratified area probability sample of 19 provinces (excluding Papua and other remote areas and provinces with small populations) proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Bahasa Indonesian
Fieldwork dates: Oct. 28–Nov. 19, 2011
Representative: Nationally representative of 87% of the adult population.
Design effect: 2.3

Iraq
Sample design: Stratified area probability sample of all 18 governorates proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic, Kurdish
Fieldwork dates: Nov. 4–Dec. 1, 2011
Representative: Nationally representative of the adult population.
Design effect: 4.9

Jordan
Sample design: Stratified area probability sample of all 12 governorates proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic
Fieldwork dates: Nov. 3–Dec. 3, 2011
Representative: Nationally representative of the adult population.
Design effect: 3.5

Kazakhstan
Sample design: Stratified area probability sample of all 14 oblasts proportional to population size and urban/rural population. Three districts each in Almaty oblast and East Kazakhstan were excluded due to government restrictions.
Mode: Face-to-face adults 18+
Languages: Kazakh, Russian
Fieldwork dates: Nov. 24–Dec. 17, 2011
Representative: Nationally representative of 98% of the adult population.
Design effect: 2.5

Kosovo
Sample design: Stratified area probability sample of all eight KFOR-administered regions proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Albanian, Serbian
Fieldwork dates: Dec. 16, 2011–Jan. 20, 2012
Representative: Nationally representative of 99% of the adult population.
Design effect: 3.7

Kyrgyzstan
Sample design: Stratified area probability sample of all seven oblasts and the cities of Bishkek and Osh proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Kyrgyz, Russian, Uzbek
Fieldwork dates: Jan. 31–Feb. 25, 2012
Representative: Nationally representative of the adult population.
Design effect: 3.3

Lebanon
Sample design: Stratified area probability sample of all seven regions (excluding areas of Beirut controlled by a militia group and a few villages in the south near the border with Israel) proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic
Fieldwork dates: Nov. 14–Dec. 8, 2011
Representative: Nationally representative of 98% of the adult population.
Design effect: 2.2

Malaysia
Sample design: Stratified area probability sample of Peninsular Malaysia, East Malaysia and the Federal Territory of Kuala Lumpur. In Peninsular Malaysia and Kuala Lumpur, interviews were conducted proportional to population size and urban/rural population. A disproportionately higher number of interviews were conducted in Sarawak and Sabah states in East Malaysia to adequately cover this geographically challenging region.
Mode: Face-to-face adults 18+
Languages: Mandarin Chinese, English, Malay
Fieldwork dates: Nov. 4, 2011–Jan. 25, 2012
Representative: Nationally representative of the adult population.
Design effect: 2.5

Morocco
Sample design: Stratified area probability sample of 15 regions proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic, French
Fieldwork dates: Nov. 3–Dec. 1, 2011
Representative: Nationally representative of the adult population.
Design effect: 2.8

Niger
Sample design: Stratified area probability sample of seven of eight regions (Agadez was excluded) and city of Niamey proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: French, Hausa
Fieldwork dates: Dec. 5–Dec. 16, 2011
Representative: Nationally representative of 97% of the adult population.
Design effect: 3.1

Pakistan
Sample design: Stratified area probability sample of all four provinces (excluding the Federally Administered Tribal Areas, Gilgit-Baltistan, and Azad Jammu and Kashmir for reasons of security as well as areas of instability in Khyber Pakhtunkhwa and Balochistan) proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Balochi, Hindko, Pashto, Punjabi, Sindhi, Saraiki, Urdu
Fieldwork dates: Nov. 10–Nov. 30, 2011
Representative: Nationally representative of 82% of the adult population.
Design effect: 4.7

Palestinian territories
Sample design: Stratified area probability sample of all five regions (excluding Bedouins and some communities near Israeli settlements due to military restrictions) proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic
Fieldwork dates: Dec. 4, 2011–Jan. 2, 2012
Representative: Nationally representative of 95% of the adult population.
Design effect: 4.1

Russia
Sample design: Area probability sample of all 80 oblasts proportional to population. In addition, an oversample of Muslims was conducted in oblasts with a higher concentration of ethnic Muslims.
Mode: Face-to-face adults 18+
Languages: Russian
Fieldwork dates: Oct. 27–Dec. 2, 2011
Representative: Nationally representative of 99% of the adult population, with a Muslim oversample.
Design effect: 0.9

Tajikistan
Sample design: Stratified area probability sample of all four oblasts and city of Dushanbe proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Russian, Tajik
Fieldwork dates: Dec. 28, 2011–Jan. 21, 2012
Representative: Nationally representative of 99% of the adult population.
Design effect: 4.4

Thailand
Sample design: Stratified area probability sample of Muslims in the provinces of Yala, Pattani, Narathiwat, Satun and Songkhla proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Thai, Yawee
Fieldwork dates: Nov. 12, 2011–Jan. 8, 2012
Representative: Representative of adult Muslims in five southern provinces.
Design effect: 3.3

Tunisia
Sample design: Stratified area probability sample of all 24 governorates proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Arabic, French
Fieldwork dates: Nov. 10–Dec. 7, 2011
Representative: Nationally representative of the adult population.
Design effect: 1.6

Turkey
Sample design: Stratified area probability sample of all 26 regions proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Turkish
Fieldwork dates: Nov. 18–Dec. 19, 2011
Representative: Nationally representative of the adult population.
Design effect: 5.2

Uzbekistan
Sample design: Stratified area probability sample of all 14 oblasts and city of Tashkent proportional to population size and urban/rural population.
Mode: Face-to-face adults 18+
Languages: Russian, Uzbek
Fieldwork dates: Feb. 2–Feb. 12, 2012
Representative: Nationally representative of 99% of the adult population.
Design effect: 2.2

Photo Credit: © SZE FEI WONG / istockphoto