Appendix A: Survey Methodology
Navigate this page:
This
survey of prison chaplains was conducted from Sept. 21 through Dec. 23, 2011,
among professional prison chaplains or religious services coordinators (the two
titles are used interchangeably for the purposes of this report) working in
prisons in all 50 states.20
Correctional authorities in each of the 50 states granted permission for the
Pew Research Center’s Forum on Religion & Public Life to contact state
prison chaplains and request their voluntary participation in the survey. The
survey was also endorsed by the
American Correctional Chaplains Association. A total of 730 interviews were
completed by Web and paper questionnaire, a response rate of nearly 50%.
Sampling and
Interviewing
The
target population for the survey was all paid prison chaplains in the 50 state
prison systems across the United States. The Pew Forum contacted officials in
each state’s department of corrections to request permission to conduct the
survey and to obtain contact information for all professional chaplains
currently employed in state prisons.21
Based on communications with each state’s board of corrections, the Pew Forum
compiled a database of chaplains thought to be eligible for the study. The
process of obtaining permission to conduct the study and assembling the list of
eligible chaplains was facilitated by the American Correctional Chaplains
Association and by some chaplains with access to the National Correctional
Chaplains Administrators Directory. A total of 1,474 state prison chaplains and
religious services coordinators from the 50 state correctional systems
comprised the total target population of interest.
Survey Administration
The
administration and data coding of the survey was handled by Social Science
Research Solutions (SSRS). Attempts were made to contact all 1,474 state prison
chaplains to request participation in the survey. In keeping with best
practices for survey research, the target sample was contacted multiple times
to request participation. The schedule of contacts was as follows:
- Invitation letter explaining the
purpose of the study was sent by mail to the entire sample, with the exception
of chaplains from New York,22
on Sept. 21, 2011.
- Email invitation with a direct
link to the Web survey was sent to those with email address information23
about eight days after the initial mailing.
- Postcard reminder was mailed to
all who had not yet completed the survey about one week after initial mailing.
- A cover letter and paper copy of
the questionnaire was sent by mail to those who had not yet completed the
survey about two weeks after the initial mailing.
- A second postcard reminder was
mailed to those who had not yet completed the survey about three weeks after
the initial mailing.
- A second email reminder was sent
to those with email address information who had not yet completed the survey
about five days after the second postcard reminder.
- A cover letter and second paper
copy of the questionnaire was sent by mail to those who had not yet completed
the survey on Oct. 24, 2011, about one month after the initial mailing.
- A final email reminder was sent to
those with an email address who had not yet completed the survey on Nov. 11,
2011.
A
number of chaplains had queries or comments about the survey, including technical
questions about accessing the Web survey. There also were questions about the
survey sponsor and the confidentiality of responses, as well as other comments
about the survey or the state prison system in which they worked. These
questions and comments were addressed either by staff at SSRS or by the
principal investigator on the study, Stephanie Boddie.
Questionnaire Design
The
questionnaire was designed by the Pew Forum with the counsel of the staff at
SSRS and the panel of 17 external advisers with expertise in the criminal
corrections field and a working group of nine prison chaplains. A draft survey
was pretested with a group of retired prison chaplains, many of whom offered
suggestions on the questionnaire design. The department of corrections in some
states includes a formal process for review of all study materials, called an
institutional review board; feedback from several state institutional review
boards also informed the questionnaire design. A draft questionnaire and study
protocol was also submitted to the institutional review board for the Federal
Bureau of Prisons. While the Federal Bureau of Prisons did not, in the end,
grant permission to conduct the study among federal prison chaplains, their
feedback and suggestions for the study informed the questionnaire design.
Several
of the questions asked chaplains to provide information about specific
religious groups among the inmate population. Advisers from some correctional
systems reported that up to 50 religious groups exist among the prison
population. To reduce respondent burden, the Pew Forum survey asked about 12
religious groups, combining some religious traditions together under broader
categories. The Pew Forum was guided in these choices by the major religious
groups identified by the Federal Bureau of Prisons, by the panel of external
advisers and by the working group of prison chaplains. However, it is important
to note that the selection of 12 religious groups involves a trade-off between
the precision of gathering distinct information about each religious tradition
and the practicality of asking about a more limited number of groups.
The
layout and design of the questionnaire sought to maximize comparability across
the mixed modes of Web and paper questionnaires while following the best
practices of visual survey design.24
The Web survey included programming for all skip patterns and follow up prompts
on questions 8, 9 and 22 if the initial responses did not appear to match the
question instructions.
Question
order and response order was fixed for all questions in order to increase the
comparability of the online and paper survey modes. This departs from typical
practice in Web surveys (and telephone surveys) where the order of response
options and questions (especially in serial lists of questions) is sometimes
randomized. The purpose of randomizing order is to control for potential
primacy and recency effects in response option order and for serial item
position effects in question order. Those sorts of randomization are not
practical for paper questionnaires, however. The fixed question order should be
kept in mind when interpreting the survey results.
The Pew Forum had final authority and
responsibility for the design of the questionnaire and retains sole
responsibility for the analysis and interpretation of survey findings.
Response Rate and
Margin of Error
The
total response rate was 49.6%.25
The table provides the full disposition of the target sample for the survey.
We
attempted to ask all eligible state prison chaplains to complete this survey. Based
on a comparison of chaplains who responded promptly to the survey and those who
were more difficult to interview, as well as an analysis of response by region,
we are assuming that the responses constitute a probability sample of the
population.
Based
on that assumption, the margin of error for the full sample of 730 respondents
from a population of 1,474 professional prison chaplains is plus or minus 2.6
percentage points. In addition to sampling error, one should bear in mind that
question wording and practical difficulties in conducting surveys can introduce
error or bias in the findings of opinion polls.
This
is a survey of individual prison chaplains, as opposed to correctional
facilities.26
Some larger facilities have more than one prison chaplain represented in the
survey. Questions about the characteristics of correctional facilities are
intended to measure the work context of the chaplains responding to this
survey. The survey findings reported here are not weighted.
Footnotes:
20 In South Dakota, religious programs are administered by a “cultural activities
coordinator.” (return to text)
21 The Pew Forum also sought permission to include federal prison chaplains in the
survey, but the Federal Bureau of Prisons decided not to allow its
approximately 250 chaplains to participate. (return to text)
22 Permission to contact New York state chaplains was not received until
mid-November. As a result, the 162 chaplains from New York state were contacted
with an initial letter of invitation on Nov. 18, 2011, and a postcard reminder
sent three days after the initial mailing. A cover letter and paper copy of the
questionnaire were sent on Dec. 1, 2011. No email addresses were available for
New York state chaplains. Fewer attempts were made to reach chaplains from New
York state compared with other states due to the late permission to conduct the
study among this group. (return to text)
23 An email address was available for 857 of the 1,474 chaplains eligible to
participate in the survey. (return to text)
24 See Don A. Dillman, Jolene D. Smyth and Leah Melani Christian, “Internet, Mail,
and Mixed-Mode Surveys: The Tailored Design Method,” Wiley, 2009. (return to text)
25 Response rate based on AAPOR’s RR4 formula. See Standard definitions: Final
dispositions of case codes and outcome rates for surveys, revised 2011. (return to text)
26 Other kinds of surveys can be designed to represent organizations or institutions
such as correctional facilities in proportion to their size in the population.
Such surveys provide estimates about characteristics of the organizations,
themselves, such as the prevalence of facilities at each security level or the
number of facilities serving particular kinds of inmates. (return to text);
Photo Credit: © Illustration Works/Corbis