Religion in Prisons - A 50-State Survey of Prison Chaplains
Appendix A: Survey Methodology
Navigate this page:
This survey of prison chaplains was conducted from Sept. 21 through Dec. 23, 2011, among professional prison chaplains or religious services coordinators (the two titles are used interchangeably for the purposes of this report) working in prisons in all 50 states.20 Correctional authorities in each of the 50 states granted permission for the Pew Research Center’s Forum on Religion & Public Life to contact state prison chaplains and request their voluntary participation in the survey. The survey was also endorsed by the American Correctional Chaplains Association. A total of 730 interviews were completed by Web and paper questionnaire, a response rate of nearly 50%.
The target population for the survey was all paid prison chaplains in the 50 state prison systems across the United States. The Pew Forum contacted officials in each state’s department of corrections to request permission to conduct the survey and to obtain contact information for all professional chaplains currently employed in state prisons.21 Based on communications with each state’s board of corrections, the Pew Forum compiled a database of chaplains thought to be eligible for the study. The process of obtaining permission to conduct the study and assembling the list of eligible chaplains was facilitated by the American Correctional Chaplains Association and by some chaplains with access to the National Correctional Chaplains Administrators Directory. A total of 1,474 state prison chaplains and religious services coordinators from the 50 state correctional systems comprised the total target population of interest.
The administration and data coding of the survey was handled by Social Science Research Solutions (SSRS). Attempts were made to contact all 1,474 state prison chaplains to request participation in the survey. In keeping with best practices for survey research, the target sample was contacted multiple times to request participation. The schedule of contacts was as follows:
- Invitation letter explaining the purpose of the study was sent by mail to the entire sample, with the exception of chaplains from New York,22 on Sept. 21, 2011.
- Email invitation with a direct link to the Web survey was sent to those with email address information23 about eight days after the initial mailing.
- Postcard reminder was mailed to all who had not yet completed the survey about one week after initial mailing.
- A cover letter and paper copy of the questionnaire was sent by mail to those who had not yet completed the survey about two weeks after the initial mailing.
- A second postcard reminder was mailed to those who had not yet completed the survey about three weeks after the initial mailing.
- A second email reminder was sent to those with email address information who had not yet completed the survey about five days after the second postcard reminder.
- A cover letter and second paper copy of the questionnaire was sent by mail to those who had not yet completed the survey on Oct. 24, 2011, about one month after the initial mailing.
- A final email reminder was sent to those with an email address who had not yet completed the survey on Nov. 11, 2011.
A number of chaplains had queries or comments about the survey, including technical questions about accessing the Web survey. There also were questions about the survey sponsor and the confidentiality of responses, as well as other comments about the survey or the state prison system in which they worked. These questions and comments were addressed either by staff at SSRS or by the principal investigator on the study, Stephanie Boddie.
The questionnaire was designed by the Pew Forum with the counsel of the staff at SSRS and the panel of 17 external advisers with expertise in the criminal corrections field and a working group of nine prison chaplains. A draft survey was pretested with a group of retired prison chaplains, many of whom offered suggestions on the questionnaire design. The department of corrections in some states includes a formal process for review of all study materials, called an institutional review board; feedback from several state institutional review boards also informed the questionnaire design. A draft questionnaire and study protocol was also submitted to the institutional review board for the Federal Bureau of Prisons. While the Federal Bureau of Prisons did not, in the end, grant permission to conduct the study among federal prison chaplains, their feedback and suggestions for the study informed the questionnaire design.
Several of the questions asked chaplains to provide information about specific religious groups among the inmate population. Advisers from some correctional systems reported that up to 50 religious groups exist among the prison population. To reduce respondent burden, the Pew Forum survey asked about 12 religious groups, combining some religious traditions together under broader categories. The Pew Forum was guided in these choices by the major religious groups identified by the Federal Bureau of Prisons, by the panel of external advisers and by the working group of prison chaplains. However, it is important to note that the selection of 12 religious groups involves a trade-off between the precision of gathering distinct information about each religious tradition and the practicality of asking about a more limited number of groups.
The layout and design of the questionnaire sought to maximize comparability across the mixed modes of Web and paper questionnaires while following the best practices of visual survey design.24 The Web survey included programming for all skip patterns and follow up prompts on questions 8, 9 and 22 if the initial responses did not appear to match the question instructions.
Question order and response order was fixed for all questions in order to increase the comparability of the online and paper survey modes. This departs from typical practice in Web surveys (and telephone surveys) where the order of response options and questions (especially in serial lists of questions) is sometimes randomized. The purpose of randomizing order is to control for potential primacy and recency effects in response option order and for serial item position effects in question order. Those sorts of randomization are not practical for paper questionnaires, however. The fixed question order should be kept in mind when interpreting the survey results.
The Pew Forum had final authority and responsibility for the design of the questionnaire and retains sole responsibility for the analysis and interpretation of survey findings.
The total response rate was 49.6%.25 The table provides the full disposition of the target sample for the survey.
We attempted to ask all eligible state prison chaplains to complete this survey. Based
on a comparison of chaplains who responded promptly to the survey and those who were more difficult to interview, as well as an analysis of response by region, we are assuming that the responses constitute a probability sample of the population.
Based on that assumption, the margin of error for the full sample of 730 respondents from a population of 1,474 professional prison chaplains is plus or minus 2.6 percentage points. In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias in the findings of opinion polls.
This is a survey of individual prison chaplains, as opposed to correctional facilities.26 Some larger facilities have more than one prison chaplain represented in the survey. Questions about the characteristics of correctional facilities are intended to measure the work context of the chaplains responding to this survey. The survey findings reported here are not weighted.
20 In South Dakota, religious programs are administered by a “cultural activities coordinator.” (return to text)
21 The Pew Forum also sought permission to include federal prison chaplains in the survey, but the Federal Bureau of Prisons decided not to allow its approximately 250 chaplains to participate. (return to text)
22 Permission to contact New York state chaplains was not received until mid-November. As a result, the 162 chaplains from New York state were contacted with an initial letter of invitation on Nov. 18, 2011, and a postcard reminder sent three days after the initial mailing. A cover letter and paper copy of the questionnaire were sent on Dec. 1, 2011. No email addresses were available for New York state chaplains. Fewer attempts were made to reach chaplains from New York state compared with other states due to the late permission to conduct the study among this group. (return to text)
23 An email address was available for 857 of the 1,474 chaplains eligible to participate in the survey. (return to text)
24 See Don A. Dillman, Jolene D. Smyth and Leah Melani Christian, “Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method,” Wiley, 2009. (return to text)
25 Response rate based on AAPOR’s RR4 formula. See Standard definitions: Final dispositions of case codes and outcome rates for surveys, revised 2011. (return to text)
26 Other kinds of surveys can be designed to represent organizations or institutions such as correctional facilities in proportion to their size in the population. Such surveys provide estimates about characteristics of the organizations, themselves, such as the prevalence of facilities at each security level or the number of facilities serving particular kinds of inmates. (return to text);
Photo Credit: © Illustration Works/Corbis