Surveying Participants to Strengthen Behavioral Health-Criminal Justice Programs
Surveying Participants to Strengthen Behavioral Health-Criminal Justice Programs
Participant satisfaction surveys help behavioral health-criminal justice programs assess the quality of service being provided and their impact on individual outcomes, as well as help them to determine if there are disparities in program and service delivery. This brief presents important considerations when developing these surveys for people with behavioral health needs in the criminal justice system and ways to integrate survey feedback meaningfully.
Gathering participant feedback to improve service delivery is standard practice for many customer-serving businesses, including behavioral health treatment programs and doctor’s offices. But behavioral health-criminal justice programs have historically refrained from soliciting input from participants because their involvement is often either mandatory in correctional facilities or a required condition of probation or parole. This thinking, however, is changing as these programs better understand the importance of using participant feedback as a data source to improve the quality of their services, determine what they are doing well, and see where there might be room for improvements.
One of the best ways to capture participant feedback is through satisfaction surveys. These surveys help to assess the quality of services and their impact on individual outcomes, as well as help to determine if there are disparities in program and service delivery. This information can be used to inform program and policy changes that ultimately improve client experiences and outcomes. This brief presents important considerations when developing participant satisfaction surveys for people with behavioral health needs in the criminal justice system and ways to integrate survey feedback meaningfully.
Key Components of Participant Satisfaction
Seeking input from the people who engage in behavioral health programming, whether they are voluntary or mandated, illustrates a commitment to understanding the impact that services have on participants’ conditions, circumstances, or behavior.1 In fact, when results from satisfaction surveys are used to inform programming, they can lead to improved engagement outcomes and greater participation in treatment.2
Before crafting participant surveys, programs need to understand the different parts of participation satisfaction. These components should be incorporated within survey questions to ensure that program administrators receive information that will best help them be responsive to participant needs. Figure 1 outlines important components to consider when developing participant satisfaction surveys and provides a sample assessment statement for each.
Fig. 1: Four Key Components of Participant Satisfaction
|Key Component||Description||Example Assessment Statements|
|Service climate||Examines whether the program’s physical setting and environment contribute to fair and equitable practices where participants feel that they can safely receive the support needed to work on their recovery||Staff respected my wishes about who is and who is not to receive information about my treatment.3|
|Program delivery||Assesses service delivery methods and program intervention strategies, including whether there are clear goals, activities that support the goals, equitable access to services and treatment, and engaging activities (e.g., group sessions, interactive exercises)||Staff helped me to develop and plan for life goals beyond managing symptoms or staying stable (e.g., employment, education, connecting with family and friends).4|
|Performance||Evaluates how well the program is executed and the quality of the services provided; also includes participants’ views on staff investment in delivering services and whether staff are trustworthy and interested in their well-being||I received high-quality medical care from this program.5|
|Participant outcomes and experience||Assesses whether the program meets participants’ expectations and whether they feel it has helped them achieve positive outcomes||As a direct result of the mental health and/or substance use disorder services I received in the last six months, I am better able to deal with crisis.6|
Development of Participant Surveys
When developing satisfaction surveys, it is important to think through what information is most valuable to obtain from participants and which methods will glean honest, constructive responses. However, this must also be balanced with a need to keep surveys concise and easy to use. Participants are more likely to complete short, direct surveys. Important questions to consider when developing surveys include the following:
What aspects of our behavioral health-criminal justice program are we assessing? Participant satisfaction surveys can cover many parts of a program experience, from referrals to quality of service to participant outcomes. However, individual programs may not oversee all aspects of service delivery. Consider and base questions only around the participant’s experience as a direct result of the services provided by the program.
What survey format/structure is easiest for participants to complete? Consider which format(s) works best for your participants (i.e., paper, online, telephone) and the implications of each. For example, if the survey is administered over the phone, staff conducting the survey will need to determine how responses are recorded, how to ensure that the participant understands the questions, and how the results will be shared. It’s also important to consider the number of questions participants will be able or willing to answer, as well as how these questions are structured, and what plans are in place if participants cannot answer the questions on their own. Yes/no answers to satisfaction surveys may not yield the most helpful information. So, think about using short answers and ranking questions for participant responses.
How can we assure participants that they can respond honestly without fear of ramifications? Understand that some may raise confidentiality concerns or worry that their candid responses will negatively impact their care. Consider how you will inform participants of the purpose of the survey, who will have access to their responses, and what rights they have to opt-out of the survey at any time.
Who should administer the surveys and when? Consider who is best positioned to administer surveys to participants. This is typically based on who has developed a rapport with participants to obtain honest, constructive responses and who has the capacity to administer them regularly. Common administrators are research partners, peer specialists, case managers, and probation officers. Relatedly, you will also need to determine when surveys will be administered and how often (e.g., every 6 months, 30 days after release from a correctional facility, at program completion). The timing of the survey is often connected to its purpose. For example, if a survey focuses solely on participants’ satisfaction with the service climate and their expectations for results, the survey will likely be administered early on.
Ways to Ensure Feedback Informs Programming
Well-crafted surveys will yield crucial information to help behavioral health-criminal justice programs identify service gaps and develop an effective action plan for quality improvement. However, to efficiently use this information, programs must have the following in place:
- Written and clear protocols for administering the survey on an ongoing basis;
- Information-sharing agreements that outline where the results are stored, who can access survey responses, and for what purposes;
- An established process for reporting survey results to key stakeholders (such as leadership and relevant frontline staff); and
- Engaged leaders and staff members who ensure that results from the survey are integrated into overall evaluation and quality assurance practices of the behavioral health programming. This might entail regular meetings to discuss survey results and specific, identified time periods to make necessary program changes to meet participant needs.
The Council of State Governments Justice Center offers free in-depth subject matter expertise and can connect you to programs that are currently implementing participant satisfaction surveys. Visit the Center for Justice and Mental Health Partnerships to learn more.
Recovery Self-Assessment RSA-R (Person in Recovery Version)
2For example, studies show that satisfied participants are more likely to engage with services and adhere to therapy, while dissatisfied participants are at a higher risk of discontinuing treatment and facing poor treatment trajectories. See Elisabetta Miglietta et al., “Scales for Assessing Patient Satisfaction with Mental Health Care: A Systematic Review,” Journal of Psychiatric Research 100 (2018): 33–46; Sarah Woodward et al., “A Systematic Review of Factors Associated with Service User Satisfaction with Psychiatric Inpatient Services,” Journal of Psychiatric Research 92 (2017): 81–93; Anna Brown et al., “Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement,” Administration and Policy in Mental Health and Mental Health Services Research 41 (2014): 434–446.
3This example statement is derived from the validated instrument Mental Health Statistics Improvement Program (MHSIP) Adult Satisfaction Survey.
4This example statement is derived from the validated instrument Recovery Self-Assessment RSA-R (Person in Recovery Version).
5This example statement is derived from the validated instrument Revised Health Care System Distrust Scale.
6This example statement is derived from the validated instrument Mental Health Statistics Improvement Program (MHSIP) Adult Satisfaction Survey.
This project was supported by Grant No. 2019-MO-BX-K001 awarded by the Bureau of Justice Assistance. The Bureau of Justice Assistance is a component of the Department of Justice’s Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, the Office for Victims of Crime, and the SMART Office. Points of view or opinions in this document are those of the author and do not necessarily represent the official position or policies of the U.S. Department of Justice.
Writing: Rachel Lee, CSG Justice Center, and Dr. Faye Taxman, George Mason University
Research: Rachel Lee, CSG Justice Center, and Dr. Faye Taxman, George Mason University
Advising: Sarah Wurzburg and Demetrius Thomas, CSG Justice Center
Editing: Darby Baham and Emily Morgan, CSG Justice Center
Design: Michael Bierman
Public Affairs: Ruvi Lopez, CSG Justice Center
ABOUT THE AUTHOR
Fifteen years after its inception, the Justice and Mental Health Collaboration Program (JMHCP) continues to support initiatives across…Read More
The Council of State Governments (CSG) Justice Center will host a four-session…Read More
This week, U.S. Representatives Bobby Scott (D-VA), Steve Chabot (R-OH), Sheila Jackson…Read More