Explainer: Using Data to Improve the Effectiveness of Community Responder Programs

More jurisdictions across the country are beginning to develop community responder programs, which position health professionals and trained crisis staff as first responders to behavioral health crises and social disturbances. There has been little guidance, however, on effective ways to measure and improve the performance of these programs. This article discusses three critical parts of this process: (1) Assessing the needs of the community; (2) Measuring program performance; and (3) Evaluating programs to determine their success.

Key Elements of Needs Assessments

Before implementing a community responder program, program developers should conduct an initial needs assessment to ensure it will effectively meet local needs. Although this is one of the first steps, conducting needs assessments should also continue throughout any pilot phases and even after full implementation to help guide future program continuation and expansion.

Police Department and Fire/EMS Data

Program developers must know how their community uses emergency services. Key information includes the number and types of calls to emergency departments, the number of interactions police have with people experiencing crises, local crime statistics, and surveys of police officers and first responders to identify prominent concerns.

Community Outreach and Engagement

It is crucial to actively engage residents when designing a community responder program to ensure it is not viewed as inaccessible and unhelpful. Community engagement can take many forms, including the following:

• Surveys to identify local demographics, health statistics, and social service needs.
• Interviews with local leaders, including people who have lived experiences with mental illness, substance use, or the criminal justice system, to assess how community services operate and determine potential gaps
• Town halls, roundtables, and focus groups for community members to share feedback and concerns.
• Meetings with existing community coalitions to learn about current community response efforts.

Asset Mapping

Community responder programs must be designed with the existing community structures in mind. One way to do this is through asset mapping—conducting a comprehensive assessment of current community services and resources. Identifying and developing relationships with other community providers is also a key step that allows for a more comprehensive care network, guides which services the community responder program offers, and allows responders to provide referrals more effectively. 

The Most Common Performance Metrics for Community Responder Programs

Once a community responder program is operational, program staff must routinely collect data during both the pilot phase and after full implementation to evaluate program performance. Although metrics will vary, there are several categories that are routinely measured.

• Call and response data: Determine whether services successfully meet the needs of the community, the ability of community responders to respond to calls they receive, and how frequently individuals are using program services. Metrics commonly collected include the following:
  • Number of incoming calls appropriate for community responder team dispatch
  • Total number of responses initiated
  • Type and number of incidents responded to
  • Response time
  • Number of unique individuals served
  • Number of repeat encounters
  • Percentage of calls diverted away from emergency services (EMS, 911)
  • Time on scene for community responders

• Outcome statistics: Measure whether the program is meeting its stated goals. By analyzing whether individuals are accepting treatment or are diverted away from emergency services, program staff can determine which elements of the program are working. Key measurements include the following:
  • Number of people engaged
  • Type and number of services provided
  • Number of people who declined services or referrals
  • Rate of referral to other services
  • Number of referrals completed
  • Number of individuals transported to another location
  • Number of needs identified and met at the scene
  • Number of incidents that require police backup
  • Number of individuals who require additional emergency services
  • Percentage of encounters resulting in diversion away from hospitals
  • Percentage of encounters resulting in diversion from jails
  • Post-contact outcomes for clients
  • Client demographics
  • Cost-benefit information

One way that community responder programs are publicly making performance measurement data available is through data dashboards that provide information on calls for services, outcomes, and other key metrics identified through community engagement processes.

The Importance of Formal Evaluations

To measure program success, it is critical to conduct evaluations at the conclusion of a program’s pilot phase and at routine intervals afterwards. This process frequently involves partnering with third-party organizations, such as local universities or community-based research firms, that can provide unbiased data analyses. Recommendations provided by these third-party evaluators can assist programs in refining their target population and tailoring services to better meet the needs of the community. Evaluations can also be used to demonstrate the success of a program to community members and other stakeholders—an asset for acquiring additional funding.

Evaluators frequently conduct community surveys, interviews, and focus groups to determine whether the public views the program as accessible, whether program services are responding to local needs, and to gather feedback on ways to improve. The most fundamental part of most evaluations includes analyzing existing data gathered by the community responder teams throughout their daily routines and comparing them to program goals. This type of analysis plays a critical role in evaluations at the end of a program’s pilot phase. If the data show the pilot was well received by the community and successfully diverted calls away from emergency response units, the evaluation may provide support for the continuation or expansion of the program.

For more information on community responder programs, visit Expanding First Response, a toolkit that serves as a central hub for states and local communities looking to establish or strengthen community responder programs.