Getting to Your Data: Lessons Learned and Guiding Principles from Pennsylvania

The Pennsylvania Stepping Up Technical Assistance Center, established by the Pennsylvania Commission on Crime and Delinquency (PCCD) and administered by The Council of State Governments (CSG) Justice Center, provides free technical assistance (TA) to Stepping Up counties in PA working to reduce the overincarceration of people with serious mental illnesses and co-occurring substance use disorders (SMIs/SUDs) in their county jails. As part of this technical assistance, in 2022 and 2023, the TA center provided intensive data-focused assistance to four counties: Bucks, Dauphin, Lehigh, and Washington counties. Each county received a final report with recommendations to strengthen their data collection processes and reporting procedures for tracking Stepping Up’s four key measures and building a sustainable infrastructure to track their data without outside resources.

Over the course of these projects, CSG Justice Center staff recognized that the counties were encountering many of the same challenges and barriers in working with their Stepping Up data, including leveraging data to assess the behavioral health needs of their jail populations and inform the identification of policies, practices, or programs to create systems-level change. Lessons learned and successful strategies for navigating these challenges are compiled in this resource to assist current and future Pennsylvania Stepping Up counties in their data efforts. This brief guides counties through practical tips for how to access, analyze, and leverage their data, whether they are just getting to their data for the first time or planning to set their baselines, measure their progress over time, and achieve results—the three core components of Stepping Up’s Set, Measure, Achieve (SMA) framework.

How to Get Started: Preparing for Data Tracking/Reviewing Data Systems

When embarking on the Stepping Up “data journey,” it can be difficult to know where to begin, especially in the absence of assigned staff with data expertise. In the early stages of data tracking, or when revisiting specific data components, certain strategies can be implemented to make the collection, interpretation, and utilization of SMA metrics manageable and sustainable for Stepping Up counties.

Create an Interagency Collaborative:

• Form a collaborative of key local Stepping Up partners to coordinate efforts and open a dialogue around working with data.
• Invite them to share what, where, and how they track their relevant information, including logistical processes such as staffing, and discuss what information is needed to inform decision-making and case planning.
• Early in the formation of the collaborative, establish clear roles and responsibilities for project management, including data entry, report generation, analyzing findings, reporting out to stakeholders, and sharing aggregated data with Stepping Up. The group should also establish mechanisms for ongoing communication and collaboration.

Develop Data Tracking Tools and Protocols:

• Conduct a data workflow exercise to determine what data exists across various systems. Create associated documentation, such as data maps or a data dictionary.
• Check for duplicative data collection within existing data systems and streamline where possible.
• If needed data does not exist within current systems, create a plan for establishing new mechanisms for collecting this data.
• Develop written protocols documenting data collection and extraction plans.

Accessing the Data

Needed information is often collected across multiple data systems and/or maintained on spreadsheets and can be difficult to access and combine for analytical and reporting purposes. However, several strategies can be utilized to simplify the data sharing and integration process for ease of data collection and interpretation.

Optimize Your Jail Management System (JMS) and Other Systems:

• Determine which system provides the most data for the project and build from that system. It is generally recommended to designate the JMS as it will allow for more robust analysis, including tracking bookings, releases, average length of stay (ALOS), and underlying charges.
• Partnering with your county IT department or jail-based IT staff and designated staff responsible for managing data collection and reporting is key to accessing information and building reports. The jail may have an active contractual relationship with their JMS vendor who can also assist in this process, although this generally involves a fee.

Leverage Other Data Tracking and Reporting Platforms as Needed:

• In the absence of a robust JMS, some counties may rely heavily on Excel spreadsheets or other data tracking mechanisms for their Stepping Up data. While sometimes necessary, data tracking and reporting can get very complicated if multiple platforms are utilized. It is recommended to minimize the volume and complexity of data tracking sheets external to the JMS.
• When utilizing tracking sheets external to the JMS, it is important to ensure case-level unique identifiers are utilized to ensure information can be integrated and synthesized across the various data systems and tracking sheets. Where possible, use drop-down categories and numerical data for ease of interpretation and conduct routine data auditing to check for data inaccuracies.
• All data sources (including Excel sheets) should have a data dictionary and associated written protocols detailing what is contained within the data source and how that data is collected and utilized.

Establish Interagency Information-Sharing Protocols:

• Among the collaborative of Stepping Up partners, determine which organization is willing and able to provide which data points.
• Assign a point person or agency responsible for cross-agency data management and reporting.
• Draft and execute an information-sharing agreement (or multiple agreements, as necessary) that specifies what information can be shared, with whom, and for what purposes. (See examples in the Strategy Lab.)
• Establish clear processes and protocols for interagency data sharing and initiate those processes on a recurring basis.

Utilizing the Data Meaningfully

Once data is collected, the next step is determining how to best utilize it to inform policy and program decision-making. Of paramount importance is ensuring that you have maintained your collaborative of cross-system stakeholders committed to reviewing the data together on a regular basis to analyze what this data means to you and how the data can support in addressing identified needs to drive your work forward.

Generate Shared Data Reports or Dashboards on a Quarterly Basis:

• Through your collaborative, leverage the expertise of county and jail IT staff to determine the reporting needs.
• Aggregate the data into a concise but comprehensive report that compares data for people in the jail with SMI and people without SMI across your selected measures.
• Automate the data reporting as much as possible so that the reporting follows the same format, using the same definitions and measuring the same data points consistently.
• Track changes over time to demonstrate progress and identify whether different strategies are needed.

Convene Collaborative to Review and Discuss Data Reports:

• The collaborative should meet to discuss the context and meaning behind data trends. For example, consider whether the implementation of a recent policy or program may be impacting numbers in one direction or another.
• Discuss the implications of data trends and assess whether changes need to be made to existing policies or practices or if an additional program or policy should be implemented.
• Identify key questions that need to be further explored with the data to guide ongoing and future discussions.
• Include the voices of people with lived experience in the criminal justice and behavioral health systems and community groups to enrich this conversation with their perspectives on jail-based behavioral health services, reentry, and other programs or practices.

Implement Identified Policy or Practice Changes:

• Through data analysis and review and the incorporation of community voices, gaps in service and program development or improvements may emerge.
• Research best practices, seek out peer examples, and confer with the PA Stepping Up TA Center for program and policy solutions to address the identified need.
• Develop a plan for implementing selected best practices and policy solutions that includes a timeline and key roles and responsibilities of involved parties.
• Provide training for stakeholders and staff to understand and create buy-in for the recommended next steps.

Building and Sustaining Data Capacity and Infrastructure

Having the internal capacity to collect, report, and utilize data to drive decision-making at the local level is a challenge that many counties face. Funding for criminal justice coordinators and data analyst positions is limited, and those roles and responsibilities are not inherently built into the jail or county organizational structure.

Commit Resources to Expanding Data Capacity:

• Consider earmarking county dollars to hire a criminal justice coordinator to manage cross-system planning and a data analyst to support this work. Investing in such positions better ensures the long-term sustainability of accessing, analyzing, and applying data to decision-making.
• Alternatively, subcontracting with a local university partner or evaluation firm can also help you reach your goals.
• If local funds are limited, pursue funding through state or federal funders or private foundations to support hiring or contracting externally.

Engage with the TA Center and Other Data and Evaluation Learning Opportunities:

• Ask for 1:1 TA support for data management from the CSG Justice Center.
• Participate in Community of Practice opportunities to leverage TA from experts in the field and in other data and evaluation learning opportunities such as those from the American Evaluation Association.
• Form connections with peers who have experienced similar challenges, have led best practice implementation, and can provide connections across stakeholders to increase support for system change.