Integrating Criminal Justice and Behavioral Health Data

Checklist for Building and Maintaining a Data Warehouse

Read pdf Online Download pdf

Integrating Criminal Justice and Behavioral Health Data: Checklist for Building and Maintaining a Data Warehouse

Data warehouses, which are central repositories used to house information from multiple sources, can consolidate individual- and system-level data and allow for data tracking to identify relationships and trends. This information can guide decision-making for criminal justice-behavioral health partnerships—built to help minimize justice involvement among people with mental illnesses and substance use disorders—and pinpoint where to target resources and interventions. This checklist is intended to help agencies involved in the criminal justice-behavioral health partnership assess their current information technology infrastructure and guide the development of a cross-system data warehouse.

Stephanie Shaw, Robert May, and Elizabeth Fleming | July 2021 | The Council of State Governments Justice Center and the Integrated Justice Information Systems Institute

Integrating Criminal Justice and Behavioral Health Data

People in the criminal justice system frequently interact with other service systems, including behavioral health and housing. Because of this, collecting and analyzing data across multiple systems is crucial to better understand these individuals’ needs and whether interventions in place are achieving the desired outcomes. Data warehouses, which are central repositories used to house information from multiple sources, can consolidate individual- and system-level data and allow for data tracking to identify relationships and trends. In turn, this information can guide decision-making for criminal justice-behavioral health partnerships—built to help minimize justice involvement among people with mental illnesses and substance use disorders—and pinpoint where to target resources and interventions.

The checklist below will help agencies involved in the criminal justice-behavioral health partnership assess their current information technology (IT) infrastructure and guide the development of a cross-system data warehouse. It is intended to be completed in three phases as communities work to create their data warehouse: during planning, development, and implementation/maintenance. Once each phase is completed, the governance group overseeing the warehouse should assess progress and agree on next steps before moving on to the next phase. The checklist should be completed by people in charge of criminal justice–behavioral health partnership efforts, and should also include input from the IT staff of the participating agencies if they are not already in the governance group. See the appendix for a graphic that describes the relationship between the criminal justice-behavioral health partnership, the governance group, IT, and all other entities needed to build the warehouse.

Use the following key to guide your response selection for each item:

Yes

Practices or policies already exist.

Underway

There has been some movement beyond conversations, and implementation has started.

Planned

There has been focused discussion, but no actions have taken place beyond planning.

There are no policies or practices in place and none have been planned.

Phase 1 Planning for a Data Warehouse

* All statements are answered with either Yes, Underway, Planned, or No.

1. Is there an established governance group to oversee the development and execution of the data warehouse?

A. Representatives from behavioral health, criminal justice, and other relevant systems and partners (including IT) are involved.

B. A high-level leader convenes the group and provides regular updates on progress to the county board or final decision-making body.

C. A project manager is selected to coordinate the group, and roles and responsibilities of members are clearly defined.

D. At least one memorandum of understanding (MOU) is created by the project manager to document the agreed-upon goals of the project and the roles and responsibilities of the participating agencies.

2. Has the governance group identified shared goals for the data warehouse?

A. The governance group has discussed and agreed on a shared vision and mission for linking cross-agency data through the data warehouse.

B. The governance group has met with agency staff across levels who will use the data warehouse to learn what they hope to see in it, including how it will be most useful on a day-to-day basis.

C. The governance group agrees on short- and long-term goals that are aligned with the strategic objectives of the broader criminal justice-behavioral health partnership. At least some of these goals center on financial stability to ensure that the data warehouse can be sustained long term. If needed, a perfomance metric subcommittee that includes agency leadership, research directors, and IT staff is created to facilitate these efforts and identify necessary data elements for the warehouse to ensure that these goals are met.

D. The shared goals reflect and center racial equity throughout the process of data integration, including a commitment to transparency and meaningful community engagement.

E. A strategic plan is initiated—and regularly revisited—that includes questions the governance group hopes the data warehouse will help to answer, what data are needed from each agency, who will have access to the warehouse, who will manage operations of the warehouse, and how progress will be measured.

3. Has the governance group analyzed existing data collection efforts from behavioral health, criminal justice, and other relevant agencies?

A. An inventory has been conducted of each agency’s case management or data information system, as well as the data that are currently collected by each agency, the quality of these data, and which data are relevant for the partnership goals to capture.

B. The inventory is used to understand what gaps are present in the data and what additional collection efforts are needed.

C. Data definitions and data reports have been cross-walked to reduce duplicate data and ensure that the data captured matches the needs for reporting.

4. Is the required IT expertise and infrastructure in place to ensure that there is capacity to support the data warehouse?

A. An inventory of data system needs, such as storage and accessibility, has been conducted by IT staff to determine system requirements for the warehouse to achieve desired functionality.

B. Existing IT staffing capacity has been examined within the participating agencies to determine whether the project requires an outside vendor to assist in developing the data warehouse or determining software needs (see Selecting a Data Warehouse Vendor for Criminal Justice–Behavioral Health Partnerships if deciding to contract a vendor).

C. A hosting platform is in place or identified, the location where it will be hosted (e.g., in the cloud, on premises) is established, and an agency is designated to manage the platform.

D. If needed, a subcommittee that includes IT staff from participating agencies is created to manage system details (such as ensuring that software requirements are fully identified and can be supported by the host platform) and oversee any work with the developer.

E. Data display and reporting needs are identified by the group and IT staff. These can include desired types of reports and content for dashboards, which can then be customized for each agency and possibly for various roles within an agency.

F. The group has determined whether ongoing data support will be managed and enhanced by a vendor or local IT staff.

5. Are the appropriate information-sharing agreements in place?

A. An information-sharing agreement (or multiple agreements, as necessary) has been drafted by the project manager; reviewed by the governance group members, designated attorneys, and privacy officers; and signed by all participating agencies. The agreement(s) outline what information can be shared, with whom, and for what purposes.

B. The information-sharing agreement is reviewed and updated annually (or as needed for events such as new agencies joining the group) by the governance group based on policy needs, new technology, updated privacy regulations, and other identified changes. If needed, a privacy subcommittee that includes a privacy officer and agency staff is created to ensure that these efforts occur regularly and that security measures and protocols comply with federal and state privacy regulations.

6. Are the necessary financial resources available to build and maintain the data warehouse?

A. Staff responsible for budget management are identified and assigned to keep the partnership informed of all budget-related items, including the development of the project budget that includes initial software and hardware (if needed), expenditures, and ongoing maintenance and hosting costs.

B. The budget specifies how current costs will be covered and how additional funding will be secured for updates, enhancements, and future data reporting needs.

Phase 2 Developing a Data Warehouse

* All statements are answered with either Yes, Underway, Planned, or No.

1. Has the governance group created a comprehensive work plan to guide the development of the data warehouse?

A. The project manager has developed a work plan, with input from IT staff across participating agencies, that builds on the strategic plan (see Phase 1, 2d) and outlines tasks, timelines, roles and responsibilities, and budget allocations.

B. If an outside vendor is selected, all related documents (such as a description of available data, the partnership strategic plan, and all MOUs and information-sharing agreements) are reviewed and the work plan is updated accordingly.

C. A review of where existing data is housed has been conducted to determine if the data are exportable.

D. The work plan includes information generated from this review, such as data dictionaries, tables, and maps that outline the specific fields, definitions, location of the data elements by each agency’s data system, and how they are connected.

E. A process is in place for moving data to the warehouse and the frequency of doing so. This process includes how often data dashboards, reports, and other features are updated with new data and the methods to do so (i.e., the math behind the dashboards and reports).

F. An agreement is established identifying who controls the warehouse and who will need to be consulted to create communication among systems.

G. Reports are created to inform governance group meetings, and information obtained from the meetings is compared against project and data warehouse goals.

2. Have the appropriate staff addressed security and privacy issues regarding data access and usage?

A. Access to data is based on individual roles and responsibilities (e.g., a case manager from probation does not need to have access to data that includes protected health information).

B. Security measures are in place to ensure that data is protected, including strong sign-in credentials and two-factor authentication.

C. Clear processes are established for the encryption and transmission of data from agency sources, as well as destruction of data.

D. Protocols are in place for what to do in case of security breaches, such as phishing, identity theft, and improper use of confidential information.

E. Protocols are in place for staff to obtain credentials to access confidential information. These protocols include signing confidentiality statements, acknowledging the rules and guidelines for data access, penalties for violations and misuses, allowed uses of the data, and restrictions on sharing credentials with others.

F. Federal and state privacy laws, including the Health Insurance Portability and Accountability Act (HIPAA), have been reviewed, and clear policies and procedures for protected health information are in place, including prohibited actions and rules around disclosure and information-sharing.

G. Vendors or IT staff use demos to explain how to approach security and discuss how they have handled HIPAA in the past.

H. Privacy solutions that are not technology based are established, such as signed consent forms from individuals to share their data.

3. Has the necessary user testing been conducted to ensure that the data warehouse is secure and functioning properly?

A. Privacy protocols are tested with sample data across various participating agencies and levels of access.

B. The data warehouse produces audit logs, which are monitored daily by IT staff to protect against unauthorized use or other suspicious activity. Audit logs are available to designated personnel in each contributing agency.

C. User testing is conducted with sample data across all features and user roles to ensure optimal functionality of the data warehouse.

D. If a vendor is needed, the vendor contract is managed by a project manager and includes requirements to train agency IT staff on how to manage the data warehouse, system requirements, use of manuals, troubleshooting common problems, and customizing reports. The contract also requires the vendor to provide training on software upgrades to the data warehouse.

Phase 3 Implementing and Maintaining a Data Warehouse

* All statements are answered with either Yes, Underway, Planned, or No.

1. Are training and ongoing supports in place to assist users of the data warehouse?

A. Users regularly acknowledge their responsibilities regarding use of information in the data warehouse and data privacy through mechanisms such as computer-based modules that can serve as logs for privacy compliance checks.

B. New staff receive an orientation on data protection and use, as well as functionality of the data warehouse and how to use data to inform their work.

C. Staff develop competencies in interpretation and use of data and reports.

D. Existing staff receive annual training on data security, compliance, phishing, and flags for misuse of the system (e.g., looking up people for personal reasons) to ensure the proper use of data and compliance with privacy requirements.

E. A help desk, frequently asked questions resources, and access to in-person support are in place, including a designated individual to troubleshoot system operational issues and communicate resolution of issues to other users.

2. What quality assurance and improvement mechanisms are in place?

A. A process is in place for reviewing and cleaning the data for accuracy before it is uploaded into the warehouse, and a timeframe of when data must be active in the system is established.

B. A system is in place to regularly assess the quality and reliability of data from the agencies participating in sharing their information.

C. The governance group regularly reviews the system and processes established above for quality assurance.

3. How is the governance group working to financially sustain the data warehouse?

A. A subcommittee or task force is established to identify and maximize existing resources and secure additional funding for both immediate and long-term needs (e.g., getting a line item in the budget of all participating agencies, talking to private funders, applying for grants, etc.).

B. Funding is in place for quality assurance and system enhancements.

C. The project manager oversees the vendor contract and the subcommittee’s use of the budget (see Phase 1, 6b) to assess future financial needs, such as potential changes in funding sources or the vendor contract (if necessary).

D. Members of the governance group promote the benefits of the data warehouse through activities such as media engagement and policymaker outreach to ensure that people with budget authority continue funding the warehouse and to attract potential funders.

4. Are policies in place to prescribe how the data warehouse will be enhanced or modified to meet evolving needs?

A. Mechanisms are in place for system users to submit feedback and provide suggestions for fixing problems and for improving the functionality with new capabilities or features.

B. IT staff are designated to oversee maintenance of the warehouse.

C. Meetings are held to review data analyses with users to improve data quality, refine the fields and screens used for capturing data, build out custom reports, and apply learnings to their work.

D. The governance group hosts regular brainstorming meetings about how the data can inform various policies and practices in their agencies and county and how the warehouse could be enhanced to meet emerging needs.

E. If a vendor is selected, ongoing maintenance agreements have been written into the contract.

5. Are stakeholders regularly informed about the value and benefits of the data warehouse to engender support and promote its use?

A. The governance group meets regularly to review success stories and system improvements and uses this information to build a public education campaign that educates stakeholders and the public on the successes of the warehouse. Included in this campaign is a public-facing document that demonstrates the process of collaboration and data integration across agencies.

B. Regular reports are provided to participating agencies, which include results of security checks to help create confidence and trust in the warehouse.

C. Data analysis and reports are created and tailored to the staff across agencies who use the warehouse to encourage staff engagement and keep them abreast of information they need from the warehouse to guide their work. This includes meetings or townhalls to discuss the information provided.

D. Regular annual reports are published outlining progress on strategic goals, activities, and outcomes produced by the governance group and the data warehouse.

Appendix Sample Organizing Stakeholder Structure to Develop a Data Warehouse

This project was supported by Grant No. 2019-MO-BX-K001 awarded by the Bureau of Justice Assistance. The Bureau of Justice Assistance is a component of the Department of Justice’s Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, the Office for Victims of Crime, and the SMART Office. Points of view or opinions in this document are those of the author and do not necessarily represent the official position or policies of the U.S. Department of Justice.

Project Credits

Writing: Stephanie Shaw, CSG Justice Center, Robert May, Integrated Justice Information Systems Institute, and Elizabeth Fleming, CSG Justice Center

Research: Robert May, Integrated Justice Information Systems Institute, and Stephanie Shaw, CSG Justice Center

Advising: Risë Haneberg, CSG Justice Center

Editing: Darby Baham and Emily Morgan, CSG Justice Center

Design: Michael Bierman

Public Affairs: Ruvi Lopez, CSG Justice Center

ABOUT THE AUTHORS

Stephanie Ueberall

Deputy Program Director, Corrections and Reentry

Stephanie Ueberall works with states and counties to improve outcomes for youth in the juvenile justice system. Stephanie has also worked to reduce the overuse of jail through initiatives such as Stepping Up and the MacArthur Foundation’s Safety and Justice

...

Challenge. She has extensive experience with youth and families involved in the justice system through direct service, research, technical assistance, and program design. Most recently, Stephanie was the director of violence prevention at the Citizens Crime Commission of New York City, where she worked to enhance community-led youth gun violence prevention. Stephanie holds a BS in psychology from the University of Arizona and an MA in community psychology from the University of New Haven.

Elizabeth Fleming

Senior Operations Specialist, Behavioral Health

Elizabeth Fleming provides writing, research, and project management support to the Behavioral Health Division. Prior to joining the CSG Justice Center, she was a fine art photographer for over a decade and served as a chapter leader for the International