Risk Assessment and Implementation
Recently, I wrote about risk assessment for attorneys. In the past, when I have worked with correctional agencies or conducted research on risk assessment, my focus has been to explain how risk assessments are used to predict reoffending. Or, sometimes the focus was on training-- understanding how to use a risk assessment tool in practice, such as conducting interviews and file review to score out an actual risk assessment tool. Writing about risk assessment for attorneys, however, was a great opportunity for me to step back from the focus on what risk assessment does, and instead, to think about what it does not do or its limitations. I will not spend my time here in this blog post re-writing what I wrote for attorneys in the August issue for Bench and Bar of Minnesota, but I do want to elaborate further on one aspect of risk assessment I did highlight briefly: the importance of implementation.
Before I talk about implementation, I will first clarify what I mean by risk assessment. Broadly speaking, risk assessment can mean a lot of things in the criminal justice system. There are risk assessments conducted at sentencing or there are risk assessments designed specifically for the pre-trial stage, which are used to predict risk of reoffending and failure to appear. Some of these implementation issues I will discuss apply to other types of risk assessments, but for purposes of this discussion, I will focus on the use of third or fourth generation risk and needs assessment in corrections. These tools are actuarial, which means they are developed out of data to identify predictors of risk1 much like car insurance companies look at data to identify people more likely to get in an accident. A risk and needs assessment tool serves a few purposes: it identifies someone’s risk to reoffend and identifies their criminogenic needs.2 A risk assessment tool identifies a risk score and a risk level, which classifies someone according to low, moderate, or high risk. In this context, risk means someone’s likelihood or chances of committing a new offense. In corrections, when it is a risk and needs assessment tool, it also includes information on criminogenic needs. These are needs that are related to someone’s risk and when they are targeted for change with treatment or interventions, we can reduce that risk.3
When I first started conducting training on risk assessment tools, I used to think of implementation in terms of the day-to-day use of a risk assessment tool. The focus was on the administration of the tool and the fidelity of the scoring. So, did the probation officer ask the right questions in the interview with the offender to get the information they needed to score out the risk assessment? And, did the probation officer interpret the scoring guide correctly? Did the officer use the risk level provided by the tool or did they override the assessment (without compelling evidence and according to policy, sometimes overrides are okay)? These types of questions are concerned with how well staff implements the risk assessment scoring, as it was designed. And by extension, that means we are also concerned with the data driving the decisions; after all, the data creates the risk level and the scoring protocol, so if it is not followed by staff then we are not following the data. The whole purpose of risk assessment is to guide our decisions based on 100’s or 1,000’s of cases, where we can pick-up predictive relationships more objectively than if we decided to use our own personal experience or judgement.
However, when you work directly with a probation department or a corrections agency, there are other aspects of implementation that go beyond how staff scores the instrument itself. It is important that risk and needs assessment tools get incorporated into actual agency decision-making. If staff or probation officers conduct a risk assessment on an offender and place the risk assessment tool in a drawer never to be looked at again, an agency is not really using risk assessment. This is problematic in many ways. First, risk assessments can cost money, and it is a waste of resources to invest money and staff time in risk assessment then never use it. But even more concerning is that risk assessments are necessary for corrections agencies to make evidence-based decisions. Research instructs us that to reduce recidivism, we need to identify both the low risk people who do not need our time and resources and then identify the higher risk people who do require supervision and resources.4 This point is also important for researchers who evaluate agencies using risk assessment tools; it is not enough to look at risk assessment data but you must understand how risk assessment is used in the agency.
At an agency level, the implementation of risk assessment can be monitored with quality assurance processes and tools. Agencies should examine things like inter-rater reliability of scoring between staff and observe staff conducting assessments. It is also important that agencies do more than audit risk assessment to check if one was conducted. Agencies must also take a close look and examine how the tool was used to make decisions. It is not enough to check a file folder and look the assessment. We also want to know things like: Do probation officers put their high risk people on more intensive supervision where they report more often? Do low risk people report less often? Does the case manager provide programming on the criminogenic needs identified for each person, or does everyone get the same programming regardless of their risk and needs results? By not applying the results of the tool, agencies are doing more than wasting resources, they are running the hazard that they are supervising low risk people at high levels (which can increase recidivism) and not providing the support and resources needed for high risk people that will give them the tools and knowledge they need to succeed in the community and stay away from criminal activity.
But beyond the agency or the staff, there are system considerations regarding implementation. I have been fortunate to work with states who have brought agencies together to align their policies and practices with evidence-based practices and tools. But bringing systems together to incorporate risk assessment and support it can be challenging. It requires a purposeful approach and ongoing support and expertise to ensure research, policy, and practice align appropriately. Some agencies are fortunate to get the resources and expertise to work with risk assessment developers and research experts that help by conducting training on tools, implementing quality assurance processes, and developing agency polices to incorporate risk assessment effectively into agency practices for supervision and interventions. But at the system level, a state might work with agencies who do not have the resources to train their staff on risk assessment or have dedicated internal resources to monitoring the fidelity of their risk assessment tool and practices.
Furthermore, at the system level, different components of the system are dealing with different goals beyond rehabilitation. This means that implementation at different stages of the criminal justice system must be concerned with how the tool gets incorporated into system decision-making. Most correctional agencies are tasked with the primary goals of maintaining public safety and rehabilitation of people when they reenter or serve their time in the community. Risk and needs assessments were developed to complement these goals, which is why they identify risk of reoffending (public safety) and they identify criminogenic needs (needs when addressed, can rehabilitate someone). But a judge who is sentencing a person to probation or prison for a certain amount of time, is concerned with more than just rehabilitation or public safety – they are also concerned with delivering a sentence in proportion to the crime that was committed and they are also considering if the person will be deterred in the future, as well as the public. Therefore, risk and needs assessment may help them understand someone’s risk to public safety or what resources they need to change, but risk and needs assessment does not account for retribution or deterrence. Risk and needs assessment might help a judge for some considerations, but only as one piece of information they might review (i.e. like in a presentence report). A judge would not want to use risk and needs assessment in the same way a probation officer might identify a supervision level based on risk. This example is used to illustrate that there is another way in which implementation maters. It is also important that risk and needs assessments tools themselves are used in the way they were designed and for the stage they were designed.
Another example might be at the pre-trial stage, where a judge might use a risk assessment to better assess someone’s likelihood of being rearrested or failing to appear, if released into the community before their trial. Here, a risk and needs assessment is not appropriate because it was not designed for pre-trial decision-making. Instead, there are actually tools designed specifically for this stage that look at predictors for both re-arrest and failure to appear. Furthermore, these tools do not create an automatic determination or decision free of judicial input. A judge considering pre-trial release must follow the state or federal laws in which they reside, which outlines what factors they must consider. Another example might be consideration of parole release. Here, risk and needs assessment is helpful for understanding someone’s chances of reoffending in the community and a risk and needs assessment goes further by providing the paroling authority with information on the areas someone needs treatment or services once released to parole. But, a parole board would not replace their decision-making fully with a risk assessment. Instead, it would be one part, and a very important part, of the factors they consider when deciding whether to grant someone release onto parole. Parole boards also consider parole readiness and as well as factors they must consider because they are outlined in law or policy, which may not tie directly to risk prediction. In the end, implementation matters at the system level too. It is important that risk and needs assessment does not get misapplied at a stage of the criminal justice system it was not designed for, since risk and needs assessments are concerned with predicting risk and identifying criminogenic needs for rehabilitative purposes.
In the end, implementation matters whether you are the staff conducting the assessment, the agency supervising by risk and needs, and a criminal justice system making decisions that consider public safety and risk reduction. Here is a list of different things to consider when implementing risk and needs assessments among staff, agencies, and the criminal justice system.
- Receive training on the risk assessment tool and seek advice from the tool’s developers on how the tool can be used effectively and appropriately in your agency.
- Adhere to the risk assessment tool’s scoring protocol and instructions; should you require modifications for the scoring guide, be sure to work with the tool’s developer so you can identify consistent and research-based adjustments.
- Evaluate or request someone to conduct research on your risk assessment tool to validate it on the people supervised by your agency/jurisdiction and on the types of people you supervise.
- Provide staff who use the tool with ongoing oversight and coaching to support staff in using the tool accurately.
- Apply the results of the risk and needs assessment to actual case decisions so that supervision and programming is matched to the risk and needs of people you supervise.
- Use the risk and needs tool according to the purpose and goals it was intended for and if structured decision-making is desired for other parts of the criminal justice system, be sure the tool is used in accordance with other factors or a more appropriate assessment instrument is selected.
- Collect data at the agency level and system level to determine how the tool is impacting your decisions and outcomes.
- Share concerns or issues with risk and needs assessment to researchers and risk assessment developers so that they might conduct additional research to provide evidence-based and data driven solutions to address those concerns.
- Engage stakeholders, funders, and policymakers on the importance of risk and needs assessment and implementation to receive the funding and support necessary to implement tools effectively.
- Remember that risk and needs assessment makes an improvement in prediction, but as with any attempt to predict human behavior, it is never perfect and therefore risk assessment is not a panacea for all problems; but structured decision-making is more likely to reduce bias and by using a tool based in data, it provides the opportunity to measure questions of fairness and impact.
For additional resources on risk assessment implementation:
Risk Assessment Quality Improvement Checklist (Council of State Governments Justice Center)
Risk and Needs Assessment 101 (Pew Charitable Trusts, Public Safety Performance Project)
Selecting and Using Risk and Need Assessments (National Drug Court Institute)
- 1Bonta, J. (2002). Offender risk assessment: Guidelines for selection and use. Criminal Justice and Behavior. 29, 355-379.
- 2Bonta (2002).
- 3Andrews, D. A. and Bonta, J. (2010a). The Psychology of Criminal Conduct, (5th ed.). Mathew Bender & Company, Inc. New Providence, NJ.
- 4Andrews, D. A., Bonta, J. and Hoge, R. D. (1990). Classification for effective rehabilitation: Rediscovering psychology. Criminal Justice and Behavior, 17, 19-52.