David Muhlhausen, Heritage Foundation: Drug and Veterans Treatment Courts: Budget Restraint and More Evaluations of Effectiveness Needed

January 09, 2012

David Muhlhausen is a Research Fellow in Empirical Policy Analysis in the Center for Data Analysis at The Heritage Foundation. Dr. Muhlhausen thanked Chairman Sheldon Whitehouse, Ranking Member Jon Kyl, and the rest of the Senate Committee Judiciary on Crime and Drugs for the opportunity to testify on Drug and Veterans Treatment Courts. His views in this testimony are his own and should not be construed as representing any official position of The Heritage Foundation.

His testimony focuses on the following points:

• As a state and local policy, drug courts represent a policy alternative to help rehabilitate non-violent offenders with drug addiction problems;
• With out-of-control spending and surging public debt threatening our nation’s stability, increased federal funding of state and local drug courts should not be a priority;
• Instead of using Drug Court Discretionary Grants to subsidize the routine operations of state and local drug courts, Congress should consider reforming the program to focus entirely on reimbursing state and local drug courts that serve recently returned combat veterans;
• More scientifically rigorous experimental evaluation of drug courts are needed to determine their effectiveness; and
• While under some circumstances in particular locations, drug courts may be more effective than traditional court responses, Congress should carefully review claims of effectiveness coming from advocates of increased federal spending on drug courts.

Drug Courts
As a state and local policy, drug courts represent a policy alternative to help rehabilitate non-violent offenders with drug addiction problems. In 2010, the National Institute of Justice counted 2,559 drug courts operating within the United States. The majority of these drug courts serve adults (1,372) and juveniles (483). Of particular importance to today’s topic, there are 31 drug courts specifically serving veterans.

Drug courts have become a good alternative for addressing non-violent drug offenders. Typically, drug courts process offenders through either diversion or postadjudication programs. Diversionary programs allow for substance abusing defendants to be diverted from the traditional court system. The diversion allows defendants to avoid traditional corrections interventions in exchange for possibility of dismissed charges or reduced sentences for successful completion of treatment. Postadjudication programs offer convicted offenders deferred or suspended sentences to those that successfully complete program requirements.

Drug court proceedings are overseen by judges who are expected to closely monitor the progress of participating defendants during status hearings. Collaborating with prosecutors, defense attorneys, probation agents, treatment providers, and other justice system officials, judges administer sanctions for noncompliance with program requirements and rewards for compliance. Through sanctions and rewards, drug court judges are expected to increase the chances of rehabilitating participating defendants.

For most drug courts, the defendant’s participation is voluntary. However, drug courts normally screen potential defendants to limit participation to those that meet the programs’ participation criteria before extending invitations to voluntarily participate. In addition to volunteering, defendants must agree to certain conditions. Such conditions often include mandatory drug testing and participation in drug treatment.

Out-of-Control Spending
With out-of-control spending and surging public debt threatening our nation’s stability, funding programs that are not the responsibility of the federal government should not be a priority. Indeed, “by the end of this year,” a June 2011 report by the Congressional Budget Office (CBO) warns, “federal debt will reach roughly 70 percent of gross domestic product (GDP)—the highest percentage since shortly after World War II.” In 2009, the CBO warned that these “Large budget deficits would reduce national savings, leading to more borrowing from abroad and less domestic investment, which in turn would depress economic growth in the United States. Over time, the accumulation of debt would seriously harm the economy.” This is hardly a good time for Congress to increase grant programs that subsidize the routine criminal justice operations of state and local governments.

Drug Court Discretionary Grant Program
Created by the Violent Crime Control and Law Enforcement Act of 1994 (P.L. 103-322), the Drug Court Discretionary Grant Program is administered by the Bureau of Justice Assistance. Congress has allocated over $530 million since creating the program. While this assistance likely contributed to the spread of drug courts, the subsidies should not continue indefinitely. However, if Congress chooses to reauthorize this program, then such legislation should require a multi-site experimental evaluation of federally funded drug courts.

When Congress creates programs, especially state and local grant programs, the funded activities are implemented in multiple cities or towns. Federal grants are intended to be spread out across the nation. For this reason, Congress should require a national, multi-site experimental evaluation of these programs.

While individual programs funded by federal grants may undergo experimental evaluations, these small-scale, single-site evaluations do not inform policymakers of the general effectiveness of national programs. Small-scale evaluations only assess the impact on a small fraction of people served by federal grant programs. The success of a single program that serves a particular jurisdiction or population does not necessarily mean that the same program will achieve similar success in other jurisdictions or among different populations. Thus, small-scale evaluations are poor substitutes for large-scale evaluations.

In addition, a multi-site experimental evaluation that examines the performance of a particular program in numerous and diverse settings can potentially produce results that are more persuasive to policymakers than results from a single locality.

Federalism Concerns
To address the problem of drug abuse and addiction appropriately, the federal government should limit itself to handling tasks that have been assigned to it by the Constitution and that state and local governments cannot perform by themselves. The tendency to search for a solution at the national level is misguided and problematic. While the problems faced by those addicted to illegal drugs and the crimes they cause are serious and common to all states, these problems are almost entirely and inherently local in nature and should be addressed by state and local governments.

Increasing the national government’s involvement in funding state and local drug courts is detrimental to quintessential federal responsibilities. Using federal agencies and grant programs to provide basic drug court operations for state and local drug abuse offenders that the states themselves could provide is a misuse of federal resources and a distraction from concerns that are truly the province of the federal government.

That being said the national government has a responsibility to assist members of the U.S. Armed Forces that are transitioning to civilian status. In particular, combat veterans arrested for substance abuse problems that have recently transitioned back into civilian life may benefit greatly from drug courts. In these cases, Congress may consider providing limited financial assistance to state and local drug courts that serve combat veterans. Such federal assistance should be limited to recent veterans whose substance abuse problems can be credibly linked to their service in the Armed Forces. The federal assistance should not go beyond paying for the cost incurred for dealing with recent combat veterans. Any provided assistance should not be used to subsidize the routine operation of state and local drug courts.

Instead of using Drug Court Discretionary Grants to subsidize the routine operations of state and local drug courts, Congress should consider reforming the program to focus entirely on reimbursing state and local government drug courts that serve recently returned combat veterans. This reform would get the federal government out of subsidizing the routine operations of state and local drug courts and, quite likely, save taxpayer dollars as well.

The Importance of Rigorous Scientific Evaluations
The effectiveness of drug courts has been widely researched. While a large number of studies find that drug courts reduce recidivism and drug use, many of these studies have significant shortcomings in scientific rigor.

The principal reason for the existence of drug courts is to rehabilitate non-violent drug abusers. Scientifically rigorous impact evaluations are necessary to determine whether these programs actually produce their intended effects. Obviously, there is little merit in the continuation of programs that fail to ameliorate the social problems they target.

Although estimating the impact of programs cannot be made with 100 percent certainty, they can be made with varying degrees of confidence. Impact evaluations face control problems that make successful impact estimates difficult. As a general rule, however, the more rigorous the research methodology, the more confident we can be of the validity of the evaluation’s findings.

Broadly speaking, two types of research designs have been used to evaluate drug courts: experimental and quasi-experimental. Experimental evaluations that use the random assignment of individuals to the intervention and control groups represent the “gold standard” of evaluation designs. Random assignment helps ensure that the control group is equivalent to the intervention group. Equivalence means that the intervention and control groups have the same composition, predispositions, and experiences. Experimental evaluations are considered to be superior to quasi-experimental evaluations.

Under quasi-experimental designs, the intervention and comparison groups are formed by a procedure other than random assignment. Quasi-experiments frequently employ methodological and statistical techniques to minimize the differences between intervention and comparison groups that influence the outcomes being measured. This design frequently matches intervention and comparison group members together based on factors thought to influence program impacts.

A major problem with quasi-experimental evaluations is selection bias. Before we can judge a drug court program to be effective, we first must understand the importance of selection. It can be astoundingly difficult to distinguish between what is working and what is not, and nowhere is this predicament truer than when the criminal justice system tries to change human behavior. For example, individuals volunteering entry into a drug court program may be more motivated than individuals not seeking the benefits of the program. Such motivational factors and other similar factors are often invisible to those assessing effectiveness. Failure to account for these crucial factors can produce a spurious association between drug court participation and recidivism and substance abuse outcomes. Most quasi-experimental evaluations of drug courts are unable to adequately deal with the problem of selection.

There is evidence that in the realm of criminal justice policy that quasi-experimental evaluations are more likely to find favorable intervention effects and less likely to find harmful intervention effects. No matter how well designed, quasi-experimental evaluations may be incapable of controlling for the factors that make individuals considered agreeable and allocated to the intervention group.

After conducting a meta-analysis of 308 criminal justice program evaluations, Professor David Weisburd of George Mason University and his colleagues found that weaker evaluation designs are more likely to find favorable intervention effects and less likely to find harmful intervention effects. Given that experimental evaluations produce the most reliable results, this finding has important ramifications for reviewing evaluations of drug courts.

Are Drug Courts Effective?
While under some circumstances in particular locations, drug courts may be more effective than traditional court responses, Congress should carefully review the claims of effectiveness coming from advocates of increased federal spending on drug courts.

A meta-analysis of 55 evaluations of drug courts by Professor David B. Wilson of George Mason University and his colleagues concluded that their findings “tentatively suggest that drug offenders participating in a drug court are less likely to reoffend than similar offenders sentenced to traditional correctional options, such as probation.” Only five of the 55 evaluations used experimental designs, while the remaining 50 studies used quasi-experimental designs.

Another major limitation of quasi-experimental evaluations of drug courts is the too frequent comparison of drug court graduates to nongraduates. The “much-heralded findings” based on this faulty methodology “show that the successes succeed and the failures fail.”

Of the five experimental evaluations, two studies suffered from high rates of attrition. If individuals voluntarily participating in drug courts drop out of the program, then attrition can seriously invalidate the integrity and benefits of random assignment. While the combined effect of the three experimental evaluations without serious attrition problems had a negative association with recidivism, the effect was not statistically significant. The findings of these experimental evaluations are presented in greater detail in the next section.

Drug Court Evaluations
Despite the substantial number of drug court evaluations, there are not enough experimental evaluations. My review of drug court evaluations focuses on the previously mentioned experimental evaluations and a particularly good multi-site quasi-experimental evaluation.

Baltimore City Drug Treatment Court. Several studies based on an experimental evaluation of the Baltimore City Drug Treatment Court, a diversionary program that primarily deals with African-American male heroin addicts, have been published. A 2002 study found that drug court participants and control group participants had a one-year re-arrest rate of 48.2 percent and 63.5 percent, respectively. The difference was statistically significant, suggesting that the program was effective. Drug court participants had fewer new arrests and new charges, but were just as likely to be reconvicted and experience new convictions. Drug court participants experienced statistically fewer charges for violent and sex crimes, but were no more or less likely to be charged with property, drug, and other crimes.

Another study published in 2003 analyzed the effect of the drug court on two-year recidivism outcomes. The treatment group had a re-arrest rate of 66.2 percent, compared to the control group’s re-arrest rate of 81.3 percent—a statistically significant finding. The treatment group had fewer average new arrests with 1.6 arrests, compared to 2.3 arrests for the control group. This difference was statistically significant. However, the reconviction rates of the treatment group (48.9 percent) and control group (53.2 percent) were not statistically different. Further, drug court participation did not lead to a decrease in the average number of convictions. While the treatment participants were statistically less likely to be charged with new drug crimes (40.6 percent versus 54.2 percent), the drug court did not appear to affect charges involving violent or sex crimes, prostitution, and other offenses.

A 2004 study analyzed the drug court’s effect on the length of time to re-arrest.The study found that the failure (arrest) rate for the treatment and control groups were identical during the first four months of follow-up.Afterwards, the failure rate of the groups began to sharply diverge with the treatment group experiencing longer periods until a new arrest for latter follow-up periods up to two years after random assignment.

Last, a 2006 study followed the treatment and control groups three years after random assignment. The treatment and control groups had re-arrest rates of 78.4 percent and 87.5 percent, respectively. The difference failed to be statistically significant. While not exactly comparable, it is interesting to note that a Bureau of Justice Statistics study of state prisoners released in 1994 whose most serious conviction was for drug possession found that 67.5 percent of these prisoners are rearrested within three years.

The same pattern held for re-conviction rates. The treatment group had a re-conviction rate of 58.3 percent, while the reconviction rate for the control group was 64.4 percent—a statistically indistinguishable difference. While the treatment group was statistically less likely to be charged with new drug crimes, drug court participation appeared to have no effect on new charges for violent or sex crimes, property crimes, and other offenses. The authors warn that their study cannot be generalized to courts unlike the Baltimore program. Further, study’s findings “do not necessarily generalize to different client populations than the one studied here.”

Wilmington , Delaware , Drug Court. A 2003 experimental evaluation analyzed the effect of a Wilmington, Delaware, drug court while defendants were participating in the program. Volunteers were assigned to either a drug court that held bi-weekly judicial status hearings (treatment group) or a program with monitoring performed by a case manager (control group). Participants of both groups were eligible to receive that same drug treatment services and level of drug testing. While both the treatment and control groups were participating in their respective programs over 14 weeks, there was no statistically significant difference in drug test results. In addition, the treatment and control groups did not experience statistically significant differences in the self-reported number of days of drug use, alcohol intoxication, and illegal activity. The results of this study should be taken with caution because it does not measure post-program outcomes. However, a 2005 evaluation of the same program reported results for 6-month and 12-month post-treatment outcomes. The evaluation “did not find post-treatment differences in outcomes for misdemeanor drug court clients who were assigned to higher versus lower doses of judicial status hearings.”

New South Wales , Australia, Drug Court. A 2004 experimental evaluation of an Australian drug court, unlike most drug courts in America, made substantial use pharmacological therapies, such as methadone treatment. While the evaluation found that the treatment group experienced longer periods of time without being arrested for new shoplifting and drug offenses, the difference was not statistically significant. Based on 13 offenses, the evaluation compared the average number of offenses committed over one year. For only one of the measures, drug court participants had a statistically significant difference. The treatment group averaged 0.08 new drug offenses, while the control group averaged 0.68 new offenses. Being in the treatment condition was not associated with fewer average incidences of theft, breaking and entering, motor vehicle theft, and other crimes.

Maricopa County , Arizona , Drug Court. A 1995 experimental evaluation of the Maricopa County, Arizona, Drug Court randomly assigned post-adjudication probationers with a first-time felony conviction for drug possession to four different tracks. The first three tracks consisted of standard probation services, but varied in the frequency of drug testing. For the fourth track, participants were entered into a drug court that administered sanctions and provided drug treatment. At the twelve-month follow-up period, drug court participants were just as likely to test positive for any drug use compared to the regular probation participants. While drug court participants were statistically less likely to test positive for cocaine and heroin use, they were statistically more likely to test positive for marijuana use.

As for recidivism, drug court participants, compared to regular probation participants, were just as likely to be arrested for any offense and also just as likely to be arrested for drug offenses. While there was statistically no difference in being arrested for any technical violation, drug court participants were statistically less likely to be arrested for drug use technical violations, compared to regular probationers (9.7 percent versus 26.4 percent). Further, the drug court did not appear to be effective at preventing future incarceration in jail or prison for new offenses or technical violations. The results of this evaluation may not reflect the current operation of the Maricopa County drug court.

Multi-Site Adult Drug Court Evaluation. Only one large-scale evaluation of drug courts has been conducted. The Multi-Site Adult Drug Court Evaluation (MADCE) was performed by the Urban Institute through a grant from the National Institute of Justice. MADCE compared treatment participants from 23 drug courts to a group of individuals from six comparison sites. The comparison sites offered such treatment services as Treatment Alternatives for Safer Communities, Breaking the Cycle program, and typical probation supervision accompanied with referral to treatment. While MADCE did not use an experimental design, the propensity score analysis used in this quasi-experimental evaluation makes this evaluation more scientifically rigorous than many other quasi-experimental designs.

The MADCE results indicate that the drug courts in the evaluation reduce drug use and criminal activity. For drug use at the six-month tracking period, drug court participants were significantly less likely to self report any drug use (40 percent), compared to the comparison group (59 percent). Drug court participants averaged fewer days of any drug use (1.5 days per month), compared to the comparison group (3.7 days per month). However, the difference in self-reported serious drug use between drug court and comparison participants was statistically indistinguishable from zero. Despite being not less likely to report serious drug use, drug court participants reported an average of 1.0 day per month of serious drug use, compared to the 2.2 days per month reported by the comparison group. This difference was statistically significant. Drug court participants were less likely to report using marijuana (13 percent versus 26 percent), alcohol (32 percent versus 52 percent), and prescription drugs (6 percent versus 10 percent). While they reported less usage of cocaine, heroin, amphetamines, and hallucinogens, these differences were not statistically significant.

At the 18-month follow-up period, drug court participants were significantly less likely to self report any drug use (56 percent), compared to the comparison group (76 percent).[39] Drug court participants still averaged fewer days of any drug use (2.1 days per month), compared to the comparison group (2.3 days per month). Contrary to the six-month follow-up, the difference in self-reported serious drug use between drug court and comparison participants was statistically different (41 percent versus 58 percent). Drug court participants reported an average of 1.1 days per month of serious drug use, compared to the 2.3 days per month reported by the comparison group. Drug court participants were less likely to report using marijuana (23 percent versus 36 percent), alcohol (47 percent versus 67 percent), methadone (2 percent versus 4 percent), and prescription drugs (6 percent versus 15 percent). While they reported less usage of cocaine, heroin, amphetamines, and hallucinogens, these differences were not statistically significant.

Also during the 18-month follow-up, non-incarcerated MADCE participants were administered oral saliva tests. Of those tested, drug court participants were significantly less likely to have positive results for any drug use (29 percent), compared to the comparison group (46 percent).[40] In addition, drug court participants were significantly less likely to test positive for PCP (0 percent versus 2 percent). Drugs courts appeared to have no effect on positive drug test results for serious drugs, marijuana, cocaine, opiates, and amphetamines.

The drug courts in the evaluation appear to have a limited ability to reduce criminal activity. While drug court participants were generally less likely to self-report criminal activity than the comparison participants, I will concentrate on official re-arrest and incarceration results. Twenty-four months after enrollment in the evaluation, the difference in re-arrest rates for drug court participants and comparison participants were not statistically different. For example, 52 percent and 62 percent of drug court and comparisons participants were arrested—a statistically indistinguishable difference. The differences in drug arrests were also statistically indistinguishable (17 percent versus 22 percent).

According to administrative data for the 24-month follow-up, drug court participants were not less likely to be incarcerated compared to the comparison participants. However, drug court participants, on average, spent significantly fewer days in custody than comparison participants (32.1 days versus 59.4 days).

Of particular importance for policymakers, MADCE performed a cost-benefit analysis. According to the evaluation, “Drug courts save an average of $5,680 per participant, returning a net benefit of $2 for every $1 of cost, but these findings are not statistically significant.” The estimated mean net benefit of drug courts is $2,213 and the standard error is $3,682. The reason why the net benefit is statistically insignificant is that the 95 percent confident interval ranges from a mean net cost of $5,004 to a net benefit of $9,430. The 95 percent confidence interval means that policymakers cannot be sure that the drug courts participating in MADCE are producing net benefits. The estimate is too imprecise to draw strong policy conclusions. The possibility that the costs of drug courts outweigh their benefits cannot be ruled out with a high degree of confidence.

Conclusion
The lesson to be learned from the three experimental evaluations and the single multi-site quasi-experimental evaluation is that the effectiveness of drug courts is a mixed bag. Evaluations do not find that drug courts are effective on all outcome measures. Nor do these evaluations find that drug courts are ineffective on all measures.

The overwhelming majority of drug court evaluations use quasi-experimental designs that may overstate effectiveness. With only a small number of experimental evaluations, Congress should carefully review claims of effectiveness coming from advocates of increased federal spending on drug courts.

Drug court programs need to be rigorously evaluated using experimental designs. I believe the need for more experimental evaluations transcends political party lines. Both Democrats and Republicans should agree on this issue.

The federal government has a responsibility to assist members of the U.S. Armed Forces that are transitioning to civilian status. In particular, recent combat veterans with substance abuse problems may benefit greatly from drug courts. In these cases, limited financial assistance provided to state and local drug courts that serve combat veterans may be warranted. However, any federal assistance should not be used to subsidize the routine operation of state and local drug courts.

Last, Congress should be wary of substantially increasing the budget authorization for the Drug Court Discretionary Grant Program. Given the sparingly few experimental evaluations of drug courts, any reauthorization or significant funding increase of the Drug Court Discretionary Grant Program should be accompanied with a congressional mandate for a large-scale, multi-site experimental evaluation of drug courts.