A New Way to Reform the Judicial System

A new initiative seeks to identify the sources of mistakes in the legal system including in laboratories where potential...
A new initiative seeks to identify the sources of mistakes in the legal system, including in laboratories where potential evidence is analyzed.PHOTOGRAPH BY CULTURA RM/ALAMY

Last year, the district attorney’s office in Montgomery County, Pennsylvania, blew a case. The chairman of the county’s Republican Party, Robert J. Kerns, had been accused of rape by a woman who worked at his law firm. The woman said that Kerns had offered her a ride after an alcohol-fuelled office party. Along the way, she said, he gave her wine and raped her in his Mercedes, and then again in her home. Hospital reports showed bruising consistent with a sexual assault, and DNA on the woman’s underwear was consistent with Kerns’s profile. A key piece of evidence was a urine test apparently showing the presence of Zolpidem, commonly known as Ambien. Prosecutors secured a grand-jury indictment on more than a dozen criminal counts, including rape and aggravated indecent assault. Afterward, they held a press conference.

Several months later, a toxicologist hired by Kerns’s defense took a closer look at the lab report. Although the word “Zolpidem” appeared, what the document indicated was that the test had detected "less than" five nanograms per milliliter, which in this case was zero. Kerns’s lawyer got in touch with the prosecuting attorneys, who were horrified to realize that they had misinterpreted the findings—a rookie mistake.

“It was a huge embarrassment,” Risa Ferman, Montgomery County’s district attorney, told me. She and her staff had plenty of evidence that Kerns had committed a sexual assault, but, because the drugging was written into the indictment, they had to drop charges and refer the case to the Commonwealth’s Attorney General’s office. A newspaper called the incident a “fiasco.”

Normally after such a mistake, the D.A. would fire the responsible parties and announce that she had cleaned house. Instead, Ferman did things differently: rather than find a culprit to blame, she held a series of meetings to discover the organizational errors that had led to the mistake. “These were skilled professionals,” she told me, who had not set out to sabotage their case. What factors, she wondered, had caused competent people to make bad choices?

In asking this question, Ferman was following a procedure alien to the justice system but entrenched in the health-care and transportation industries. A few decades ago, administering anesthesia was one of the most dangerous medical procedures, and had a mortality rate of about one in ten thousand. By analyzing the circumstances of those deaths through an independent review process, experts learned that a few simple equipment changes could save lives: making the nozzles and hoses of oxygen and nitrogen incompatible, for instance, so that patients could not be given the wrong gas. Today, the death rate involving anesthesia hovers at around one in a hundred and eighty thousand.

Nowadays, flying a commercial airplane is one of the safest things that you can do, notwithstanding high-profile tragedies such as the crash of a Germanwings flight last week. That’s because, after each accident, the National Transportation Safety Board conducts a thorough and objective review, protecting involved parties from prosecution and liability, and focussing solely on improving safety. (Even though the N.T.S.B.’s findings are made public, the information is not admissible as evidence in court.) Many improvements, from the strip lighting along the aisle to the way in which cockpit staff communicate with each other, resulted from this review process, often referred to as “sentinel event analysis.”

A series of experiments over the past year has aimed to build similar safeguards into the justice system. A veteran Boston defense attorney, James Doyle, observed the proliferation of exoneration cases in the post-DNA era and has worked on a number of reform efforts. “No one gets into this job to convict innocent people,” he told me. “The real problem is developing the capability for dealing with inevitable mistakes.” He wondered if “sentinel event analysis”— reviewing legal errors in a blame-free environment—could tease out the sequence of factors that might have contributed to a mistake and, perhaps, lead to a more accident-proof legal system.

Sponsored by the National Institute of Justice, Doyle travelled the country interviewing police, prosecutors, defense attorneys, and victims’-rights groups, among others, culminating in a kind of summit meeting in Washington, D.C. Based on his work, the Institute organized an experiment in which three jurisdictions—Milwaukee, Baltimore, and a third, in Philadelphia—volunteered to do a systems analysis of a high-profile failure. The Montgomery County experiment, conducted in parallel with the N.I.J. study, was a fourth.

In every case, the horrendous legal accident turned out to have multiple causes embedded in the legal system. There was no single bad actor. The Milwaukee case involved an eighteen-year-old named Markus Evans who murdered a seventeen-year-old girl, Jonoshia Alexander. First arrested when he was seven, Evans later shot a cousin with a shotgun, when he was fifteen. He spent only fourteen months in a juvenile facility. Released without any supervision, he grabbed his shotgun one day and killed Alexander as she walked home from school.

“This was a kid who had red flags all over him,” John Chisholm, the Milwaukee County district attorney, told me. “Why was he still in the community?” Rather than blame the judge who had given Evans the short sentence, Chisholm convened a group of more than thirty people representing every agency that had made contact with Evans, including the public-health department, the school system, probation departments, and the police. Their meetings, over a period of several months, revealed that, in almost every incident, the people who made decisions about the boy had not seen his larger pattern of violent behavior because they did not have access to his complete records, or did not see them. The Milwaukee police did not have access to his juvenile records because, by the age of seventeen, he was legally an adult. Had the police seen the records, “they certainly would have had a heightened awareness of him in the community,” Chisholm told me. In response to the meetings, Milwaukee authorities have expanded the availability of children's-court data so that everyone involved has access to the whole picture. They have also scheduled regular meetings among agencies that deal with troubled young people, to systematize the sharing of information.

In Baltimore, the police department conducted a systems review of a police officer who had committed a series of violations throughout his career, finally leading to criminal charges and prison. Captain Martin Bartness, who led the review team, said that he could not discuss the case in detail because it involved confidential personnel records. But he did say that the review allowed the department to identify seemingly minor perturbations—poor performance evaluations, excessive medical leaves, discourtesy complaints—as warning signs for early intervention.

In Philadelphia, the group reviewed the response to the worst mass shooting in the city's modern history: the Lex Street massacre, in December, 2000, in which seven people were murdered in a crack house. After the shooting, police arrested four men based on a confession and a corroborating eyewitness. They held the four men for eighteen months. Just before the trial, the police decided that they did not have a case, set the men free, and settled on four other suspects, who were later found guilty. Even though the case had a fortuitous ending, it was widely seen as a bungled investigation, and the four innocent men won a $1.9 million settlement from the city of Philadelphia.

This case was reviewed with the assistance of the Quattrone Center for the Fair Administration of Justice, at the University of Pennsylvania Law School, which was established, in 2013, to explore and promote root-cause systems analysis in the legal profession. The center’s director, John Hollway, said that “the idea is to create a culture of learning from error—to look at what went wrong, what factored in the cases, and how to change the system so that doesn't keep happening.” The group—which included people from every aspect of the case, including police, defense attorneys, and the media—is still formulating its findings and corrective actions. Recognizing false confessions will be high on the list.

Ferman, the Montgomery County D.A., had attended a conference at the Quattrone Center about root-cause analysis, and when the Kerns scandal broke she thought it could be a perfect test run of the approach. Working with the center, she and her colleagues assembled a review team that, in addition to legal professionals, included two doctors familiar with systems analysis and a former managing director of the National Transportation Safety Board. “I stressed the fact that, although it's perfectly reasonable to be angry at a staff member who makes a mistake, you're deluding yourself if you think simply firing someone gets to the underlying cause of the error in the first place,” David Mayer, the former managing director of the N.T.S.B., said.

A complicated web of events and conditions emerged. It took the victim more than a day to collect her wits after the assault; she had woken up bruised, and with gaps in her memory. Family members took her to the hospital, where doctors found injuries consistent with rape and took urine for a lab test. A few days later, after the hospital told her the results, she told the police that she had been raped and that the lab showed the presence of Zolpidem.

That was when things went off track. When the state’s attorneys obtained the lab report, they saw “Zolpidem” with a quantity next to it. This was not the lab that they normally used, and the results did not appear in the usual format. The attorneys, having been notified by the victim of the results, wrongly assumed the numbers to be significant. The case was so politically sensitive that only a few people had access to the material. That ruled out a more general review, including by expert toxicologists who might have corrected the mistake. Furthermore, those attorneys who had doubts about the evidence did not feel entirely free to speak up. It was only later, when an outside toxicologist reviewed the results, that they realized that they had to drop the case.

It was a classic organizational error: a series of small slip-ups that cascaded into an important mistake.

The meetings showed Ferman that, even in a busy office like hers, she needed to create a step in which everyone could pause during certain complex or high-profile cases and have someone else take a fresh look at the evidence. She and her staff made structural changes, creating two new staff positions to offer independent review of such cases, and to serve as ombudsmen for attorneys who might have concerns. At the same time, the Montgomery County Detective Bureau formed a new division specifically to investigate cases involving violent crime and technology. Kerns, meanwhile, did not get off completely. He agreed to a plea deal with the Commonwealth in which he would spend fifteen years as a registered sex offender.

Howard Spivak, the deputy director of the National Institute of Justice, said that there’s “real excitement” about the results of these experiments. It won’t be easy for blame-free analyses to become the norm: unlike the transportation and medical industries, the legal system is inherently adversarial and resistant to self-evaluation. Yet the need for such reviews confronts us every day. As the recent U.S. Justice Department report on Ferguson has shown, the shooting of Michael Brown and the subsequent riots did not occur in a vacuum: systemic racism and revenue-oriented policing set the stage. Other cases, such as that of Eric Garner, who was suffocated by a New York police officer, also suggest the need for system-oriented analysis.

“You have to look at a whole range of questions,” Doyle told me. "What was the policy wisdom of criminalizing revenue offenses like selling loose cigarettes? Who made a command decision to clean up that corner? Did the officers get trained in deëscalation, and in the proper takedown techniques? Was there a better way to do this than the way it was done?”