Agencies should be liable for expert bias

Scientific experts are prone to bias, overestimate their certainty and government systems are not good at adjusting to new science.

Those admonitions come from Supreme Court Justice Sonia Sotomayor in a statement addressing the Supreme Court’s refusal to hear the case of Charles C. McCrory v. Alabama. In the piece, Sotomayor asks what courts should do when faced with convictions “resting on science that has now been wholly discredited?”

The question offer lessons for how government and politicians should apply science and technology with humility and flexibility.

These could easily be applied to other areas of government, including environmental policy. Agencies often rely on so-called “experts” to offer an assessment on the political agenda of an agency. Often, the agency sets the rules and context for those “expert” opinions, increasing the risk of bias and reducing the likelihood agencies will accept contradictory science — or admit they were wrong.

Addressing changes in forensic science, Sotomayor wrote, “One in four people exonerated since 1989 were wrongfully convicted based on false or misleading forensic evidence introduced at their trials.” Just such an error is alleged in the McCrory case.

In 1985, Charles McCrory was convicted of the murder of his wife when a bite-mark expert testified that “the bite marks on the victim matched the appellant’s upper teeth.” Subsequently, that expert recanted his testimony and bite mark analysis has been found to be unreliable.

In spite of that, courts have refused to retry the case, nearly 40 years after he was first put in prison. Last year, he was again denied parole.

Sotomayor noted the areas of science that are more error prone are those that rely on the judgment of “experts” as opposed to laboratory science. She cited a 2009 report from the National Academy of Sciences that assessed the accuracy of forensic science.

Contrasting DNA evidence with the judgment of experts, the authors wrote, “In terms of scientific basis, the analytically based disciplines generally hold a notable edge over disciplines based on expert interpretation.” They added that, “The findings of forensic science experts are vulnerable to cognitive and contextual bias.”

All of these same things could be said about the judgment of experts in other areas of public policy. Their assessments are subject to change over time. They are prone to bias. And they can convey more certainty than is warranted.

Courts, government agencies and politicians have incentives to ignore the limitations of expert judgment. Courts and policy makers can’t always wait until the science is “settled.”

In another Supreme Court case from 1992, Daubert v. Merrell Dow Pharmaceuticals, Inc., the majority wrote, “there are important differences between the quest for truth in the courtroom and the quest for truth in the laboratory. Scientific conclusions are subject to perpetual revision. Law, on the other hand, must resolve disputes finally and quickly.”

To deal with these various pressures and uncertainties, government systems should take into account the fact that scientific understanding will improve over time and may change entirely. Justice Sotomayor added that “science evolves slowly rather than in conclusive bursts,” making it important that court procedures allow for new information to be considered long after a sentence has been handed down.

This is particularly true when dealing with the subjective judgements of “experts.”

Although the court turned down the case, Sotomayor calls on Legislatures to address the problem, allowing courts to consider new scientific understanding.

Policy makers and courts should also take steps to reduce the impact of bias among experts in all fields.

These problems are not unique to the justice system. Independent oversight is an important part of holding government agencies accountable for their decisions and ensuring they are responding to science and data, not ideology or political pressure.

Justice Sotomayor did a service in raising this issue, not only to provide justice for those wrongly accused, but for policy makers everywhere looking to make government policy more accurate and accountable.

— Todd Myers is the vice president of research for the Washington Policy Center. Email him at [email protected].