When a Computer Program Keeps You in Jail
By REBECCA WEXLER
CreditSally Deng
The criminal justice system is becoming automated. At every stage — from policing and investigations to bail, evidence, sentencing and parole — computer systems play a role. Artificial intelligence deploys cops on the beat. Audio sensors generate gunshot alerts. Forensic analysts use probabilistic software programs to evaluate fingerprints, faces and DNA. Risk-assessment instruments help to determine who is incarcerated and for how long.
Technological advancement is, in theory, a welcome development. But in practice, aspects of automation are making the justice system less fair for criminal defendants.
The root of the problem is that automated criminal justice technologies are largely privately owned and sold for profit. The developers tend to view their technologies as trade secrets. As a result, they often refuse to disclose details about how their tools work, even to criminal defendants and their attorneys, even under a protective order, even in the controlled context of a criminal proceeding or parole hearing.
Take the case of Glenn Rodríguez. An inmate at the Eastern Correctional Facility in upstate New York, Mr. Rodríguez was denied parole last year despite having a nearly perfect record of rehabilitation. The reason? A high score from a computer system called Compas. The company that makes Compas considers the weighting of inputs to be proprietary information. That forced Mr. Rodríguez to rely on his own ingenuity to figure out what had gone wrong.Continue reading the main story
RECENT COMMENTS
Loomy 1 day ago
Only in America! If DNA matching finds the suspects DNA in /at a Murder scene /on a weapon etc etc then that is one piece of strong evidence...
Ian MacFarlane 1 day ago
Only in America are the inmates running the asylum.
L’Osservatore 1 day ago
We can't even get DNA samples available for the justice system with a year, can we? Is the federal gov't going to have to fund evaluating...
SEE ALL COMMENTS
This year, Mr. Rodríguez returned to the parole board with the same faulty Compas score. He had identified an error in one of the inputs for his Compas assessment. But without knowing the input weights, he was unable to explain the effect of this error, or persuade anyone to correct it. Instead of challenging the result, he was left to try to argue for parole despite the result.
Mr. Rodríguez was lucky. In the end, he made parole and left Eastern Correctional in mid-May. But had he been able to examine and contest the logic of the Compas system to prove that its score gave a distorted picture of his life, he might have gone home much earlier.
Or consider the case of Billy Ray Johnson, a defendant in California who was sentenced to life without parole for a series of burglaries and sexual assaults that he says he did not commit. The prosecution relied on the results of a software program called TrueAllele that was used to analyze traces of DNA from the crime scenes.
Newsletter Sign UpContinue reading the main story
Sign Up for the Opinion Today Newsletter
Every weekday, get thought-provoking commentary from Op-Ed columnists, the Times editorial board and contributing writers from around the world.
Sign Up
You agree to receive occasional updates and special offers for The New York Times's products and services.
SEE SAMPLE
PRIVACY POLICY
OPT OUT OR CONTACT US ANYTIME
When an expert witness for Mr. Johnson sought to review the TrueAllele source code in order to confront and cross-examine its programmer about how the software works, the developer claimed it was a trade secret, and the court refused to order the code disclosed — even though Mr. Johnson’s attorney offered to sign a protective order that would safeguard the code. Mr. Johnson was thus unable to fully challenge the evidence used to find him guilty.
TrueAllele’s developer maintains this decision was right. It has submitted affidavits to courts across the country alleging that disclosing the program’s source code to defense attorneys would cause “irreperable harm” to the company because it would allow competitors to steal the code. Most judges have credited this claim, quashing defense subpoenas for the source code and citing the company’s intellectual property interests as a rationale.
In 2015, a California Appeals Court upheld a trade secret evidentiary privilege in a criminal proceeding — for what is likely the first time in the nation’s history — to shield TrueAllele source code from disclosure to the defense. That decision, People v. Chubbs, is now being cited across the country to deny defendants access to trade secret evidence.
TrueAllele is not alone. In another case, an organization that produces cybercrime investigative software tried to invoke a trade secret evidentiary privilege to withhold its source code, despite concerns that the program violated the Fourth Amendment by surreptitiously scanning computer hard drives. In still other instances, developers of face recognition technology have refused to disclose the user manuals for their software programs, potentially obstructing defense experts’ ability to evaluate whether a program has been calibrated for certain racial groups and not for others.
Likewise, the algorithms used to generate probabilistic matches for latent fingerprint analysis, and to search ballistic information databases for firearm and cartridge matches, are treated as trade secrets and remain inaccessible to independent auditors.
This is a new and troubling feature of the criminal justice system. Property interests do not usually shield relevant evidence from the accused. And it’s not how trade secrets law is supposed to work, either. The most common explanation for why this form of intellectual property should exist is that people will be more likely to invest in new ideas if they can stop their business competitors from free riding on the results. The law is designed to stop business competitors from stealing confidential commercial information, not to justify withholding information from the defense in criminal proceedings.
Defense advocacy is a keystone of due process, not a business competition. And defense attorneys are officers of the court, not would-be thieves. In civil cases, trade secrets are often disclosed to opposing parties subject to a protective order. The same solution should work for those defending life or liberty.
The Supreme Court is currently considering hearing a case, Wisconsin v. Loomis, that raises similar issues. If it hears the case, the court will have the opportunity to rule on whether it violates due process to sentence someone based on a risk-assessment instrument whose workings are protected as a trade secret. If the court declines the case or rules that this is constitutional, legislatures should step in and pass laws limiting trade-secret safeguards in criminal proceedings to a protective order and nothing more.
The future of the criminal justice system may depend on it.
Rebecca Wexler is a fellow at the Legal Aid Society and a resident at the Data and Society Research Institute.
www.fotavgeia.blogspot.com
By REBECCA WEXLER
CreditSally Deng
The criminal justice system is becoming automated. At every stage — from policing and investigations to bail, evidence, sentencing and parole — computer systems play a role. Artificial intelligence deploys cops on the beat. Audio sensors generate gunshot alerts. Forensic analysts use probabilistic software programs to evaluate fingerprints, faces and DNA. Risk-assessment instruments help to determine who is incarcerated and for how long.
Technological advancement is, in theory, a welcome development. But in practice, aspects of automation are making the justice system less fair for criminal defendants.
The root of the problem is that automated criminal justice technologies are largely privately owned and sold for profit. The developers tend to view their technologies as trade secrets. As a result, they often refuse to disclose details about how their tools work, even to criminal defendants and their attorneys, even under a protective order, even in the controlled context of a criminal proceeding or parole hearing.
Take the case of Glenn Rodríguez. An inmate at the Eastern Correctional Facility in upstate New York, Mr. Rodríguez was denied parole last year despite having a nearly perfect record of rehabilitation. The reason? A high score from a computer system called Compas. The company that makes Compas considers the weighting of inputs to be proprietary information. That forced Mr. Rodríguez to rely on his own ingenuity to figure out what had gone wrong.Continue reading the main story
RECENT COMMENTS
Loomy 1 day ago
Only in America! If DNA matching finds the suspects DNA in /at a Murder scene /on a weapon etc etc then that is one piece of strong evidence...
Ian MacFarlane 1 day ago
Only in America are the inmates running the asylum.
L’Osservatore 1 day ago
We can't even get DNA samples available for the justice system with a year, can we? Is the federal gov't going to have to fund evaluating...
SEE ALL COMMENTS
This year, Mr. Rodríguez returned to the parole board with the same faulty Compas score. He had identified an error in one of the inputs for his Compas assessment. But without knowing the input weights, he was unable to explain the effect of this error, or persuade anyone to correct it. Instead of challenging the result, he was left to try to argue for parole despite the result.
Mr. Rodríguez was lucky. In the end, he made parole and left Eastern Correctional in mid-May. But had he been able to examine and contest the logic of the Compas system to prove that its score gave a distorted picture of his life, he might have gone home much earlier.
Or consider the case of Billy Ray Johnson, a defendant in California who was sentenced to life without parole for a series of burglaries and sexual assaults that he says he did not commit. The prosecution relied on the results of a software program called TrueAllele that was used to analyze traces of DNA from the crime scenes.
Newsletter Sign UpContinue reading the main story
Sign Up for the Opinion Today Newsletter
Every weekday, get thought-provoking commentary from Op-Ed columnists, the Times editorial board and contributing writers from around the world.
Sign Up
You agree to receive occasional updates and special offers for The New York Times's products and services.
SEE SAMPLE
PRIVACY POLICY
OPT OUT OR CONTACT US ANYTIME
When an expert witness for Mr. Johnson sought to review the TrueAllele source code in order to confront and cross-examine its programmer about how the software works, the developer claimed it was a trade secret, and the court refused to order the code disclosed — even though Mr. Johnson’s attorney offered to sign a protective order that would safeguard the code. Mr. Johnson was thus unable to fully challenge the evidence used to find him guilty.
TrueAllele’s developer maintains this decision was right. It has submitted affidavits to courts across the country alleging that disclosing the program’s source code to defense attorneys would cause “irreperable harm” to the company because it would allow competitors to steal the code. Most judges have credited this claim, quashing defense subpoenas for the source code and citing the company’s intellectual property interests as a rationale.
In 2015, a California Appeals Court upheld a trade secret evidentiary privilege in a criminal proceeding — for what is likely the first time in the nation’s history — to shield TrueAllele source code from disclosure to the defense. That decision, People v. Chubbs, is now being cited across the country to deny defendants access to trade secret evidence.
TrueAllele is not alone. In another case, an organization that produces cybercrime investigative software tried to invoke a trade secret evidentiary privilege to withhold its source code, despite concerns that the program violated the Fourth Amendment by surreptitiously scanning computer hard drives. In still other instances, developers of face recognition technology have refused to disclose the user manuals for their software programs, potentially obstructing defense experts’ ability to evaluate whether a program has been calibrated for certain racial groups and not for others.
Likewise, the algorithms used to generate probabilistic matches for latent fingerprint analysis, and to search ballistic information databases for firearm and cartridge matches, are treated as trade secrets and remain inaccessible to independent auditors.
This is a new and troubling feature of the criminal justice system. Property interests do not usually shield relevant evidence from the accused. And it’s not how trade secrets law is supposed to work, either. The most common explanation for why this form of intellectual property should exist is that people will be more likely to invest in new ideas if they can stop their business competitors from free riding on the results. The law is designed to stop business competitors from stealing confidential commercial information, not to justify withholding information from the defense in criminal proceedings.
Defense advocacy is a keystone of due process, not a business competition. And defense attorneys are officers of the court, not would-be thieves. In civil cases, trade secrets are often disclosed to opposing parties subject to a protective order. The same solution should work for those defending life or liberty.
The Supreme Court is currently considering hearing a case, Wisconsin v. Loomis, that raises similar issues. If it hears the case, the court will have the opportunity to rule on whether it violates due process to sentence someone based on a risk-assessment instrument whose workings are protected as a trade secret. If the court declines the case or rules that this is constitutional, legislatures should step in and pass laws limiting trade-secret safeguards in criminal proceedings to a protective order and nothing more.
The future of the criminal justice system may depend on it.
Rebecca Wexler is a fellow at the Legal Aid Society and a resident at the Data and Society Research Institute.
www.fotavgeia.blogspot.com
Δεν υπάρχουν σχόλια:
Δημοσίευση σχολίου