The scales of justice/Government of PEI.
I recently wrote a blog on the lack of transparency in e-voting as proposed for adoption in many Ontario municipalities. Traditionally, it is not enough to correctly identify election winners. The process must be open to inspection and challenge, thus legitimating the result.
In the New York Times, Adam Liptak raises a similar issue about criminal sentencing. Last year, Eric L. Loomis of Wisconsin was given a six-year prison sentence based, in part, on a risk assessment generated by a software system called Compas. The software considers information about the nature of a crime, the background of the accused, etc. and generates a report estimating how likely it is that the accused would re-offend. Judges may use this information when setting sentences.
Since Compas is proprietary, the rationale it uses to generate its assessments are not disclosed to the accused or even the judge. For all intents and purposes, it is a black box.
This situation raises at least two questions. First, are its assessments well-founded? A study made by ProPublica argued that Compas is biased against African Americans, assessing them as riskier than similar, white defendants. Northpointe, the provider of Compas, disagrees.
Second, is the use of a black box itself acceptable? Mr. Loomis argues that it is not:
Mr. Loomis says his right to due process was violated by a judge’s consideration of a report generated by the software’s secret algorithm, one Mr. Loomis was unable to inspect or challenge.
His complaint was rejected by the Wisconsin Supreme Court on the grounds that a transparent process would have arrived at the same result in his case. In other words, so long as the assessment is well-founded, its secrecy is of no import.
In defence of this position, Wisconsin’s attorney general added that Mr. Loomis "was free to question the assessment and explain its possible flaws." This ridiculous assertion could be straight from Kafka's The Trial: How could Mr. Loomis identify flaws in an assessment that he is not permitted to examine?
Whether we are outsourcing elections or risk assessments, the problem of transparency arises. Traditionally, transparency has been required to provide assurance that public functions are legitimate and justified. When these functions are outsourced to private companies that provide them using proprietary means, transparency is lost. Is that acceptable? Or, is the outcome all that matters?
The US Supreme Court is considering whether or not to hear an appeal.