In lieu of the traditional Noether lecture, the Association for Women in Mathematics held a panel on equity, ethics, and bias in mathematics research. The event featured Loretta Cheeks, president and CEO of Strong Ties and DS Innovation, Kristian Lum of the University of Pennsylvania, Maria De-Arteaga of UT Austin, and Suresh Venkatasubramanian of the University of Utah.
Yesterday’s events at the Capitol Building brought misinformation and polarization to the top of my mind. So when an audience member asked about how mathematicians can effectively persuade policy-makers that an algorithm is unethical, Dr. Cheeks’ words about speaking to people with different perspectives felt timely and important. Her research explores the space between the computational sphere and the “informal space” of unstructured information–hints, subjective narratives, and the like. She noted that people are getting more and more of their information online, from indirect sources. This introduces an additional layer of bias beyond hearing, say, a friend’s personal story. Cheeks pointed out that once someone has very deeply held beliefs, their commitment to them goes beyond the familiar phenomena of confirmation bias and groupthink. It’s only by finding someone who can relate to their perspective that a connection can be made.
These communication problems can even arise from simple bids for algorithmic transparency. One participant asked how translating policy goals into something that can be used in an algorithm complicates things. Dr. De-Arteaga noted that this can cause extra friction between groups when their goals don’t align perfectly. Algorithms require highly specific and quantifiable inputs and objectives, while policy often benefits from ambiguity that glosses over disagreements between policymakers. Moreover, explaining algorithms can have unintended consequences: recent research suggests explanations may make users overconfident by giving them an unearned sense of understanding.
This highlights the importance of other fields in dealing with the points where mathematics meets society. A social scientist is likely to know what kinds of explanations cause overconfidence, and can analyze the content of data more effectively than a mathematician. What’s more, their scientific training involves thinking critically about assumptions. Dr. Venkatasubramanian believes, however, that mathematicians are still uniquely valuable in discussions of algorithmic ethics. The precision of mathematical thinking lends itself well to reasoning about systems’ capabilities and limitations.
Moreover, mathematics and data still holds special standing in the eyes of many. Dr. Venkatasubramanian and Dr. Lum both recalled the extra credibility they command in settings with lawyers and politicians. And when citizens face people in positions of power in meetings or in court, they can greatly increase their leverage by having data and analyses at their disposal, says Dr. Lum.
All the panelists agreed that it is far too early for mathematicians to be able to draw up a rigid, universal set of ethical guidelines like the AMA Code of Medical Ethics. But there are plenty of ethical and equity questions that modellers can ask themselves before they forge ahead with their work–about the bias in the data, about what assumptions are going into the model and the questions being answered, and about what, specifically, is being output and optimized for. As algorithms become more and more vital to the functioning of our society, mathematicians will have a special role to play that goes beyond the fallacy of math as truly objective.