Workshopping on the Comfort Board. From left: Chair of the Digital Council Mitchell Pham, Andrew Tobin, Ana Reade, Tim Parkin, Anna Pendergrast, Anna Brown, Marianne Elliott, Elena Higgison, Victoria Wray, Colin Gavaghan, Tom Barraclough and Curtis Barnes.
New Zealanders are worried about bias in automated decision-making, according to research conducted by Massey University research group Toi Āria: Design for Public Good.
A new report Towards trustworthy and trusted automated decision-making in Aotearoa released by the Digital Council for Aotearoa aimed to answer the question: What is needed to ensure people in Aotearoa New Zealand have the right levels of trust required to harness the full societal benefits of digital and data-driven technologies?
Toi Āria, a research group within Toi Rauwhārangi College of Creative Arts, ran a series of in-person and online workshops with 186 people during 2020. Workshop participants not only shared their insights about how automated decision-making should be used now, but also their vision for building a better digital future that centres on the needs and aspirations of people.
Director of Toi Āria Associate Professor Anna Brown says the findings identified several areas of concern and what was needed to provide reassurance to New Zealanders. “A key concern across the workshops was around bias and discrimination, and for these to be allayed people needed more transparency and better communication, as well as better involvement and representation.
“People appreciated that automated decision-making is useful for processing data at speed and at scale, and as an ‘assistant’ to people, but people worry that systems, programmers and decision-makers can introduce bias into these,” Professor Brown says.
The process used Toi Āria’s Comfort Board methodology which is based on the principle that the people destined to use or be affected by a system should play a critical role in designing it. Workshop attendees included blind and vision impaired people; ethnic community leaders and youth; Māori and Pacific youth; women with migrant and refugee backgrounds; Whānau Ora navigators (Māori health advocates); young people with care experience; and members of the general public.
With the Digital Council, Toi Āria developed six scenarios based on real-world situations where some aspects of the decision-making processes are carried out or informed by computer algorithms, including: media consumption; recruitment; youth support; immigration; parole decisions; and surgical waiting lists.
As a result of the research, the Digital Council for Aotearoa has provided seven recommendations to Government. For more information about the findings please read the report here and visit Toi Āria's website here.