Opinion: Data deluge needs human analytical filter

Friday 12 January 2018

New Zealand faces a critical shortage of skilled practitioners able to analyse, interpret and disseminate the wealth of data needed to inform and improve health services for New Zealanders.

Opinion: Data deluge needs human analytical filter - image1

"We continue to amass data at an increasing rate, but we are not turning this into intelligence," Professor Borman says.

Last updated: Thursday 4 August 2022

By Professor Barry Borman

New Zealand faces a critical shortage of skilled practitioners able to analyse, interpret and disseminate the wealth of data needed to inform and improve health services for New Zealanders. The current formula of gathering endless amounts of data, which is then not analysed and interpreted in a meaningful way, poses potential health risks to New Zealanders and needs to be addressed urgently.

We continue to amass data at an increasing rate, but we are not turning this into intelligence. Raw data remains at the bottom of the hierarchy even though we are often quick to invest in software and hardware. Organisations are seemingly reluctant to invest in analytic experts that can move raw data up to the apex of the hierarchy, to create actionable information to support decision making.

This is at odds with current developments. We are in an environment where the health sector is under increasing pressure to perform and we need to be making decisions and developing policy that is evidence-based. Internationally there is a clear demand for people with specific health analytics skills, and this is predicted to grow. There is a huge amount of data being collected, but a gap remains in the number of qualified people trained in how to analyse, interpret and translate the faceless figures into meaningful information that can be used to improve our health services and patient outcomes. While New Zealand is competing in the global space in this area, we need to generate homegrown capability to fill a real and societally relevant need.

Opinion: Data deluge needs human analytical filter - image2

Professor Barry Borman.

Humans make data useful – intelligence, not data, is power

Across the globe there is a common belief that faster computers and better software will result in more information, but analysts are best at making sense of, and understanding the implications and meanings of data. The machines can crunch the numbers, but they can’t translate them or assess them in a broader social, economic and policy context.

Two of the biggest data gathering enterprises – NASA’s space programme and the Large Hadron Collider – are the most technologically advanced pieces of machinery invented, but they are run by thousands of people. They require scientists to evaluate and interpret the results of data analysis, otherwise the technology itself is useless.

Our focus needs to be on the ‘so what?’ questions and in my view, the health sector will only progress when we start asking and answering those questions – ‘what does it mean?’ ‘How can we fix that?’ ‘What will the outcome be?’

The possibilities are truly endless. In Italy, researchers have been studying the placement of ambulances, and worked out if you place an ambulance strategically, they can reach accident sites quicker, getting patients medical care quicker. This led to a substantial reduction in death rates. Imagine the lives we could save here.

In the Netherlands, the medical data of all residents is centralised, meaning if they are in a car accident away from home, practitioners have access to their medical data, so know immediately what they are allergic to, and what medications they are on. Our data is held by each individual district health board, meaning it is more likely there will be a ‘trial and error’ approach on the side of the road if we are injured. Doesn’t sound ideal to me!

While I understand there are questions and concerns around the ethics of gathering data, in the healthcare sector, it could be the difference between life and death. Currently we are collecting data in the expectation it might be analysed by someone, sometime, somewhere. When it isn’t, we collect more data. We aren’t able to analyse the data currently available, because we simply don’t have the analysts.

New mindset needed

There also needs to be a mentality change. Managers seem at a loss as to what is capable of being done using data and the possibilities are tremendous. Doctors could better diagnose illnesses or predict future health outcomes for patients. DHBs could prioritise budgets. The government could get a better understanding of what New Zealand’s future looks like and what health care will be needed as the ageing population continues to grow. Education is key for the realisation of a mentality change. To provide students with the skills and tools to turn currently collected data into information, Massey University has collaborated with Statistics New Zealand to establish New Zealand’s first teaching laboratory with access to the Integrated Data Infrastructure (IDI), a large research database holding data on life events, such as education, income, benefits, migration, justice, and health. The data comes from government agencies, Statistics New Zealand surveys, and non-government organisations.

Students will be able to apply their new knowledge to generate robust evidence for policy development and decision making across the health sector. The research to come out of the lab will help answer questions about complex issues that affect New Zealanders across health, business and public policy.

This step towards better understanding data will help us prepare for the data tsunami heading our way, but we need bright, determined and invested people to take up the challenge and fill the information abyss as it continues to widen. Managers also need to update their analytical thinking to know the right questions to ask of their analysts and to understand the resulting information.

Professor Barry Borman is the Director of Massey University’s Environmental Health Indicators Programme, based in Wellington.