May 17, 2020

Building trust in healthcare, AI and automated decision-making

Health IT
Health technology
Artificial intelligence
Ben Taylor, CEO at Rainbird an...
6 min
The AI juggernaut is moving forward at enormous pace. The worlds of finance, insurance, professional services and many, many more are turning to AI and...

The AI juggernaut is moving forward at enormous pace. The worlds of finance, insurance, professional services and many, many more are turning to AI and automated decision-making as a way to transform their business processes, to improve efficiency, reduce cost and ultimately make better decisions.   

One area where AI has the potential to achieve some highly beneficial advantages for everyone — especially patients and medical professionals, but also management and operations — is the healthcare sector.

In the UK a recent report by former Labour and Conservative health ministers Lord Darzi and Lord Prior for The Institute for Public Policy Research’s (IPPR) suggested that artificial intelligence (AI) and automation technologies could save the NHS £12.5bn (10% of its annual operational expenses). The report also claims a further £6bn could be saved by automating the repetitive decision making and admin tasks in social care.

ABI Research’s June 2018 report predicts a rapid growth in patient monitoring devices, which include AI for home-based preventative healthcare and predictive analytics, could save hospitals around $52bn by 2021.

Accenture’s Digital Health Technology Vision 2018 report goes so far as to claim that 85% of health executives in the US believe that every human will be directly impacted on a daily basis by an AI-based decision within the next three years.

Showcasing the possibilities

We’re already starting to see a growing number of early examples of AI algorithms and automated decision-making tools being trialled in the healthcare environment, here are some examples;  

On a clinical level Mount Sinai Hospital in New York recently announced a partnership with the AI healthcare start-up RenalytixAI to create an AI tool that identifies patients at the hospital who are at risk of advanced kidney disease. 

Baidu's AI researchers also claim to have built an algorithm that can spot cancerous tumours in breast tissue.

There are numerous news reports being published about Google’s new algorithm. Google claims it is able to quickly sift through thousands of digital documents in patients’ health records to find important information that can help doctors make better predictions about how long patients may stay in a hospital, or the likelihood of when they may die.

And at the aftercare/rehabilitation end of the scale my company has worked with a community healthcare service, Fluid Motion, that delivers group aquatic rehabilitation sessions in local swimming pools for the treatment, management and prevention of musculoskeletal conditions.

We worked closely with Fluid Motion’s qualified physiotherapists and osteopaths to encode their model of patient treatment into our platform, including the suitability of numerous exercises to individual patient symptoms. The resulting AI tool now allows them to create a tailored, illustrated rehabilitation plan for patients as a result of an interaction with a virtual physiotherapist or osteopath.

See also

Finally, at an administration level, tools like ZocDoc are showing the potential of AI to make it easier for patients and healthcare providers to digitally verify patients’ insurance information. In the US this administrative cost currently makes up about 15 percent of all healthcare expenditures and office managers spend nearly a quarter of their time dealing with insurance.

That’s just the tip of the iceberg.

But there’s a problem

Despite all the positive exuberance for AI in the healthcare sector, an interesting consumer survey by the Royal Society of Arts (RSA) and market research firm YouGov tells a very different story about the public’s general lack of trust in automated decision making. The survey found that in general only 32% of people are aware of automated decision-making systems. More specifically 74% lack any familiarity about the use of these systems to make decisions in healthcare and half (48%) said they were actively opposed to the use of the technology in healthcare.

That’s a lot of fear, scepticism and ‘ignorance’ from arguably the primary stakeholder in any healthcare service.

My interpretation is that much of this opposition stems from science fiction movies and mainstream media scaremongering, but the fact is that many AI and automated decision platforms forgo transparency and are consciously created as black box systems. The techniques used make it very hard to know why a decision or prediction was made.

How can you build public trust and garner support in systems that effectively remain free of legal scrutiny? The truth is that you can’t. While these technologies remain opaque, and operate outside of public view, confusion and distrust will continue.

While the majority of the public take no issue with mainstream consumer AI applications such as Netflix recommendations, auto-tagging on Facebook or suggestions from Siri, it is an entirely different matter when data is used to make healthcare decisions that could potentially be life-altering.

The House of Lords Select Committee on AI has already expressed its view that “…it is unacceptable to deploy any AI system that could have a substantial impact on an individuals’ life, unless it can generate a full and satisfactory explanation for the decisions it will take.”

Trust, safety and compliance in the healthcare AI domain will all rely on technology’s ability to explain the process it used to arrive at decisions. The RSA and YouGov’s research corroborates this from the perspective of the general public whose support for these systems increased by 36% if users could be granted the right to request an explanation of the organisational steps or processes undertaken to reach a decision with an AI system.

We also need to think about the impact of healthcare AI in a much wider context than just the patients alone. Doctors, nurses and NHS leadership also have to trust automated decision making if adoption of the technology is going to happen en-masse. If a consultant is using AI to make a decision on a terminal diagnosis, they need to believe the technology is trustworthy and dependable. If diagnoses are challenged there needs to be an audit trail showing how decisions were reached and the level of certainty the technology had when it recommended the decision.

We need more algorithms for automated decision-making that are modelled on human expertise – rather than black box data. At its heart AI is for collaboration with people. By building human/medical expertise into the development of healthcare AI platforms we can enable these systems to provide clear explanations for the decisions they make. And they can do this in a format that those same human experts understand and can confirm. Human-down AI is critical to the technologies future in healthcare and in building patients’ and healthcare professionals’ trust in automated decision making.

The current disconnect between how automated decision-making works and the public’s understanding of it cannot continue. Transparent, auditable and accountable platforms must be the norm rather than the exception.

If the public at large’s fear and distrust in healthcare AI isn’t properly addressed we risk wasting the enormous potential AI in healthcare has to benefit each and every one of us. We can’t let that happen.                              

Share article

Jun 18, 2021

Skin Analytics wins NHSX award for AI skin cancer tool 

2 min
Skin Analytics uses AI to detect skin cancer and will be deployed across the NHS to ease patient backlogs

An artificial intelligence-driven tool that identifies skin cancers has received an award from NHSX, the NHS England and Department of Health and Social Care's initiative to bring technology into the UK's national health system. 

NHSX has granted the Artificial Intelligence in Health and Care Award to DERM, an AI solution that can identify 11 types of skin lesion. 

Developed by Skin Analytics, DERM analyses images of skin lesions using algorithms. Within primary care, Skin Analytics will be used as an additional tool to help doctors with their decision making. 

In secondary care, it enables AI telehealth hubs to support dermatologists with triage, directing patients to the right next step. This will help speed up diagnosis, and patients with benign skin lesions can be identified earlier, redirecting them away from dermatology departments that are at full capacity due to the COVID-19 backlog. 

Cancer Research has called the impact of the pandemic on cancer services "devastating", with a 42% drop in the number of people starting cancer treatment after screening. 

DERM is already in use at University Hospitals Birmingham and Mid and South Essex Health & Care Partnership, where it has led to a significant reduction in unnecessary referrals to hospital.

Now NHSX have granted it the Phase 4 AI in Health and Care Award, making DERM available to clinicians across the country. Overall this award makes £140 million available over four years to accelerate the use of artificial intelligence technologies which meet the aims of the NHS Long Term Plan.

Dr Lucy Thomas, Consultant Dermatologist at Chelsea & Westminster Hospital, said: “Skin Analytics’ receipt of this award is great news for the NHS and dermatology departments. It will allow us to gather real-world data to demonstrate the benefits of AI on patient pathways and workforce challenges. 

"Like many services, dermatology has severe backlogs due to the COVID-19 pandemic. This award couldn't have come at a better time to aid recovery and give us more time with the patients most in need of our help.”

Share article