May 17, 2020

Building trust in healthcare, AI and automated decision-making

Health IT
Health technology
Artificial intelligence
Ben Taylor, CEO at Rainbird an...
6 min
The AI juggernaut is moving forward at enormous pace. The worlds of finance, insurance, professional services and many, many more are turning to AI and...

The AI juggernaut is moving forward at enormous pace. The worlds of finance, insurance, professional services and many, many more are turning to AI and automated decision-making as a way to transform their business processes, to improve efficiency, reduce cost and ultimately make better decisions.   

One area where AI has the potential to achieve some highly beneficial advantages for everyone — especially patients and medical professionals, but also management and operations — is the healthcare sector.

In the UK a recent report by former Labour and Conservative health ministers Lord Darzi and Lord Prior for The Institute for Public Policy Research’s (IPPR) suggested that artificial intelligence (AI) and automation technologies could save the NHS £12.5bn (10% of its annual operational expenses). The report also claims a further £6bn could be saved by automating the repetitive decision making and admin tasks in social care.

ABI Research’s June 2018 report predicts a rapid growth in patient monitoring devices, which include AI for home-based preventative healthcare and predictive analytics, could save hospitals around $52bn by 2021.

Accenture’s Digital Health Technology Vision 2018 report goes so far as to claim that 85% of health executives in the US believe that every human will be directly impacted on a daily basis by an AI-based decision within the next three years.

Showcasing the possibilities

We’re already starting to see a growing number of early examples of AI algorithms and automated decision-making tools being trialled in the healthcare environment, here are some examples;  

On a clinical level Mount Sinai Hospital in New York recently announced a partnership with the AI healthcare start-up RenalytixAI to create an AI tool that identifies patients at the hospital who are at risk of advanced kidney disease. 

Baidu's AI researchers also claim to have built an algorithm that can spot cancerous tumours in breast tissue.

There are numerous news reports being published about Google’s new algorithm. Google claims it is able to quickly sift through thousands of digital documents in patients’ health records to find important information that can help doctors make better predictions about how long patients may stay in a hospital, or the likelihood of when they may die.

And at the aftercare/rehabilitation end of the scale my company has worked with a community healthcare service, Fluid Motion, that delivers group aquatic rehabilitation sessions in local swimming pools for the treatment, management and prevention of musculoskeletal conditions.

We worked closely with Fluid Motion’s qualified physiotherapists and osteopaths to encode their model of patient treatment into our platform, including the suitability of numerous exercises to individual patient symptoms. The resulting AI tool now allows them to create a tailored, illustrated rehabilitation plan for patients as a result of an interaction with a virtual physiotherapist or osteopath.

See also

Finally, at an administration level, tools like ZocDoc are showing the potential of AI to make it easier for patients and healthcare providers to digitally verify patients’ insurance information. In the US this administrative cost currently makes up about 15 percent of all healthcare expenditures and office managers spend nearly a quarter of their time dealing with insurance.

That’s just the tip of the iceberg.

But there’s a problem

Despite all the positive exuberance for AI in the healthcare sector, an interesting consumer survey by the Royal Society of Arts (RSA) and market research firm YouGov tells a very different story about the public’s general lack of trust in automated decision making. The survey found that in general only 32% of people are aware of automated decision-making systems. More specifically 74% lack any familiarity about the use of these systems to make decisions in healthcare and half (48%) said they were actively opposed to the use of the technology in healthcare.

That’s a lot of fear, scepticism and ‘ignorance’ from arguably the primary stakeholder in any healthcare service.

My interpretation is that much of this opposition stems from science fiction movies and mainstream media scaremongering, but the fact is that many AI and automated decision platforms forgo transparency and are consciously created as black box systems. The techniques used make it very hard to know why a decision or prediction was made.

How can you build public trust and garner support in systems that effectively remain free of legal scrutiny? The truth is that you can’t. While these technologies remain opaque, and operate outside of public view, confusion and distrust will continue.

While the majority of the public take no issue with mainstream consumer AI applications such as Netflix recommendations, auto-tagging on Facebook or suggestions from Siri, it is an entirely different matter when data is used to make healthcare decisions that could potentially be life-altering.

The House of Lords Select Committee on AI has already expressed its view that “…it is unacceptable to deploy any AI system that could have a substantial impact on an individuals’ life, unless it can generate a full and satisfactory explanation for the decisions it will take.”

Trust, safety and compliance in the healthcare AI domain will all rely on technology’s ability to explain the process it used to arrive at decisions. The RSA and YouGov’s research corroborates this from the perspective of the general public whose support for these systems increased by 36% if users could be granted the right to request an explanation of the organisational steps or processes undertaken to reach a decision with an AI system.

We also need to think about the impact of healthcare AI in a much wider context than just the patients alone. Doctors, nurses and NHS leadership also have to trust automated decision making if adoption of the technology is going to happen en-masse. If a consultant is using AI to make a decision on a terminal diagnosis, they need to believe the technology is trustworthy and dependable. If diagnoses are challenged there needs to be an audit trail showing how decisions were reached and the level of certainty the technology had when it recommended the decision.

We need more algorithms for automated decision-making that are modelled on human expertise – rather than black box data. At its heart AI is for collaboration with people. By building human/medical expertise into the development of healthcare AI platforms we can enable these systems to provide clear explanations for the decisions they make. And they can do this in a format that those same human experts understand and can confirm. Human-down AI is critical to the technologies future in healthcare and in building patients’ and healthcare professionals’ trust in automated decision making.

The current disconnect between how automated decision-making works and the public’s understanding of it cannot continue. Transparent, auditable and accountable platforms must be the norm rather than the exception.

If the public at large’s fear and distrust in healthcare AI isn’t properly addressed we risk wasting the enormous potential AI in healthcare has to benefit each and every one of us. We can’t let that happen.                              

Share article

May 13, 2021

Birdie aims to reinvent elderly care with tech

3 min
We take a look at homecare software startup Birdie, who are aiming to transform elderly care in the UK

British startup Birdie has announced it has raised £8.2 million to invest in innovation and scale up the business. 

The company's announcement is timely as it follows the criticism of the UK government over their lack of a plan for social care, despite acknowledging the sector is in crisis - around a quarter of the UK's home care providers are on the brink of bankruptcy due to a lack of funds and staffing. 

Birdie was born with a mission to  "radically improve the lives of millions of older adults", by using app-based solutions, IoT and machine learning to put preventative care at the forefront.  The company was founded by Max Parmentier,  after experiencing his own frustrations with the care system - his grandfather struggled with the impact of life in a care home, but lacked any other option. 

In 2017 Parmentier partnered with venture builder Kamet Ventures to  set up Birdie, in a bid to fix this problem. Since then, Birdie has partnered with almost 500 providers across the UK, and supports more than 20,000 older people every week. In the past 12 months alone the number of people Birdie supports has got six times greater. 

Birdie’s solution is an app to help care providers deliver more coordinated, personalised and preventative care, by giving them access to digital assessments, medication scheduling and planning tools. By using digital tools to take care of admin, staff have more time to spend with their care recipients. 

The new investment will be used to fund Birdie’s next phase of growth in the UK, as the company scales to meet the rapidly growing demand of the aging population. The company will also invest in product innovation, creating new features to address customer requests.

In addition, Birdie is piloting new care models, including partnering with the NHS to identify COVID-19 symptoms, building predictive pharmacy models with AI, and helping health authorities to detect early warning signs of patients’ health risks.

Internally, Birdie is committed to having a progressive company ethos. All salaries are transparent, and staff work asynchronously to maximise flexibility and equity. Staff members also volunteer in their local community during office hours, and the company offsets all its emissions.

These efforts have led to numerous awards, including having the best SME culture in the UK, an Honorable Mention in the Health category of Fast Company’s 2021 World Changing Ideas Awards, and innovation in care at the LangBuisson awards. 

“We believe the future of care for older people should be helping them to live at home for as long as possible through the delivery of personalised and preventative care" Parmentier said. 

"Birdie is already the partner of choice for caregivers up and down the UK, and this new funding will help us rapidly increase the number we partner with and what we can offer them - meaning more people benefiting from more affordable, quality care. We’re proud of our mission and the values we embody to pursue it.” 

Share article