Artificial intelligence has proven itself useful in reading medical imaging and even shown it can pass doctors’ licensing exams. Now, a new AI tool has demonstrated the ability to read physicians’ notes and accurately anticipate a person’s risk of death, readmission to the hospital and other outcomes important to their care.
Designed by a team at NYU Grossman School of Medicine, the software is currently in use at the university’s affiliated hospitals throughout New York, with the hope that it will become a standard part of health care. A study on its predictive value was published on 7th June in the journal Nature.
Lead author Eric Oermann, an NYU neurosurgeon and computer scientist, told AFP that while non-AI predictive models have been around in medicine for a long time, they were hardly used in practice because the data they needed requires cumbersome reorganisation and formatting.
But “one thing that’s common in medicine everywhere, is physicians writing notes about what they’ve seen in clinics and what they’ve discussed with patients,” he said. “So our basic insight was, can we start with medical notes as our source of data and then build predictive models on top of it?”
Read more:
IIT Madras creates AI tool to identify glioblastoma
Indian researchers develop AI tool to better understand the gut microbe
IIT Guwahati researchers develop AI to predict how thigh fractures heal
The large language model, called NYUTron, was trained on millions of clinical notes from the health records of 387,000 people who received care within NYU Langone hospitals between January 2011 and May 2020. These included any records written by doctors, such as patient progress notes, radiology reports and discharge instructions, resulting in a 4.1-billion-word corpus.
One of the key challenges for the software was interpreting the natural language that physicians write in, which varies greatly among individuals, including the abbreviations they choose. By looking back at records of what happened, researchers were able to calculate how often the software’s predictions turned out to be accurate. They also tested the tool in live environments, training it on the records from, for example, a hospital in Manhattan and then seeing how it fared in a Brooklyn hospital, with different demographics.
Not a substitute for humans
Overall, NYUTron identified an unnerving 95 per cent of people who died in the hospital before they were discharged and 80 per cent of patients who would be readmitted within 30 days. It outperformed most doctors on its predictions, as well as the non-AI computer models used today. But, to the team’s surprise, “the most senior physician who’s actually a very famous physician had superhuman performance, better than the model,” said Oermann. “The sweet spot for technology and medicine isn’t that it’s going to always deliver necessarily superhuman results, but it’s going to really bring up that baseline.”
NYUTron also correctly estimated 79 per cent of patients’ actual length of stay, 87 per cent of cases where patients were denied coverage by insurance, and 89 per cent of cases where a patient’s primary disease was accompanied by additional conditions. AI will never be a substitute for the physician-patient relationship, said Oermann. Rather, they will help “provide more information for physicians seamlessly at the point of care so they can make more informed decisions.”
