IBM Watson Lost MD Anderson, but Has Plenty of Momentum

An audit released last week by the University of Texas System Administration about the MD Anderson Cancer Center’s IBM Watson project was very critical and it confirmed that the system is no longer in use. IBM was paid $39 million and PwC drew in another $23 million in project support for a total project cost of about $62 million. Despite the large price tag, support for the Oncology Expert Advisor (OEA) system was ended in September. The report concluded:

As of September 2016, the system is not in clinical use and has not been piloted outside of MD Anderson. IBM “[ended] support for the OEA Pilot System and for the OEA Demo System effective September 1, 2016.” The IBM agreement currently in effect states that the system “is not ready for human investigational or clinical use, and its use in the treatment of patients is prohibited” except as needed to test and evaluate the system. Internal pilots for Leukemia and Lung OEA were conducted using the prior medical records system (ClinicStation); OEA has not been updated to integrate with the current system (Epic).

That last sentence may be the most important in the report. Apparently the system was making a positive impression on physicians according to a June 2015 Washington Post article. The piece cites comments by an MD Anderson assistant professor in leukemia, Koichi Takahashi: “Watson’s ability to synthesize a patient’s history is ‘amazing.’ He beats me.” Another physician, Candida Vitale, commented, “I was surprised. Even if you work all night, it would be impossible to be able to put this much information together like that.”

So in June, 2015 there were positive reviews of the project, but 15 months later it is shut down. In the intervening months a new electronic medical records (EMS) system from Epic was installed. Could there be a link here? Epic has faced criticism in the industry and all the way to the halls of U.S. Congress due to its lack of interoperability with other healthcare technologies. Epic has also come under fire from MD Anderson which suggested the Epic implementation resulted in $405 million cost due to higher expenses and lower patient revenue.

Procurement Practices, Not Performance Says IBM

IBM has not made this a point in its response to questions about the program. That may be in part for legal reasons and in part for business reasons. IBM works with Epic in other parts of its consulting business and also announced a program 18 months ago with Mayo Clinic and Epic to integrate with Watson. In addition, IBM points out that the report was largely critical of procurement practices and didn’t disparage the performance or utility of IBM Watson for cancer diagnosis. IBM’s official response concludes:

The report assessed procurement practices, not the value or functionality of the OEA system. IBM is proud of the early research work that was accomplished to apply Watson technology in the Oncology Expert Advisor program at MD Anderson Cancer Center. We created an oncology advisor for lung cancer and leukemia, and as reported in the state’s finding, Watson achieved 90% accuracy in lung cancer decision support.

This is a fair reading of the report. The report focuses primarily on lapses in procurement practices and the role of the original program director, Dr. Lydia Chin. Forbes reported that MD Anderson has issued an RFP to restart the program as is “not excluding companies that had previously worked with it from job, implying that it might choose to go with IBM to reboot the project.”

IBM Watson Shows Plenty of Healthcare Momentum Elsewhere

IBM has five healthcare announcements this morning at the HIMSS annual conference in Orlando and all involve Watson. The Watson Medical Imaging collaborative has expanded to 24 organizations. The Central New York Care Collaborative (CNYCC) will “connect more than 2,000 healthcare and community based providers…,” with Watson to decrease “the amount of avoidable emergency department visits and hospital readmissions by 25 percent.” Atrius Health will use Watson “to improve the physician-patient experience,” through shared decision making for 675,000 patients in eastern Massachusetts.

In addition, IBM made an announcement in October 2016 with Memorial Sloan Kettering Cancer Center, MIT and Harvard about using Watson with genomic tumor sequencing. IBM also announced an agreement earlier this month with Jupiter Medical Center to “adopt Watson for Oncology trained by Memorial Sloan Kettering.” The solution is scheduled to go live in March. It appears that IBM and Watson retain a good amount of confidence from the healthcare community despite any issues at MD Anderson.
Follow @bretkinsella

IBM Unveils Watson-Power Imaging Solutions for Healthcare Providers

Japanese Insurance Company Replaces Office Workers with IBM’s Watson

Videos: IBM Watson To Assist Cybersecurity Analysts Using QRadar SIEM