Article 2 – How Radiology and AI Will Come Together

An efficient, accurate resident with an experienced attending is a force to be reckoned with. The team can crush the list of patients and take trainwreck cases in stride. The resident can pull up labs and history as needed, and they can get the right person on the phone at the right time and handle all but the most complicated clinical scenarios. It takes years of training to achieve this.

Let’s take a look at Dr. Keith Dreyer’s outline of what clinical diagnostics looks like.

Start with the patient in the lower left and move left to right.











Now, this clinical workflow outline is very general, but we can trace the steps of the patient entering the healthcare system to the provider ordering imaging to the radiologist doing what he/she does best. It provides a good overview of the steps taken within a patient’s diagnostic path. The next image will provide more context in terms of radiology.









The clinical pathway from above is narrowed to follow a patient with acute neurological symptoms. From left to right: symptom, imaging, image interpretation and clinical management.

From the above slide, we can categorize the vast array of functions that a radiologist is a part of on a day to day basis, multiple times a day. Specific points in the process involve the radiologist’s knowledge with some focus on teamwork throughout. Managing the care for one patient takes a whole team, interrelating factors, amidst a complex working environment.

Moreover, as we all know, there can be points in the workflow that are tedious to complete and that could be automated. This could improve the radiologist’s workflow, and potentially that improved efficiency will translate into improved patient outcomes and real dollars saved.

This is where artificial intelligence can come into play. In the above slide, the AI algorithms come into the Detect stage. Generally speaking, this is where most of machine learning is currently being focused.  However, in the future, we can imagine a world where most of these steps can be augmented or assisted by a smart algorithm to improve workflow efficiency. As you move further along this series of articles, this combination of machine and human interactions will be more clear.


Radiology has tended to be at the forefront of technology. “Medical imaging has learnt itself well into modern medicine as a hallmark in the last 30 years and revolutionized health care delivery and medical industry.” (1) The MRI and PACS systems have revolutionized how imaging is delivered, consumed, and interpreted. These have paved the way for the general notion that radiology should be at the head of the pack.

Innovations in imaging can not only equip the radiologist to better understand an underlying disease process, but can also translate into improved patient outcomes.


So, where does AI fit in?

One of the first applications is in cardiac MRI. Typically, radiologists (fellows/residents) complete the very tedious task of drawing outlines of the cardiac outflow tracts. This could take hours. Now, in theory, this can be completely automated. The potential algorithm can, like Facebook, automatically detect the aortic root and draw it’s outline. Repeat for every needed outline. Once the outlines are completed, the calculations for flow, stenosis and regurgitation could also be automated.

From an attending physician’s perspective, if your 4th year resident consistently underestimated the aortic root circumference, you would encourage them to improve on this specific task using your educational style and technique. Which, if you remember from residency, varied from staff to staff.

Ideally, algorithms will be better than residents in this scenario. As you manipulate the images to correct any errors, the algorithm will automatically learn what is accepted and what is wrong. Now think of this on a scaled version within an institution or even a cluster of institutions. It should be nearly transparent to you. It will have an intuitive system, so when you make a change it makes small course corrections and continually improves.

This is just one scenario among many that will be investigated as intelligence is embedded within the clinical workflow.


“Extending human vision into the very nature of disease, medical imaging is enabling new and more powerful generation of diagnosis and intervention” (1).

The radiology community is part of a unique subset of physicians – we tend to be early adopters of technology. As such, we are primed to be on the cutting edge of what is being developed as tools that will pervade our workspace. No – we do not need to know how to physically program a neural network; we can leave that to the Kaggle developers and others mentioned in the previous article. But it will be very helpful to understand the basics, underlying motivation, and implications of machine learning.

Next week:

Article 3 – Machine Learning and Artificial Intelligence Introduction



  1. W. Gao, X. (2011). The State of the Art of Medical Imaging Technology: from Creation to   Archive and Back. The Open Medical Informatics Journal, 5(1), 73-85. doi:10.2174/1874431101105010073
  2. Dreyer, K Deep Learning, Clinical Data Science and Radiology