December 3, 2018 – People may think that an automated artificial intelligence (AI) system for pathology that achieves a performance of 90 percent outperforms pathologists. However, pathologists are using microscopes for a task that includes complex computations – something humans are not good at, and where humans would greatly benefit from the use of computers. So, are we making the right comparison?
In manual microscopy, the standard of care and current practice in pathology, a pathologist “takes a look” at a histology slide under a microscope. There are many issues with the microscopy alone. For instance, microscopes are not FDA-cleared medical devices (they pre-date the FDA), different microscopes have different optics and even light sources (e.g. blue vs. orange light), and the microscopes in use are often not properly calibrated. We are now asking the pathologists to assess anywhere in the ball park of half a million to more than a million cells that can have considerable heterogeneity across the slide, and reduce all that information to a very simplified diagnosis or summary score. To give you an example, in the case of a simplistic IHC score, we ask the pathologist to determine the percentage of cells (to be evaluated against a threshold, e.g. > 10 percent) of a certain cell type (e.g. tumor cells) that have a staining (e.g. DAB) in a certain cell compartment (e.g. nucleus) that is above an absolute threshold. This is an incredibly challenging computational task that obviously will lead to high inter- and intra-pathologist variability.
To make things even more challenging, immuno-oncology (IO), a major advancement in drug development, requires pathologists to deal with more than one stain, consider the tissue context and apply far more complex scoring schemes. This is becoming an impossible task with just a microscope.
Obviously, we are asking for a lot of things that humans have a hard time doing, but a computer can do without a fault. With the adoption of digital pathology, which enables the analysis of images of histology sides by a computer, it is time to replace the microscope with the right tools!
Rather than going “all in” with an automated AI system with the intent of replacing pathologists, we should use AI as an aid to pathologists, allowing them to provide high-performance high-complexity tissue analysis, and in turn providing the best personalized healthcare to the patients.
The right tool for a pathologist is a pathology AI system that uses machine learning for cell classification, where the pathologist will bring in their tissue expertise (identifying different cell types and verifying proper cell classification), and allow the computer to provide the computational tasks of counting the cells, calculating objective measurements and providing complex scoring results to help oncologists.
A critical and often underestimated problem with automation in medicine is that medical doctors at large are doing much more than just visual pattern recognition. Probably the most important skill of medical doctors is their ability to integrate the data they get or what they see with the clinical information about the patients. Replacing a pathologist’s visual pattern recognition task with well-trained deep learning networks only gets you so far.
Automated AI systems will have a hard time to increase their performance to 95 percent or even 99 percent, which might be the requirement to match the performance of a pathologist using the right tools.
The key problem for automated AI systems for pathology are the variations that exists within the same tumor type between different patients. In a disease state, no two patient samples look identical. Any AI system has to learn to distinguish different cell types (even if it is hidden in some obscure features in a deep learning network). The challenge is that the same cell type has different characteristics in different patients, which are often contradictory, something pathologists are well trained to recognize. This increases the complexity of the cell classification problem and requires a lot of training data to ensure that all patient types are included and that there is no bias from the training data. Getting an automated AI system’s performance from 90 percent performance to 95 percent or even 99 percent becomes exponentially harder as remaining exceptional cases become harder to find.
The best way, and in our opinion the only viable way at a reasonable price point, to create enough training data is to automatically generate this data as part of the standard clinical workflow.
A perfect intermediate step between manual microscopy and automated AI would be to provide pathologists with the right tools: the pathology AI system described above. This system would generate an abundance of free training data (pathologist-verified cell classification for the whole slide for all slides) as part of a normal workflow, which could be used to train an automated AI system. As the performance increases with more training data, it will become obvious when it is the right time to commercialize an automated AI system.
Let’s provide pathologists with the right tools: replace the microscope, not the pathologists!
Do you struggle with inter- and intra-pathologist variations? Would you like pathologists to be able to provide high-performance high-complexity tissue analysis? How do you get the data to train your AI system? When do you think AI will replace pathologists?
Flagship Biosciences have been developing our own pathology AI system over the last 8 years to solve the most challenging real-world tissue image analysis problems across the entire pharmaceutical industry. Our pathology AI system provides our pathologists with the right tools: replacing the microscope, not the pathologists! Pathologists are at the center of our pathology AI system, where they provide their expertise and provide high-performance, high-complexity tissue analysis. Our workflow automatically generates free training data sets for automated AI systems. With those data sets we are well positioned to commercialize AI systems for pathology.
If you want to learn more about the technology behind our pathology AI system, check out our previous LinkedIn articles, where we presented a short lecture series about the different aspects about pathology AI intended for a broad non-technical audience.
Holger Lange, PhD
Chief Technology Officer
Flagship Biosciences