MENU

Google microscope combines AI, AR for real-time cancer detection

Google microscope combines AI, AR for real-time cancer detection

Technology News |
By Rich Pell


The augmented reality (AR) microscope platform consists of a modified light microscope that enables real-time image analysis and presentation of the results of machine learning algorithms directly into the field of view. Google researchers say they believe that the microscope can possibly help “accelerate and democratize” the adoption of deep learning tools to help make pathologists more efficient and save lives around the world.

Currently, direct viewing of tissue samples using a standard compound optical (or “light”) microscope is the predominant means by which pathologists diagnose illness. A critical barrier to the widespread adoption of deep learning in pathology, says Google, is the dependence on having a digital representation of the microscopic tissue.

The augmented reality microscope (ARM) developed by Google researchers aims to change that. It can be retrofitted into existing light microscopes found in hospitals and clinics around the world using low-cost, readily-available components, and without the need for whole slide digital versions of the tissue being analyzed.

“Modern computational components and deep learning models, such as those built upon TensorFlow, will allow a wide range of pre-trained models to run on this platform,” say the researchers. “As in a traditional analog microscope, the user views the sample through the eyepiece. A machine learning algorithm projects its output back into the optical path in real-time. This digital projection is visually superimposed on the original (analog) image of the specimen to assist the viewer in localizing or quantifying features of interest.”

Importantly, the researchers note, the computation and visual feedback update quickly. The current implementation runs at approximately 10 frames per second, resulting in the model output updating seamlessly as the user scans the tissue by moving the slide or changing magnification. In addition, the researchers say, the ARM should be able to offer a range of visual feedback options – such as text, arrows, contours, heatmaps, or animations – and is capable of running many types of machine learning algorithms aimed at solving different problems such as object detection, quantification, or classification.

To demonstrate the ARM’s potential, the researchers configured it to run two different cancer detection algorithms: one for breast cancer, and the other for prostate cancer. The models can run at magnifications between 4-40x, and the result of a given model is displayed by outlining detected tumor regions with a green contour (image) to help draw the pathologist’s attention to areas of interest without obscuring the underlying tumor cell appearance.

According to the researchers, the models – which were originally trained on images from a whole slide scanner with a significantly different optical configuration – performed “remarkably well” on the ARM with no additional re-training. They expect that additional training on digital images captured directly from the ARM itself would further improve the models’ performance.

“We believe that the ARM has potential for a large impact on global health, particularly for the diagnosis of infectious diseases, including tuberculosis and malaria, in developing countries,” the researchers say. “Furthermore, even in hospitals that will adopt a digital pathology workflow in the near future, ARM could be used in combination with the digital workflow where scanners still face major challenges or where rapid turnaround is required. Of course, light microscopes have proven useful in many industries other than pathology, and we believe the ARM can be adapted for a broad range of applications across healthcare, life sciences research, and material science.”

For more, see “An Augmented Reality Microscope for Real­time Automated Detection of Cancer.”

Google Research

Related articles:
Deep learning improves medical imaging
AI speeds up precision medicine, says IBM Watson study
Nvidia supercomputing platform to ‘revolutionize’ medical imaging
GE, Nvidia partner on next-gen intelligent healthcare


Share:

Linked Articles
10s