Tissue biopsy slides stained using hematoxylin and eosin (H&E) dyes are a cornerstone of histopathology, especially for pathologists needing to diagnose and determine the stage of cancers. A research team led by MIT scientists at the Media Lab, in collaboration with clinicians at Stanford University School of Medicine and Harvard Medical School, now shows that digital scans of these biopsy slides can be stained computationally, using deep learning algorithms trained on data from physically dyed slides.
Pathologists who examined the computationally stained H&E slide images in a blind study could not tell them apart from traditionally stained slides while using them to accurately identify and grade prostate cancers. What’s more, the slides could also be computationally “de-stained” in a way that resets them to an original state for use in future studies, the researchers conclude in their May 20 study published in JAMA Network.
This process of computational digital staining and de-staining preserves small amounts of tissue biopsied from cancer patients and allows researchers and clinicians to analyze slides for multiple kinds of diagnostic and prognostic tests, without needing to extract additional tissue sections.
“Our development of a de-staining tool may allow us to vastly expand our capacity to perform research on millions of archived slides with known clinical outcome data,” says Alarice Lowe, an associate professor of pathology and director of the Circulating Tumor Cell Lab at Stanford University, who was a co-author on the paper. “The possibilities of applying this work and rigorously validating the findings are really limitless.”
The researchers also analyzed the steps by which the deep learning neural networks stained the slides, which is key for clinical translation of these deep learning systems, says Pratik Shah, MIT principal research scientist and the study’s senior author.
“The problem is tissue, the solution is an algorithm, but we also need ratification of the results generated by these learning systems,” he says. “This provides explanation and validation of randomized clinical trials of deep learning models and their findings for clinical applications.”
Other MIT contributors are joint first author and technical associate Aman Rana (now at Amazon) and MIT postdoc Akram Bayat in Shah’s lab. Pathologists at Harvard Medical School, Brigham and Women’s Hospital, Boston University School of Medicine, and Veterans Affairs Boston Healthcare provided clinical validation of the findings.
Creating “sibling” slides
To create computationally dyed slides, Shah and colleagues have been training deep neural networks, which learn by comparing digital image pairs of biopsy slides before and after H&E staining. It’s a task well-suited for neural networks, Shah said, “since they are quite powerful at learning a distribution and mapping of data in a manner that humans cannot learn well.”
Shah calls the pairs “siblings,” noting that the process trains the network by showing them thousands of sibling pairs. After training, he said, the network only needs the “low-cost, and widely available easy-to-manage sibling,”— non-stained biopsy images—to generate new computationally H&E stained images, or the reverse where an H&E dye stained image is virtually de-stained.
In the current study, the researchers trained the network using 87,000 image patches (small sections of the entire digital images) scanned from biopsied prostate tissue from 38 men treated at Brigham and Women’s Hospital between 2014 and 2017. The tissues and the patients’ electronic health records were de-identified as part of the study.
When Shah and colleagues compared regular dye-stained and computationally stained images pixel by pixel, they found that the neural networks performed accurate virtual H&E staining, creating images that were 90-96 percent similar to the dyed versions. The deep learning algorithms could also reverse the process, de-staining computationally colored slides back to their original state with a similar degree of accuracy.
“This work has shown that computer algorithms are able to reliably take unstained tissue and perform histochemical staining using H&E,” says Lowe, who said the process also “lays the groundwork” for using other stains and analytical methods that pathologists use regularly.
Computationally stained slides could help automate the time-consuming process of slide staining, but Shah said the ability to de-stain and preserve images for future use is the real advantage of the deep learning techniques. “We’re not really just solving a staining problem, we’re also solving a save-the-tissue problem,” he said.
Software as a medical device
As part of the study, four board-certified and trained expert pathologists labeled 13 sets of computationally stained and traditionally stained slides to identify and grade potential tumors. In the first round, two randomly selected pathologists were provided computationally stained images while H&E dye-stained images were given to the other two pathologists. After a period of four weeks, the image sets were swapped between the pathologists, and another round of annotations were conducted. There was a 95 percent overlap in the annotations made by the pathologists on the two sets of slides. “Human readers could not tell them apart,” says Shah.
The pathologists’ assessments from the computationally stained slides also agreed with majority of the initial clinical diagnoses included in the patient’s electronic health records. In two cases, the computationally stained images overturned the original diagnoses, the researchers found.
“The fact that diagnoses with higher accuracy were able to be rendered on digitally stained images speaks to the high fidelity of the image quality,” Lowe says.
Another important part of the study involved using novel methods to visualize and explain how the neural networks assembled computationally stained and de-stained images. This was done by creating a pixel-by-pixel visualization and explanation of the process using activation maps of neural network models corresponding to tumors and other features used by clinicians for differential diagnoses.
This type of analysis helps to create a verification process that is needed when evaluating “software as a medical device,” says Shah, who is working with the U.S. Food and Drug Administration on ways to regulate and translate computational medicine for clinical applications.
“The question has been, how do we get this technology out to clinical settings for maximizing benefit to patients and physicians?” Shah says. “The process of getting this technology out involves all these steps: high quality data, computer science, model explanation and benchmarking performance, image visualization, and collaborating with clinicians for multiple rounds of evaluations.”
The JAMA Network study was supported by the Media Lab Consortium and the Department of Pathology at Brigham and Women’s Hospital.
Original post: http://news.mit.edu/2020/deep-learning-provides-accurate-staining-digital-biopsy-slides-0522
I like the helpful info you provide in your articles.
I’ll bookmark your weblog and check again here regularly.
I’m quite certain I will learn many new stuff right here!
Best of luck for the next!
Its not my first time to pay a quick visit this site, i
am visiting this web site dailly and obtain pleasant information from here everyday.
I’m the business owner of JustCBD Store brand (justcbdstore.com) and am seeking to expand my wholesale side of business. I am hoping anybody at targetdomain is able to provide some guidance ! I thought that the most ideal way to do this would be to talk to vape shops and cbd stores. I was really hoping if anybody at all could recommend a reliable web-site where I can get Vape Shop Business Marketing Data I am already examining creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the most suitable option and would appreciate any assistance on this. Or would it be easier for me to scrape my own leads? Suggestions?
I’m the manager of JustCBD Store label (justcbdstore.com) and am seeking to expand my wholesale side of company. It would be great if anybody at targetdomain can help me 🙂 I thought that the very best way to accomplish this would be to connect to vape stores and cbd retailers. I was hoping if anybody could suggest a dependable web site where I can buy CBD Shops Business Marketing Data I am already examining creativebeartech.com, theeliquidboutique.co.uk and wowitloveithaveit.com. On the fence which one would be the best choice and would appreciate any support on this. Or would it be simpler for me to scrape my own leads? Ideas?
An impressive share! I have just forwarded this onto a colleague who has been doing a little research on this. And he in fact bought me dinner simply because I discovered it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanks for spending time to discuss this subject here on your site.
bookmarked!!, I love your web site!
Good day! I just would like to give you a huge thumbs up for the excellent info you’ve got right here on this post. I will be returning to your site for more soon.
I truly love your blog.. Great colors & theme. Did you build this amazing site yourself? Please reply back as I’m trying to create my very own site and would love to know where you got this from or exactly what the theme is called. Kudos!
You ought to be a part of a contest for one of the most useful blogs on the web. I am going to highly recommend this blog!
Greetings! Very helpful advice in this particular article! It is the little changes that make the largest changes. Many thanks for sharing!
Oh my goodness! Incredible article dude! Many thanks, However I am going through troubles with your RSS. I don’t understand why I can’t subscribe to it. Is there anyone else getting the same RSS issues? Anyone that knows the solution can you kindly respond? Thanks!!
Pretty! This has been an incredibly wonderful post. Thank you for providing this info.
This site truly has all of the information I wanted concerning this subject and didn’t know who to ask.
This website certainly has all of the info I needed about this subject and didn’t know who to ask.
Having read this I thought it was extremely enlightening. I appreciate you finding the time and effort to put this informative article together. I once again find myself personally spending a significant amount of time both reading and posting comments. But so what, it was still worth it!
Achieving your fitness goal does not have to require a certified personal trainer or an expensive gym membership, it is not hard to exercise at home. It is easy to go down a training and fitness rabbit hole, however, when you are looking for the best home exercise equipment to outfit your personal home gym.
I must thank you for the efforts you have put in penning this site. I’m hoping to check out the same high-grade blog posts from you later on as well. In truth, your creative writing abilities has inspired me to get my very own website now 😉
Hi, I do think this is a great blog. I stumbledupon it 😉 I’m going to come back yet again since I saved as a favorite it. Money and freedom is the best way to change, may you be rich and continue to help others.
This site truly has all the information and facts I needed about this subject and didn’t know who to ask.
I would like to thank you for the efforts you have put in writing this site. I’m hoping to view the same high-grade blog posts by you later on as well. In truth, your creative writing abilities has motivated me to get my own, personal website now 😉
When I initially left a comment I seem to have clicked on the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I get 4 emails with the exact same comment. There has to be a means you can remove me from that service? Thanks!
I love it whenever people get together and share opinions. Great blog, stick with it!
Excellent article. I absolutely appreciate this website. Keep it up!
Hello there! This blog post could not be written any better! Looking through this post reminds me of my previous roommate! He always kept talking about this. I am going to send this article to him. Pretty sure he’ll have a good read. Many thanks for sharing!
I could not refrain from commenting. Perfectly written!
I must thank you for the efforts you have put in penning this site. I’m hoping to check out the same high-grade blog posts by you in the future as well. In truth, your creative writing abilities has motivated me to get my very own website now 😉
You made some really good points there. I checked on the internet to find out more about the issue and found most individuals will go along with your views on this site.
Your style is so unique compared to other people I have read stuff from. Many thanks for posting when you have the opportunity, Guess I’ll just bookmark this site.
Great information. Lucky me I came across your website by chance (stumbleupon). I have book-marked it for later!
An impressive share! I have just forwarded this onto a friend who had been conducting a little homework on this. And he in fact bought me dinner because I stumbled upon it for him… lol. So allow me to reword this…. Thanks for the meal!! But yeah, thanks for spending some time to talk about this issue here on your site.
Hello there, I think your blog could be having web browser compatibility problems. Whenever I take a look at your website in Safari, it looks fine however, when opening in I.E., it’s got some overlapping issues. I merely wanted to provide you with a quick heads up! Aside from that, excellent website!
I really like looking through a post that will make men and women think. Also, thank you for allowing for me to comment!
This is a very good tip particularly to those new to the blogosphere. Short but very precise info… Thanks for sharing this one. A must read article!