A search engine for better disease diagnosis and treatment 

Hamid Tizhoosh was looking for a new idea, a fresh start, when he began talking to doctors about how they do their jobs and how they might do them better.

Six months into his consultations, with his engineering lab at the University of Waterloo reduced to a one-man show by a failed artificial intelligence (AI) startup, he heard something that almost floored him.

Pathologists in the 21st century still rely on atlases — books of images from biopsy samples — and flip through them for potential matches to help diagnose new cases.

Really? Books of old images? That was it, the spark that sent the systems design engineering professor roaring down a productive new research path.

“They were using a very Stone Age type of search,” he recalls. “When I learned that, I said, ‘For heaven’s sake, we should do this automatically. It is image search. Computers can do it.’”

Seven years later, Tizhoosh has turned that basic concept into new technology he hopes will revolutionize health care by giving doctors a simple, powerful tool to help diagnose, treat and research disease via search in large medical image archives.

Partnering with industry to secure $3.14 million

To help realize that goal when his early work started showing promise, he approached a local company, Huron Digital Pathology of St. Jacobs, for support in return for commercialization rights.

Huron later became the industrial partner in a consortium led by Tizhoosh and researchers in his Laboratory for Knowledge Inference in Medical Image Analysis (KIMIA Lab) that secured $3.14 million in funding through the Ontario Research Fund: Research Excellence program in 2018.

The technology at the heart of the project is essentially a specialized search engine that allows doctors to comb archives of digital images of tissue samples for the closest matches to new cases.

The original images contain massive amounts of digital data, so search is only possible because Tizhoosh came up with a way of using AI to identify key features and convert them into bunches of barcodes.

That reduces the size of images to a tiny fraction of the originals and indexes them, enabling the search engine to find matches in archives of millions of images in a split second using ordinary computers.

“We designed the search from the beginning to be super-efficient, to do this without heavy-duty computational power,” Tizhoosh says.

By finding similar images, the search engine instantly connects doctors to a treasure trove of information on old cases — the diagnosis report, the treatment plan, the eventual outcome — that is now just sitting in archives.

Waterloo tech will modernize world's largest tissue archive

A hint of its potential came last fall when the KIMIA Lab was selected by the World Health Organization to contribute to a global research project on cancer categorization using its image-retrieval technology. 

Around the same time, the United States military signed on as Huron’s first paying customer to modernize the Joint Pathology Centre, home to the largest collection of preserved tissue samples in the world.

That deal put the project two years ahead of schedule in terms of commercialization, but Tizhoosh is convinced they have still only scratched the surface.

He looks forward to the day the system is used everywhere — including areas of the developing world where pathologists are especially scarce — to virtually eliminate diagnostic error, personalize treatment plans and fuel drug development.

“I think we’re only at the very beginning, to be honest,” Tizhoosh says. “I believe this is disruptive technology that will eventually touch every area of the medical field.

“Even we don’t know what the full impact of image search in medical archives will be. Every time we talk to doctors, they give us new ideas.”